May 09 09:04:24.352348 np0000173033 systemd[1]: Started devstack@neutron-ovn-maintenance-worker.service - Devstack devstack@neutron-ovn-maintenance-worker.service.
May 09 09:04:24.670619 np0000173033 neutron-ovn-maintenance-worker[105597]: /opt/stack/data/venv/lib/python3.12/site-packages/oslo_concurrency/lockutils.py:41: EventletDeprecationWarning:
May 09 09:04:24.670619 np0000173033 neutron-ovn-maintenance-worker[105597]: Eventlet is deprecated. It is currently being maintained in bugfix mode, and
May 09 09:04:24.670619 np0000173033 neutron-ovn-maintenance-worker[105597]: we strongly recommend against using it for new projects.
May 09 09:04:24.670619 np0000173033 neutron-ovn-maintenance-worker[105597]: If you are already using Eventlet, we recommend migrating to a different
May 09 09:04:24.670619 np0000173033 neutron-ovn-maintenance-worker[105597]: framework.  For more detail see
May 09 09:04:24.670619 np0000173033 neutron-ovn-maintenance-worker[105597]: https://eventlet.readthedocs.io/en/latest/asyncio/migration.html
May 09 09:04:24.670619 np0000173033 neutron-ovn-maintenance-worker[105597]:   import eventlet
May 09 09:04:25.745866 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.common.config [-] Logging enabled!
May 09 09:04:25.746511 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-ovn-maintenance-worker version 28.1.0.dev201
May 09 09:04:25.746511 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-ovn-maintenance-worker --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini {{(pid=105597) setup_logging /opt/stack/neutron/neutron/common/config.py:124}}
May 09 09:04:25.746511 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.server.ovn_maintenance [-] OVN maintenance process starting...
May 09 09:04:25.746511 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "plugin-directory" by "neutron_lib.plugins.directory._create_plugin_directory" {{(pid=105597) inner /opt/stack/data/venv/lib/python3.12/site-packages/oslo_concurrency/lockutils.py:506}}
May 09 09:04:25.746511 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG oslo_concurrency.lockutils [-] Lock "plugin-directory" acquired by "neutron_lib.plugins.directory._create_plugin_directory" :: waited 0.000s {{(pid=105597) inner /opt/stack/data/venv/lib/python3.12/site-packages/oslo_concurrency/lockutils.py:519}}
May 09 09:04:25.746833 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG oslo_concurrency.lockutils [-] Lock "plugin-directory" "released" by "neutron_lib.plugins.directory._create_plugin_directory" :: held 0.000s {{(pid=105597) inner /opt/stack/data/venv/lib/python3.12/site-packages/oslo_concurrency/lockutils.py:538}}
May 09 09:04:25.746833 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "manager" by "neutron.manager.NeutronManager._create_instance" {{(pid=105597) inner /opt/stack/data/venv/lib/python3.12/site-packages/oslo_concurrency/lockutils.py:506}}
May 09 09:04:25.746833 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG oslo_concurrency.lockutils [-] Lock "manager" acquired by "neutron.manager.NeutronManager._create_instance" :: waited 0.000s {{(pid=105597) inner /opt/stack/data/venv/lib/python3.12/site-packages/oslo_concurrency/lockutils.py:519}}
May 09 09:04:25.746833 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.manager [-] Loading core plugin: ml2
May 09 09:04:26.019946 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.securitygroup.SecurityGroup'>> rbac-policy before_create 55550000, False {{(pid=105597) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:26.019946 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.securitygroup.SecurityGroup'>> rbac-policy before_update 55550000, False {{(pid=105597) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:26.020150 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.securitygroup.SecurityGroup'>> rbac-policy before_delete 55550000, False {{(pid=105597) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:26.020701 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.address_group.AddressGroup'>> rbac-policy before_create 55550000, False {{(pid=105597) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:26.020800 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.address_group.AddressGroup'>> rbac-policy before_update 55550000, False {{(pid=105597) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:26.020903 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.address_group.AddressGroup'>> rbac-policy before_delete 55550000, False {{(pid=105597) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:26.054889 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.network.Network'>> rbac-policy before_create 55550000, False {{(pid=105597) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:26.055057 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.network.Network'>> rbac-policy before_update 55550000, False {{(pid=105597) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:26.055057 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.network.Network'>> rbac-policy before_delete 55550000, False {{(pid=105597) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:26.167105 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.qos.policy.QosPolicy'>> rbac-policy before_create 55550000, False {{(pid=105597) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:26.167199 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.qos.policy.QosPolicy'>> rbac-policy before_update 55550000, False {{(pid=105597) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:26.167226 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.qos.policy.QosPolicy'>> rbac-policy before_delete 55550000, False {{(pid=105597) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:26.186212 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.subnetpool.SubnetPool'>> rbac-policy before_create 55550000, False {{(pid=105597) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:26.186287 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.subnetpool.SubnetPool'>> rbac-policy before_update 55550000, False {{(pid=105597) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:26.186474 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.subnetpool.SubnetPool'>> rbac-policy before_delete 55550000, False {{(pid=105597) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:26.186763 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.address_scope.AddressScope'>> rbac-policy before_create 55550000, False {{(pid=105597) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:26.186836 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.address_scope.AddressScope'>> rbac-policy before_update 55550000, False {{(pid=105597) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:26.186895 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.address_scope.AddressScope'>> rbac-policy before_delete 55550000, False {{(pid=105597) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:26.271177 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <function _update_segment_host_mapping_for_agent at 0x73c887cd2660> agent after_create 55550000, False {{(pid=105597) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:26.271357 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <function _update_segment_host_mapping_for_agent at 0x73c887cd2660> agent after_update 55550000, False {{(pid=105597) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:26.271357 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <function _add_segment_host_mapping_for_segment at 0x73c887cd2700> segment precommit_create 55550000, False {{(pid=105597) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:26.271357 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <function _delete_segments_for_network at 0x73c887cd27a0> network precommit_delete 55550000, False {{(pid=105597) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:26.271588 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <function _prevent_segment_delete_with_port_bound at 0x73c887cd39c0> segment before_delete 55550000, False {{(pid=105597) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:26.323726 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method External_net_db_mixin._before_network_delete_handler of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x73c8891afe60>> network before_delete 55550000, False {{(pid=105597) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:26.323871 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <function DVRDbMixin._delete_mac_associated_with_agent at 0x73c887cc20c0> agent before_delete 55550000, False {{(pid=105597) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:26.323932 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method SecurityGroupDbMixin._ensure_default_security_group_handler of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x73c8891afe60>> network before_create 55550000, False {{(pid=105597) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:26.323990 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method SecurityGroupDbMixin._ensure_default_security_group_handler of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x73c8891afe60>> port before_create 55550000, False {{(pid=105597) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:26.324067 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method SecurityGroupDbMixin._ensure_default_security_group_handler of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x73c8891afe60>> port before_update 55550000, False {{(pid=105597) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:26.324067 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Ml2Plugin._handle_segment_change of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x73c8891afe60>> segment precommit_create 55550000, False {{(pid=105597) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:26.324174 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Ml2Plugin._handle_segment_change of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x73c8891afe60>> segment precommit_delete 55550000, False {{(pid=105597) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:26.324245 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Ml2Plugin._handle_segment_change of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x73c8891afe60>> segment after_create 55550000, False {{(pid=105597) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:26.324245 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Ml2Plugin._handle_segment_change of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x73c8891afe60>> segment after_delete 55550000, False {{(pid=105597) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:26.324353 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Ml2Plugin._network_delete_after_delete_handler of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x73c8891afe60>> network after_delete 55550000, False {{(pid=105597) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:26.324353 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Ml2Plugin._network_delete_precommit_handler of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x73c8891afe60>> network precommit_delete 0, False {{(pid=105597) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:26.324468 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Ml2Plugin._port_provisioned of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x73c8891afe60>> port provisioning_complete 55550000, False {{(pid=105597) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:26.324468 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method External_net_db_mixin._process_ext_policy_create of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x73c8891afe60>> rbac-policy before_create 55550000, False {{(pid=105597) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:26.324577 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method External_net_db_mixin._process_ext_policy_delete of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x73c8891afe60>> rbac-policy after_delete 55550000, False {{(pid=105597) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:26.324577 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Ml2Plugin._retry_binding_revived_agents of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x73c8891afe60>> agent after_update 55550000, False {{(pid=105597) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:26.324694 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Ml2Plugin._subnet_delete_after_delete_handler of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x73c8891afe60>> subnet after_delete 55550000, False {{(pid=105597) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:26.324694 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Ml2Plugin._subnet_delete_precommit_handler of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x73c8891afe60>> subnet precommit_delete 0, False {{(pid=105597) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:26.324839 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method External_net_db_mixin._validate_ext_not_in_use_by_project of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x73c8891afe60>> rbac-policy before_update 55550000, False {{(pid=105597) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:26.324839 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method External_net_db_mixin._validate_ext_not_in_use_by_project of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x73c8891afe60>> rbac-policy before_delete 55550000, False {{(pid=105597) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:26.324958 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method DhcpAgentSchedulerDbMixin.auto_schedule_new_network_segments of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x73c8891afe60>> segment_host_mapping after_create 55550000, False {{(pid=105597) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:26.324958 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Ml2Plugin.delete_agent_notified of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x73c8891afe60>> agent after_delete 55550000, False {{(pid=105597) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:26.325076 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method NeutronDbPluginV2.validate_network_rbac_policy_change of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x73c8891afe60>> rbac-policy before_create 55550000, False {{(pid=105597) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:26.325135 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method NeutronDbPluginV2.validate_network_rbac_policy_change of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x73c8891afe60>> rbac-policy before_update 55550000, False {{(pid=105597) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:26.325192 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method NeutronDbPluginV2.validate_network_rbac_policy_change of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x73c8891afe60>> rbac-policy before_delete 55550000, False {{(pid=105597) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:26.325519 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.quota.resource_registry [-] Tracking information for resource: network configured {{(pid=105597) set_tracked_resource /opt/stack/neutron/neutron/quota/resource_registry.py:203}}
May 09 09:04:26.325599 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.quota.resource_registry [-] Tracking information for resource: port configured {{(pid=105597) set_tracked_resource /opt/stack/neutron/neutron/quota/resource_registry.py:203}}
May 09 09:04:26.325599 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.quota.resource_registry [-] Tracking information for resource: subnet configured {{(pid=105597) set_tracked_resource /opt/stack/neutron/neutron/quota/resource_registry.py:203}}
May 09 09:04:26.325708 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.quota.resource_registry [-] Tracking information for resource: subnetpool configured {{(pid=105597) set_tracked_resource /opt/stack/neutron/neutron/quota/resource_registry.py:203}}
May 09 09:04:26.325708 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.quota.resource_registry [-] Tracking information for resource: security_group configured {{(pid=105597) set_tracked_resource /opt/stack/neutron/neutron/quota/resource_registry.py:203}}
May 09 09:04:26.325855 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.quota.resource_registry [-] Tracking information for resource: security_group_rule configured {{(pid=105597) set_tracked_resource /opt/stack/neutron/neutron/quota/resource_registry.py:203}}
May 09 09:04:26.326065 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.plugins.ml2.managers [-] Configured type driver names: ['local', 'flat', 'vlan', 'geneve']
May 09 09:04:26.326756 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.plugins.ml2.drivers.type_flat [-] Allowable flat physical_network names: ['public']
May 09 09:04:26.327349 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.plugins.ml2.drivers.type_local [-] ML2 LocalTypeDriver initialization complete
May 09 09:04:26.327856 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.plugins.ml2.drivers.type_vlan [-] Network VLAN ranges: OrderedDict({'public': [(1, 4094)]})
May 09 09:04:26.327932 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.plugins.ml2.managers [-] Loaded type driver names: ['flat', 'geneve', 'local', 'vlan']
May 09 09:04:26.328071 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.plugins.ml2.managers [-] Registered types: dict_keys(['flat', 'geneve', 'local', 'vlan'])
May 09 09:04:26.328145 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.plugins.ml2.managers [-] Project network_types: ['geneve']
May 09 09:04:26.328341 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.plugins.ml2.managers [-] Configured extension driver names: ['port_security', 'qos']
May 09 09:04:26.328762 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.plugins.ml2.managers [-] Loaded extension driver names: ['port_security', 'qos']
May 09 09:04:26.328839 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.plugins.ml2.managers [-] Registered extension drivers: ['port_security', 'qos']
May 09 09:04:26.328891 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.plugins.ml2.managers [-] Configured mechanism driver names: ['ovn']
May 09 09:04:26.364870 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.plugins.ml2.managers [-] Loaded mechanism driver names: ['ovn']
May 09 09:04:26.364870 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.plugins.ml2.managers [-] Registered mechanism drivers: ['ovn']
May 09 09:04:26.365135 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.plugins.ml2.managers [-] No mechanism drivers provide segment reachability information for agent scheduling.
May 09 09:04:26.365517 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Notifier._send_nova_notification of <neutron.notifiers.nova.Notifier object at 0x73c88766b560>> floatingip before_response 55550000, False {{(pid=105597) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:26.365590 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Notifier._send_nova_notification of <neutron.notifiers.nova.Notifier object at 0x73c88766b560>> port before_response 55550000, False {{(pid=105597) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:26.383463 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.plugins.ml2.managers [-] Initializing driver for type 'flat'
May 09 09:04:26.383597 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.plugins.ml2.drivers.type_flat [-] ML2 FlatTypeDriver initialization complete
May 09 09:04:26.383597 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.plugins.ml2.managers [-] Initializing driver for type 'geneve'
May 09 09:04:26.383696 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.plugins.ml2.drivers.type_tunnel [-] geneve ID ranges: [(1, 65536)]
May 09 09:04:26.428449 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO dbcounter [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Registered counter for database neutron
May 09 09:04:26.443416 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG oslo_db.sqlalchemy.engines [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=105597) _check_effective_sql_mode /opt/stack/data/venv/lib/python3.12/site-packages/oslo_db/sqlalchemy/engines.py:325}}
May 09 09:04:26.856409 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG dbcounter [-] [105597] Writer thread running {{(pid=105597) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:102}}
May 09 09:04:27.065429 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.plugins.ml2.managers [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Initializing driver for type 'local'
May 09 09:04:27.065606 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.plugins.ml2.managers [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Initializing driver for type 'vlan'
May 09 09:04:27.186374 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.plugins.ml2.drivers.type_vlan [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] VlanTypeDriver initialization complete
May 09 09:04:27.186557 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.plugins.ml2.managers [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Initializing extension driver 'port_security'
May 09 09:04:27.186557 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.plugins.ml2.extensions.port_security [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] PortSecurityExtensionDriver initialization complete
May 09 09:04:27.186677 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.plugins.ml2.managers [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Initializing extension driver 'qos'
May 09 09:04:27.186677 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.plugins.ml2.extensions.qos [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] QosExtensionDriver initialization complete {{(pid=105597) initialize /opt/stack/neutron/neutron/plugins/ml2/extensions/qos.py:31}}
May 09 09:04:27.186832 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.plugins.ml2.managers [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Initializing mechanism driver 'ovn'
May 09 09:04:27.186832 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Starting OVNMechanismDriver
May 09 09:04:27.188315 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron_lib.callbacks.manager [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Subscribe: <bound method OVNMechanismDriver.pre_fork_initialize of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x73c887a0aea0>> process before_spawn 55550000, False {{(pid=105597) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:27.188446 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron_lib.callbacks.manager [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Subscribe: <bound method OVNMechanismDriver.post_fork_initialize of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x73c887a0aea0>> process after_init 55550000, True {{(pid=105597) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:27.188514 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron_lib.callbacks.manager [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Subscribe: <bound method OVNMechanismDriver._add_segment_host_mapping_for_segment of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x73c887a0aea0>> segment after_create 55550000, False {{(pid=105597) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:27.188577 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron_lib.callbacks.manager [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Subscribe: <bound method OVNMechanismDriver.create_segment_provnet_port of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x73c887a0aea0>> segment after_create 55550000, False {{(pid=105597) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:27.188663 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron_lib.callbacks.manager [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Subscribe: <bound method OVNMechanismDriver.delete_segment_provnet_port of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x73c887a0aea0>> segment after_delete 55550000, False {{(pid=105597) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:27.188738 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron_lib.callbacks.manager [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Subscribe: <bound method OVNMechanismDriver._validate_allowed_address_pairs of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x73c887a0aea0>> allowed_address_pair before_create 55550000, False {{(pid=105597) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:27.188838 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron_lib.callbacks.manager [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Subscribe: <bound method OVNMechanismDriver._create_security_group_precommit of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x73c887a0aea0>> security_group precommit_create 55550000, False {{(pid=105597) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:27.188925 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron_lib.callbacks.manager [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Subscribe: <bound method OVNMechanismDriver._update_security_group of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x73c887a0aea0>> security_group after_update 55550000, False {{(pid=105597) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:27.188987 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron_lib.callbacks.manager [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Subscribe: <bound method OVNMechanismDriver._create_security_group of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x73c887a0aea0>> security_group after_create 55550000, False {{(pid=105597) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:27.189081 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron_lib.callbacks.manager [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Subscribe: <bound method OVNMechanismDriver._delete_security_group_precommit of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x73c887a0aea0>> security_group precommit_delete 55550000, False {{(pid=105597) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:27.189158 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron_lib.callbacks.manager [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Subscribe: <bound method OVNMechanismDriver._delete_security_group of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x73c887a0aea0>> security_group after_delete 55550000, False {{(pid=105597) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:27.189251 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron_lib.callbacks.manager [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Subscribe: <bound method OVNMechanismDriver._create_sg_rule_precommit of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x73c887a0aea0>> security_group_rule precommit_create 55550000, False {{(pid=105597) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:27.189596 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron_lib.callbacks.manager [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Subscribe: <bound method OVNMechanismDriver._process_sg_rule_notification of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x73c887a0aea0>> security_group_rule after_create 55550000, False {{(pid=105597) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:27.189596 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron_lib.callbacks.manager [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Subscribe: <bound method OVNMechanismDriver._process_sg_rule_notification of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x73c887a0aea0>> security_group_rule before_delete 55550000, False {{(pid=105597) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:27.189596 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron_lib.callbacks.manager [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Subscribe: <bound method OVNMechanismDriver._process_ag_notification of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x73c887a0aea0>> address_group after_create 55550000, False {{(pid=105597) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:27.189596 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron_lib.callbacks.manager [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Subscribe: <bound method OVNMechanismDriver._process_ag_notification of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x73c887a0aea0>> address_group after_update 55550000, False {{(pid=105597) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:27.189890 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron_lib.callbacks.manager [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Subscribe: <bound method OVNMechanismDriver._process_ag_notification of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x73c887a0aea0>> address_group after_delete 55550000, False {{(pid=105597) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:27.189890 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron_lib.callbacks.manager [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Subscribe: <bound method DriverBase._register of <neutron.services.qos.drivers.ovn.driver.OVNQosDriver object at 0x73c88768ee70>> qos_plugin after_init 55550000, False {{(pid=105597) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:27.190021 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron_lib.callbacks.manager [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Subscribe: <bound method OVNTrunkDriver.register of <neutron.services.trunk.drivers.ovn.trunk_driver.OVNTrunkDriver object at 0x73c8878efc20>> trunk_plugin after_init 55550000, False {{(pid=105597) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:27.190134 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron_lib.callbacks.manager [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Subscribe: <bound method DriverBase._register of <neutron.services.logapi.drivers.ovn.driver.OVNDriver object at 0x73c886c358b0>> logging-plugin after_init 55550000, False {{(pid=105597) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:27.190629 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.services.logapi.drivers.ovn.driver [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] OVN logging driver registered
May 09 09:04:27.191785 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.plugins.ml2.plugin [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] neutron.plugins.ml2.plugin.Ml2Plugin method _start_rpc_notifiers called with arguments () {} {{(pid=105597) wrapper /opt/stack/data/venv/lib/python3.12/site-packages/oslo_log/helpers.py:77}}
May 09 09:04:27.191988 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron_lib.callbacks.manager [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x73c886c35b20>> port after_create 55550000, False {{(pid=105597) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:27.192058 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron_lib.callbacks.manager [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x73c886c35b20>> port after_update 55550000, False {{(pid=105597) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:27.192137 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron_lib.callbacks.manager [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x73c886c35b20>> port after_delete 55550000, False {{(pid=105597) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:27.192228 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron_lib.callbacks.manager [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x73c886c35a90>> subnet after_create 55550000, False {{(pid=105597) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:27.192306 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron_lib.callbacks.manager [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x73c886c35a90>> subnet after_update 55550000, False {{(pid=105597) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:27.192366 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron_lib.callbacks.manager [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x73c886c35a90>> subnet after_delete 55550000, False {{(pid=105597) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:27.192452 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron_lib.callbacks.manager [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x73c886c35a30>> network after_create 55550000, False {{(pid=105597) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:27.192525 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron_lib.callbacks.manager [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x73c886c35a30>> network after_update 55550000, False {{(pid=105597) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:27.192610 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron_lib.callbacks.manager [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x73c886c35a30>> network after_delete 55550000, False {{(pid=105597) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:27.192692 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron_lib.callbacks.manager [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x73c886c35a00>> security_group after_create 55550000, False {{(pid=105597) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:27.192813 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron_lib.callbacks.manager [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x73c886c35a00>> security_group after_update 55550000, False {{(pid=105597) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:27.192879 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron_lib.callbacks.manager [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x73c886c35a00>> security_group after_delete 55550000, False {{(pid=105597) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:27.192938 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron_lib.callbacks.manager [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x73c886c35a60>> security_group_rule after_create 55550000, False {{(pid=105597) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:27.192999 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron_lib.callbacks.manager [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x73c886c35a60>> security_group_rule after_update 55550000, False {{(pid=105597) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:27.193068 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron_lib.callbacks.manager [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x73c886c35a60>> security_group_rule after_delete 55550000, False {{(pid=105597) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:27.193165 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron_lib.callbacks.manager [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x73c886c35970>> address_group after_create 55550000, False {{(pid=105597) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:27.193241 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron_lib.callbacks.manager [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x73c886c35970>> address_group after_update 55550000, False {{(pid=105597) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:27.193302 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron_lib.callbacks.manager [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x73c886c35970>> address_group after_delete 55550000, False {{(pid=105597) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:27.193362 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.plugins.ml2.ovo_rpc [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] ML2 OVO RPC backend initialized. {{(pid=105597) __init__ /opt/stack/neutron/neutron/plugins/ml2/ovo_rpc.py:117}}
May 09 09:04:27.194053 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.quota [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loaded quota_driver: <neutron.db.quota.driver_nolock.DbQuotaNoLockDriver object at 0x73c886c35940>.
May 09 09:04:27.194121 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.plugins.ml2.plugin [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Modular L2 Plugin initialization complete
May 09 09:04:27.194245 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.manager [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading services supported by the core plugin {{(pid=105597) _load_services_from_core_plugin /opt/stack/neutron/neutron/manager.py:170}}
May 09 09:04:27.194311 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.plugins.ml2.managers [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Got port-security extension from driver 'port_security'
May 09 09:04:27.194572 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.manager [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading service plugins: ['ovn-router', 'auto_allocate', 'tag', 'timestamp', 'network_ip_availability', 'flavors', 'revisions'] {{(pid=105597) _load_service_plugins /opt/stack/neutron/neutron/manager.py:195}}
May 09 09:04:27.194641 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.manager [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading Plugin: ovn-router
May 09 09:04:27.204771 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron_lib.callbacks.manager [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Subscribe: <bound method ExtraGatewaysDbOnlyMixin._delete_router_remove_external_gateways of <neutron.services.ovn_l3.plugin.OVNL3RouterPlugin object at 0x73c886c372c0>> router before_delete 55550000, False {{(pid=105597) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:27.204771 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron_lib.callbacks.manager [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Subscribe: <function L3RpcNotifierMixin._notify_gateway_port_ip_changed at 0x73c887b1fec0> port after_update 55550000, False {{(pid=105597) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:27.204771 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron_lib.callbacks.manager [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Subscribe: <function L3RpcNotifierMixin._notify_routers_callback at 0x73c887b24040> port after_delete 55550000, False {{(pid=105597) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:27.204771 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron_lib.callbacks.manager [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Subscribe: <function L3RpcNotifierMixin._notify_subnet_gateway_ip_update at 0x73c887b1ff60> subnet after_update 55550000, False {{(pid=105597) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:27.204771 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron_lib.callbacks.manager [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Subscribe: <function L3RpcNotifierMixin._notify_subnetpool_address_scope_update at 0x73c887b1fe20> subnetpool_address_scope after_update 55550000, False {{(pid=105597) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:27.205002 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron_lib.callbacks.manager [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Subscribe: <function OVNL3RouterPlugin._port_update at 0x73c887093920> port before_update 55550000, False {{(pid=105597) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:27.205002 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron_lib.callbacks.manager [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Subscribe: <function OVNL3RouterPlugin._port_update at 0x73c887093920> port after_update 55550000, False {{(pid=105597) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:27.205002 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron_lib.callbacks.manager [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Subscribe: <bound method L3_NAT_dbonly_mixin._precommit_delete_port_callback of <neutron.services.ovn_l3.plugin.OVNL3RouterPlugin object at 0x73c886c372c0>> port precommit_delete 55550000, False {{(pid=105597) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:27.205002 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron_lib.callbacks.manager [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Subscribe: <function L3_NAT_dbonly_mixin._prevent_internal_ip_change_for_fip at 0x73c887b176a0> port before_update 55550000, False {{(pid=105597) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:27.205002 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron_lib.callbacks.manager [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Subscribe: <function L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback at 0x73c887b17600> port before_delete 55550000, False {{(pid=105597) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:27.205323 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron_lib.callbacks.manager [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Subscribe: <bound method RouterAvailabilityZoneMixin._process_az_request of <neutron.services.ovn_l3.plugin.OVNL3RouterPlugin object at 0x73c886c372c0>> router precommit_create 55550000, False {{(pid=105597) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:27.205323 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron_lib.callbacks.manager [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Subscribe: <bound method ExtraGatewaysDbOnlyMixin._process_bfd_ecmp_create of <neutron.services.ovn_l3.plugin.OVNL3RouterPlugin object at 0x73c886c372c0>> router precommit_create 55550000, False {{(pid=105597) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:27.205323 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron_lib.callbacks.manager [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Subscribe: <bound method ExtraGatewaysDbOnlyMixin._process_bfd_ecmp_update of <neutron.services.ovn_l3.plugin.OVNL3RouterPlugin object at 0x73c886c372c0>> router precommit_update 55550000, False {{(pid=105597) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:27.205323 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron_lib.callbacks.manager [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Subscribe: <function OVNL3RouterPlugin._subnet_update at 0x73c887093880> subnet after_update 55550000, False {{(pid=105597) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:27.205323 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.quota.resource_registry [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Tracking information for resource: router configured {{(pid=105597) set_tracked_resource /opt/stack/neutron/neutron/quota/resource_registry.py:203}}
May 09 09:04:27.205533 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.quota.resource_registry [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Tracking information for resource: floatingip configured {{(pid=105597) set_tracked_resource /opt/stack/neutron/neutron/quota/resource_registry.py:203}}
May 09 09:04:27.205533 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.services.ovn_l3.plugin [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Starting OVNL3RouterPlugin
May 09 09:04:27.205533 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron_lib.callbacks.manager [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Subscribe: <bound method OVNPortForwarding.register of <neutron.services.portforwarding.drivers.ovn.driver.OVNPortForwarding object at 0x73c88705c4a0>> port_forwarding_plugin after_init 55550000, False {{(pid=105597) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:27.205533 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron_lib.callbacks.manager [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Subscribe: <bound method DriverController._check_router_request of <neutron.services.ovn_l3.service_providers.driver_controller.DriverController object at 0x73c886c36960>> router before_create 55550100, False {{(pid=105597) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:27.205533 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron_lib.callbacks.manager [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Subscribe: <bound method DriverController._clear_router_provider of <neutron.services.ovn_l3.service_providers.driver_controller.DriverController object at 0x73c886c36960>> router precommit_delete 55550100, False {{(pid=105597) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:27.205797 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron_lib.callbacks.manager [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Subscribe: <bound method DriverController._set_router_provider of <neutron.services.ovn_l3.service_providers.driver_controller.DriverController object at 0x73c886c36960>> router precommit_create 55550100, False {{(pid=105597) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:27.205797 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron_lib.callbacks.manager [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Subscribe: <bound method DriverController._update_router_provider of <neutron.services.ovn_l3.service_providers.driver_controller.DriverController object at 0x73c886c36960>> router precommit_update 55550100, False {{(pid=105597) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:27.207053 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.services.provider_configuration [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Service providers = [] {{(pid=105597) parse_service_provider_opt /opt/stack/neutron/neutron/services/provider_configuration.py:170}}
May 09 09:04:27.207140 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.db.servicetype_db [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Adding provider configuration for service L3_ROUTER_NAT {{(pid=105597) add_provider_configuration /opt/stack/neutron/neutron/db/servicetype_db.py:45}}
May 09 09:04:27.208247 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron_lib.callbacks.manager [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Subscribe: <bound method OVN_L3_HA_db_mixin._precommit_router_create of <neutron.services.ovn_l3.service_providers.ovn.OvnDriver object at 0x73c886c373b0>> router precommit_create 55549900, False {{(pid=105597) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:27.208321 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron_lib.callbacks.manager [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Subscribe: <bound method OvnDriver._process_add_router_interface of <neutron.services.ovn_l3.service_providers.ovn.OvnDriver object at 0x73c886c373b0>> router_interface after_create 55550000, False {{(pid=105597) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:27.208417 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron_lib.callbacks.manager [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Subscribe: <bound method OvnDriver._process_floatingip_create of <neutron.services.ovn_l3.service_providers.ovn.OvnDriver object at 0x73c886c373b0>> floatingip after_create 55550000, False {{(pid=105597) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:27.208502 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron_lib.callbacks.manager [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Subscribe: <bound method OvnDriver._process_floatingip_create_update_precommit of <neutron.services.ovn_l3.service_providers.ovn.OvnDriver object at 0x73c886c373b0>> floatingip precommit_create 55550000, False {{(pid=105597) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:27.208584 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron_lib.callbacks.manager [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Subscribe: <bound method OvnDriver._process_floatingip_create_update_precommit of <neutron.services.ovn_l3.service_providers.ovn.OvnDriver object at 0x73c886c373b0>> floatingip precommit_update 55550000, False {{(pid=105597) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:27.208667 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron_lib.callbacks.manager [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Subscribe: <bound method OvnDriver._process_floatingip_delete of <neutron.services.ovn_l3.service_providers.ovn.OvnDriver object at 0x73c886c373b0>> floatingip after_delete 55550000, False {{(pid=105597) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:27.208769 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron_lib.callbacks.manager [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Subscribe: <bound method OvnDriver._process_floatingip_status_update of <neutron.services.ovn_l3.service_providers.ovn.OvnDriver object at 0x73c886c373b0>> floatingip after_status_update 55550000, False {{(pid=105597) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:27.208844 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron_lib.callbacks.manager [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Subscribe: <bound method OvnDriver._process_floatingip_update of <neutron.services.ovn_l3.service_providers.ovn.OvnDriver object at 0x73c886c373b0>> floatingip after_update 55550000, False {{(pid=105597) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:27.208939 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron_lib.callbacks.manager [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Subscribe: <bound method OvnDriver._process_portforwarding_create of <neutron.services.ovn_l3.service_providers.ovn.OvnDriver object at 0x73c886c373b0>> port_forwarding after_create 55550000, False {{(pid=105597) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:27.209025 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron_lib.callbacks.manager [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Subscribe: <bound method OvnDriver._process_remove_router_interface of <neutron.services.ovn_l3.service_providers.ovn.OvnDriver object at 0x73c886c373b0>> router_interface after_delete 55550000, False {{(pid=105597) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:27.209101 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron_lib.callbacks.manager [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Subscribe: <bound method OvnDriver._process_router_create of <neutron.services.ovn_l3.service_providers.ovn.OvnDriver object at 0x73c886c373b0>> router after_create 55550000, False {{(pid=105597) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:27.209182 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron_lib.callbacks.manager [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Subscribe: <bound method OvnDriver._process_router_create_precommit of <neutron.services.ovn_l3.service_providers.ovn.OvnDriver object at 0x73c886c373b0>> router precommit_create 55550000, False {{(pid=105597) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:27.209260 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron_lib.callbacks.manager [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Subscribe: <bound method OvnDriver._process_router_delete of <neutron.services.ovn_l3.service_providers.ovn.OvnDriver object at 0x73c886c373b0>> router after_delete 55550000, False {{(pid=105597) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:27.209343 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron_lib.callbacks.manager [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Subscribe: <bound method OvnDriver._process_router_update of <neutron.services.ovn_l3.service_providers.ovn.OvnDriver object at 0x73c886c373b0>> router after_update 55550000, False {{(pid=105597) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:27.209462 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.services.service_base [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loaded 'neutron.services.ovn_l3.service_providers.ovn.OvnDriver' provider for service L3_ROUTER_NAT {{(pid=105597) load_drivers /opt/stack/neutron/neutron/services/service_base.py:49}}
May 09 09:04:27.209570 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron_lib.callbacks.manager [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Subscribe: <bound method OVNL3RouterPlugin._post_fork_initialize of <neutron.services.ovn_l3.plugin.OVNL3RouterPlugin object at 0x73c886c372c0>> process after_init 55550001, True {{(pid=105597) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:27.209678 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.manager [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Successfully loaded L3_ROUTER_NAT plugin. Description: L3 Router Service Plugin for basic L3 forwarding using OVN {{(pid=105597) _create_and_add_service_plugin /opt/stack/neutron/neutron/manager.py:239}}
May 09 09:04:27.209768 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.manager [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading Plugin: auto_allocate
May 09 09:04:27.210579 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron_lib.callbacks.manager [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Subscribe: <function _ensure_external_network_default_value_callback at 0x73c8870bdbc0> network precommit_update 55550000, False {{(pid=105597) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:27.210653 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron_lib.callbacks.manager [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Subscribe: <function _ensure_external_network_default_value_callback at 0x73c8870bdbc0> network precommit_create 55550000, False {{(pid=105597) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:27.210855 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.manager [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Successfully loaded auto-allocated-topology plugin. Description: Auto Allocated Topology - aka get me a network. {{(pid=105597) _create_and_add_service_plugin /opt/stack/neutron/neutron/manager.py:239}}
May 09 09:04:27.210932 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.manager [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading Plugin: tag
May 09 09:04:27.213577 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.manager [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Successfully loaded TAG plugin. Description: Tag support {{(pid=105597) _create_and_add_service_plugin /opt/stack/neutron/neutron/manager.py:239}}
May 09 09:04:27.213652 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.manager [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading Plugin: timestamp
May 09 09:04:27.214793 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.manager [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Successfully loaded timestamp plugin. Description: Adds timestamps to Neutron resources with standard attributes {{(pid=105597) _create_and_add_service_plugin /opt/stack/neutron/neutron/manager.py:239}}
May 09 09:04:27.214849 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.manager [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading Plugin: network_ip_availability
May 09 09:04:27.216121 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.manager [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Successfully loaded network-ip-availability plugin. Description: Provides IP availability data for each network and subnet. {{(pid=105597) _create_and_add_service_plugin /opt/stack/neutron/neutron/manager.py:239}}
May 09 09:04:27.216196 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.manager [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading Plugin: flavors
May 09 09:04:27.217012 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.manager [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Successfully loaded FLAVORS plugin. Description: Neutron Flavors and Service Profiles manager plugin {{(pid=105597) _create_and_add_service_plugin /opt/stack/neutron/neutron/manager.py:239}}
May 09 09:04:27.217086 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.manager [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading Plugin: revisions
May 09 09:04:27.217897 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.manager [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Successfully loaded revision_plugin plugin. Description: Adds revision numbers to resources. {{(pid=105597) _create_and_add_service_plugin /opt/stack/neutron/neutron/manager.py:239}}
May 09 09:04:27.218008 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG oslo_concurrency.lockutils [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Lock "manager" "released" by "neutron.manager.NeutronManager._create_instance" :: held 1.471s {{(pid=105597) inner /opt/stack/data/venv/lib/python3.12/site-packages/oslo_concurrency/lockutils.py:538}}
May 09 09:04:27.218180 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] get_extension_paths = OrderedDict({'/opt/stack/neutron/neutron/extensions': 1}) {{(pid=105597) get_extensions_path /opt/stack/neutron/neutron/api/extensions.py:640}}
May 09 09:04:27.218262 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Initializing extension manager.
May 09 09:04:27.218538 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading extension file: __init__.py {{(pid=105597) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 09:04:27.218627 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading extension file: __pycache__ {{(pid=105597) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 09:04:27.218703 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading extension file: address_group.py {{(pid=105597) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 09:04:27.218997 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Ext name="Address group" alias="address-group" description="Support address group" updated="2020-07-15T10:00:00-00:00" {{(pid=105597) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 09:04:27.219207 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loaded extension: address-group
May 09 09:04:27.219283 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading extension file: address_scope.py {{(pid=105597) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 09:04:27.219522 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Ext name="Address scope" alias="address-scope" description="Address scopes extension." updated="2015-07-26T10:00:00-00:00" {{(pid=105597) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 09:04:27.219675 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loaded extension: address-scope
May 09 09:04:27.219760 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading extension file: admin_state_down_before_update.py {{(pid=105597) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 09:04:27.219964 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Ext name="Enforce Router's Admin State Down Before Update Extension" alias="router-admin-state-down-before-update" description="Ensure that the admin state of a router is DOWN (admin_state_up=False) before updating the distributed attribute" updated="2019-07-02T15:56:00" {{(pid=105597) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 09:04:27.220102 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Extension router-admin-state-down-before-update not supported by any of loaded plugins
May 09 09:04:27.220173 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading extension file: agent.py {{(pid=105597) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 09:04:27.220428 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Ext name="agent" alias="agent" description="The agent management extension." updated="2013-02-03T10:00:00-00:00" {{(pid=105597) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 09:04:27.220573 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loaded extension: agent
May 09 09:04:27.220647 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading extension file: agent_resources_synced.py {{(pid=105597) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 09:04:27.220851 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Ext name="Agent's Resource View Synced to Placement" alias="agent-resources-synced" description="Stores success/failure of last sync to Placement" updated="2018-12-19T00:00:00-00:00" {{(pid=105597) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 09:04:27.220991 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Extension agent-resources-synced not supported by any of loaded plugins
May 09 09:04:27.221068 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading extension file: agent_sort_key.py {{(pid=105597) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 09:04:27.221246 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Ext name="Agents Sort Key" alias="agent_sort_key" description="Enabling the keyword sort_key for sorting in agents." updated="2013-02-03T10:00:00-00:00" {{(pid=105597) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 09:04:27.221396 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Extension agent_sort_key not supported by any of loaded plugins
May 09 09:04:27.221469 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading extension file: allowedaddresspairs.py {{(pid=105597) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 09:04:27.222056 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Ext name="Allowed Address Pairs" alias="allowed-address-pairs" description="Provides allowed address pairs" updated="2013-07-23T10:00:00-00:00" {{(pid=105597) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 09:04:27.222213 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loaded extension: allowed-address-pairs
May 09 09:04:27.222286 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading extension file: auto_allocated_topology.py {{(pid=105597) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 09:04:27.222566 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Ext name="Auto Allocated Topology Services" alias="auto-allocated-topology" description="Auto Allocated Topology Services." updated="2016-01-01T00:00:00-00:00" {{(pid=105597) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 09:04:27.222712 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loaded extension: auto-allocated-topology
May 09 09:04:27.222803 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading extension file: availability_zone.py {{(pid=105597) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 09:04:27.223027 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Ext name="Availability Zone" alias="availability_zone" description="The availability zone extension." updated="2015-01-01T10:00:00-00:00" {{(pid=105597) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 09:04:27.223166 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loaded extension: availability_zone
May 09 09:04:27.223234 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading extension file: availability_zone_filter.py {{(pid=105597) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 09:04:27.223420 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Ext name="Availability Zone Filter Extension" alias="availability_zone_filter" description="Add filter parameters to AvailabilityZone resource" updated="2018-06-22T10:00:00-00:00" {{(pid=105597) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 09:04:27.223551 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Extension availability_zone_filter not supported by any of loaded plugins
May 09 09:04:27.223622 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading extension file: data_plane_status.py {{(pid=105597) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 09:04:27.223821 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Ext name="Port data plane status extension" alias="data-plane-status" description="Expose status of underlying data plane" updated="2017-01-24T10:00:00-00:00" {{(pid=105597) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 09:04:27.223954 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Extension data-plane-status not supported by any of loaded plugins
May 09 09:04:27.224022 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading extension file: default_subnetpools.py {{(pid=105597) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 09:04:27.224200 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Ext name="Default Subnetpools" alias="default-subnetpools" description="Provides ability to mark and use a subnetpool as the default." updated="2016-02-18T18:00:00-00:00" {{(pid=105597) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 09:04:27.224332 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loaded extension: default-subnetpools
May 09 09:04:27.224400 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading extension file: dhcpagentscheduler.py {{(pid=105597) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 09:04:27.224709 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Ext name="DHCP Agent Scheduler" alias="dhcp_agent_scheduler" description="Schedule networks among dhcp agents" updated="2013-02-07T10:00:00-00:00" {{(pid=105597) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 09:04:27.224864 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loaded extension: dhcp_agent_scheduler
May 09 09:04:27.224936 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading extension file: dns.py {{(pid=105597) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 09:04:27.225115 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Ext name="DNS Integration" alias="dns-integration" description="Provides integration with DNS." updated="2015-08-15T18:00:00-00:00" {{(pid=105597) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 09:04:27.225250 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loaded extension: dns-integration
May 09 09:04:27.225315 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading extension file: dns_domain_ports.py {{(pid=105597) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 09:04:27.225496 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Ext name="dns_domain for ports" alias="dns-domain-ports" description="Allows the DNS domain to be specified for a network port." updated="2017-04-24T10:00:00-00:00" {{(pid=105597) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 09:04:27.225626 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loaded extension: dns-domain-ports
May 09 09:04:27.225701 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading extension file: dns_integration_domain_keywords.py {{(pid=105597) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 09:04:27.225893 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Ext name="DNS domain names with keywords allowed" alias="dns-integration-domain-keywords" description="Allows to use keywords like <project_id>, <project_name>, <user_id> and <user_name> as DNS domain name" updated="2020-06-15T18:00:00-00:00" {{(pid=105597) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 09:04:27.226026 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loaded extension: dns-integration-domain-keywords
May 09 09:04:27.226097 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading extension file: dvr.py {{(pid=105597) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 09:04:27.226298 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Ext name="Distributed Virtual Router" alias="dvr" description="Enables configuration of Distributed Virtual Routers." updated="2014-06-1T10:00:00-00:00" {{(pid=105597) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 09:04:27.226448 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Extension dvr not supported by any of loaded plugins
May 09 09:04:27.226518 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading extension file: ecmp_routes.py {{(pid=105597) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 09:04:27.226692 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Ext name="Equal Cost Multipath Routing" alias="ecmp_routes" description="Allows traffic to reach thesame destination via multiple different links " updated="2020-11-26T18:00:00-00:00" {{(pid=105597) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 09:04:27.226847 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Extension ecmp_routes not supported by any of loaded plugins
May 09 09:04:27.226908 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading extension file: empty_string_filtering.py {{(pid=105597) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 09:04:27.227083 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Ext name="Empty String Filtering Extension" alias="empty-string-filtering" description="Allow filtering by attributes with empty string value" updated="2018-05-01T10:00:00-00:00" {{(pid=105597) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 09:04:27.227214 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loaded extension: empty-string-filtering
May 09 09:04:27.227282 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading extension file: expose_l3_conntrack_helper.py {{(pid=105597) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 09:04:27.227455 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Ext name="Expose CT target rules for conntrack helper" alias="expose-l3-conntrack-helper" description="Expose allow adding CT target rules for conntrack helper" updated="2019-04-04T10:00:00-00:00" {{(pid=105597) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 09:04:27.227585 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Extension expose-l3-conntrack-helper not supported by any of loaded plugins
May 09 09:04:27.227655 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading extension file: expose_port_forwarding_in_fip.py {{(pid=105597) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 09:04:27.227884 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Ext name="Expose Floating IP port forwardings" alias="expose-port-forwarding-in-fip" description="Expose port forwarding fields in floating IP response" updated="2018-06-28T16:00:00-00:00" {{(pid=105597) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 09:04:27.228021 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Extension expose-port-forwarding-in-fip not supported by any of loaded plugins
May 09 09:04:27.228091 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading extension file: external_net.py {{(pid=105597) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 09:04:27.228275 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Ext name="Neutron external network" alias="external-net" description="Adds external network attribute to network resource." updated="2013-01-14T10:00:00-00:00" {{(pid=105597) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 09:04:27.228409 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loaded extension: external-net
May 09 09:04:27.228480 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading extension file: extra_dhcp_opt.py {{(pid=105597) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 09:04:27.228655 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Ext name="Neutron Extra DHCP options" alias="extra_dhcp_opt" description="Extra options configuration for DHCP. For example PXE boot options to DHCP clients can be specified (e.g. tftp-server, server-ip-address, bootfile-name)" updated="2013-03-17T12:00:00-00:00" {{(pid=105597) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 09:04:27.228812 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loaded extension: extra_dhcp_opt
May 09 09:04:27.228886 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading extension file: extraroute.py {{(pid=105597) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 09:04:27.229064 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Ext name="Neutron Extra Route" alias="extraroute" description="Extra routes configuration for L3 router" updated="2013-02-01T10:00:00-00:00" {{(pid=105597) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 09:04:27.229196 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loaded extension: extraroute
May 09 09:04:27.229266 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading extension file: extraroute_atomic.py {{(pid=105597) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 09:04:27.229458 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Ext name="Atomically add/remove extra routes" alias="extraroute-atomic" description="Edit extra routes of a router on server side by atomically adding/removing extra routes" updated="2019-07-10T00:00:00+00:00" {{(pid=105597) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 09:04:27.229593 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loaded extension: extraroute-atomic
May 09 09:04:27.229662 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading extension file: filter_validation.py {{(pid=105597) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 09:04:27.229885 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Ext name="Filter parameters validation" alias="filter-validation" description="Provides validation on filter parameters." updated="2018-07-04T10:00:00-00:00" {{(pid=105597) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 09:04:27.230020 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loaded extension: filter-validation
May 09 09:04:27.230089 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading extension file: fip_pf_description.py {{(pid=105597) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 09:04:27.230270 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Ext name="Floating IP Port Forwarding new attribute description" alias="floating-ip-port-forwarding-description" description="Add a description field to Port Forwarding rules" updated="2019-11-01T10:00:00-00:00" {{(pid=105597) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 09:04:27.230447 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Extension floating-ip-port-forwarding-description not supported by any of loaded plugins
May 09 09:04:27.230551 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading extension file: fip_pf_detail.py {{(pid=105597) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 09:04:27.230691 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Ext name="Floating IP port forwarding detail" alias="floating-ip-port-forwarding-detail" description="Allow list floating ip return more port forwarding data, include'id' and 'internal_port_id'" updated="2022-05-06T10:00:00-00:00" {{(pid=105597) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 09:04:27.230861 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loaded extension: floating-ip-port-forwarding-detail
May 09 09:04:27.230935 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading extension file: fip_pf_port_range.py {{(pid=105597) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 09:04:27.231112 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Ext name="Floating IP Port Forwarding support port ranges" alias="floating-ip-port-forwarding-port-ranges" description="Add support to port ranges" updated="2020-07-01T10:00:00-00:00" {{(pid=105597) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 09:04:27.231241 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Extension floating-ip-port-forwarding-port-ranges not supported by any of loaded plugins
May 09 09:04:27.231310 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading extension file: fip_port_details.py {{(pid=105597) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 09:04:27.231485 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Ext name="Floating IP Port Details Extension" alias="fip-port-details" description="Add port_details attribute to Floating IP resource" updated="2018-04-09T10:00:00-00:00" {{(pid=105597) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 09:04:27.231647 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loaded extension: fip-port-details
May 09 09:04:27.231760 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading extension file: flavors.py {{(pid=105597) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 09:04:27.232218 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Ext name="Neutron Service Flavors" alias="flavors" description="Flavor specification for Neutron advanced services." updated="2015-09-17T10:00:00-00:00" {{(pid=105597) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 09:04:27.232414 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loaded extension: flavors
May 09 09:04:27.232511 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading extension file: floating_ip_port_forwarding.py {{(pid=105597) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 09:04:27.232965 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Ext name="Floating IP Port Forwarding" alias="floating-ip-port-forwarding" description="Allow port forwarding from floating IP port to an internal port" updated="2018-01-19T10:00:00-00:00" {{(pid=105597) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 09:04:27.233203 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Extension floating-ip-port-forwarding not supported by any of loaded plugins
May 09 09:04:27.233317 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading extension file: floatingip_pools.py {{(pid=105597) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 09:04:27.233621 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Ext name="Floating IP Pools Extension" alias="floatingip-pools" description="Provides a floating IP pools API." updated="2018-03-21T10:00:00-00:00" {{(pid=105597) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 09:04:27.233858 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loaded extension: floatingip-pools
May 09 09:04:27.233969 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading extension file: ip_allocation.py {{(pid=105597) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 09:04:27.234233 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Ext name="IP Allocation" alias="ip_allocation" description="IP allocation extension." updated="2016-06-10T23:00:00-00:00" {{(pid=105597) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 09:04:27.234467 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Extension ip_allocation not supported by any of loaded plugins
May 09 09:04:27.234568 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading extension file: ip_substring_port_filtering.py {{(pid=105597) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 09:04:27.234882 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Ext name="IP address substring filtering" alias="ip-substring-filtering" description="Provides IP address substring filtering when listing ports" updated="2017-11-28T09:00:00-00:00" {{(pid=105597) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 09:04:27.235097 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Extension ip-substring-filtering not supported by any of loaded plugins
May 09 09:04:27.235178 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading extension file: l2_adjacency.py {{(pid=105597) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 09:04:27.235424 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Ext name="L2 Adjacency" alias="l2_adjacency" description="Display L2 Adjacency for Neutron Networks." updated="2016-04-12T16:00:00-00:00" {{(pid=105597) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 09:04:27.235612 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Extension l2_adjacency not supported by any of loaded plugins
May 09 09:04:27.235760 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading extension file: l3.py {{(pid=105597) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 09:04:27.236295 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Ext name="Neutron L3 Router" alias="router" description="Router abstraction for basic L3 forwarding between L2 Neutron networks and access to external networks via a NAT gateway." updated="2012-07-20T10:00:00-00:00" {{(pid=105597) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 09:04:27.236491 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loaded extension: router
May 09 09:04:27.236598 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading extension file: l3_conntrack_helper.py {{(pid=105597) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 09:04:27.237321 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Ext name="L3 Conntrack helper" alias="l3-conntrack-helper" description="Allow adding CT target rules for conntrack helper" updated="2019-04-04T10:00:00-00:00" {{(pid=105597) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 09:04:27.237510 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Extension l3-conntrack-helper not supported by any of loaded plugins
May 09 09:04:27.237609 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading extension file: l3_enable_default_route_bfd.py {{(pid=105597) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 09:04:27.237911 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Ext name="Enable Default Route BFD" alias="enable-default-route-bfd" description="Enables configurable BFD behavior for default routes" updated="2023-02-27T00:00:00-00:00" {{(pid=105597) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 09:04:27.238176 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loaded extension: enable-default-route-bfd
May 09 09:04:27.238276 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading extension file: l3_enable_default_route_ecmp.py {{(pid=105597) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 09:04:27.238633 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Ext name="Enable Default Route ECMP" alias="enable-default-route-ecmp" description="Enables configurable ECMP behavior for default routes" updated="2023-02-27T00:00:00-00:00" {{(pid=105597) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 09:04:27.238792 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loaded extension: enable-default-route-ecmp
May 09 09:04:27.238864 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading extension file: l3_ext_gw_mode.py {{(pid=105597) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 09:04:27.239051 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Ext name="Neutron L3 Configurable external gateway mode" alias="ext-gw-mode" description="Extension of the router abstraction for specifying whether SNAT should occur on the external gateway" updated="2013-03-28T10:00:00-00:00" {{(pid=105597) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 09:04:27.239191 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loaded extension: ext-gw-mode
May 09 09:04:27.239305 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading extension file: l3_ext_ha_mode.py {{(pid=105597) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 09:04:27.239535 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Ext name="HA Router extension" alias="l3-ha" description="Adds HA capability to routers." updated="2014-04-26T00:00:00-00:00" {{(pid=105597) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 09:04:27.239706 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loaded extension: l3-ha
May 09 09:04:27.239822 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading extension file: l3_ext_ndp_proxy.py {{(pid=105597) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 09:04:27.240047 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Ext name="Router support to enable NDP proxy" alias="l3-ext-ndp-proxy" description="Router extension to enable ndp proxy" updated="2020-08-25T00:00:00-00:00" {{(pid=105597) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 09:04:27.240219 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Extension l3-ext-ndp-proxy not supported by any of loaded plugins
May 09 09:04:27.240307 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading extension file: l3_extra_gws.py {{(pid=105597) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 09:04:27.240597 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Ext name="Neutron L3 External Gateway Multihoming" alias="external-gateway-multihoming" description="Allow multiple external gateway ports per router" updated="2023-01-18T00:00:00-00:00" {{(pid=105597) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 09:04:27.240731 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loaded extension: external-gateway-multihoming
May 09 09:04:27.240821 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading extension file: l3_flavors.py {{(pid=105597) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 09:04:27.241050 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Ext name="Router Flavor Extension" alias="l3-flavors" description="Flavor support for routers." updated="2016-05-17T00:00:00-00:00" {{(pid=105597) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 09:04:27.241218 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loaded extension: l3-flavors
May 09 09:04:27.241302 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading extension file: l3_ndp_proxy.py {{(pid=105597) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 09:04:27.241622 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Ext name="name" alias="l3-ndp-proxy" description="description" updated="2021-06-29T10:00:00-00:00" {{(pid=105597) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 09:04:27.241808 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Extension l3-ndp-proxy not supported by any of loaded plugins
May 09 09:04:27.241893 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading extension file: l3_port_ip_change_not_allowed.py {{(pid=105597) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 09:04:27.242123 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Ext name="Prevent L3 router ports IP address change extension" alias="l3-port-ip-change-not-allowed" description="Prevent change of IP address for some L3 router ports" updated="2018-10-09T10:00:00-00:00" {{(pid=105597) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 09:04:27.242269 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Extension l3-port-ip-change-not-allowed not supported by any of loaded plugins
May 09 09:04:27.242382 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading extension file: l3agentscheduler.py {{(pid=105597) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 09:04:27.242794 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Ext name="L3 Agent Scheduler" alias="l3_agent_scheduler" description="Schedule routers among l3 agents" updated="2013-02-07T10:00:00-00:00" {{(pid=105597) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 09:04:27.242984 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Extension l3_agent_scheduler not supported by any of loaded plugins
May 09 09:04:27.242984 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading extension file: local_ip.py {{(pid=105597) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 09:04:27.243244 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Ext name="Local IP" alias="local_ip" description="Support Node Local Virtual IP" updated="2021-07-26T10:00:00-00:00" {{(pid=105597) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 09:04:27.243355 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Extension local_ip not supported by any of loaded plugins
May 09 09:04:27.243441 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading extension file: logging.py {{(pid=105597) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 09:04:27.243714 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Ext name="Logging API Extension" alias="logging" description="Provides a logging API for resources such as security group and firewall." updated="2017-01-01T10:00:00-00:00" {{(pid=105597) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 09:04:27.243868 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Extension logging not supported by any of loaded plugins
May 09 09:04:27.243868 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading extension file: metering.py {{(pid=105597) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 09:04:27.244517 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Ext name="Neutron Metering" alias="metering" description="Neutron Metering extension." updated="2013-06-12T10:00:00-00:00" {{(pid=105597) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 09:04:27.244517 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Extension metering not supported by any of loaded plugins
May 09 09:04:27.244517 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading extension file: metering_source_and_destination_fields.py {{(pid=105597) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 09:04:27.244717 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Ext name="Neutron Metering with source and destination filtering" alias="metering_source_and_destination_fields" description="Neutron Metering extension that enables the use of source and destination IP prefixes to create metering label rules." updated="2020-08-21T08:12:00-00:00" {{(pid=105597) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 09:04:27.244810 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Extension metering_source_and_destination_fields not supported by any of loaded plugins
May 09 09:04:27.244893 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading extension file: multiprovidernet.py {{(pid=105597) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 09:04:27.245077 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Ext name="Multi Provider Network" alias="multi-provider" description="Expose mapping of virtual networks to multiple physical networks" updated="2013-06-27T10:00:00-00:00" {{(pid=105597) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 09:04:27.245162 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loaded extension: multi-provider
May 09 09:04:27.245237 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading extension file: netmtu.py {{(pid=105597) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 09:04:27.245404 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Ext name="Network MTU" alias="net-mtu" description="Provides MTU attribute for a network resource." updated="2015-03-25T10:00:00-00:00" {{(pid=105597) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 09:04:27.245532 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loaded extension: net-mtu
May 09 09:04:27.245608 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading extension file: netmtu_writable.py {{(pid=105597) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 09:04:27.245812 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Ext name="Network MTU (writable)" alias="net-mtu-writable" description="Provides a writable MTU attribute for a network resource." updated="2017-07-12T00:00:00-00:00" {{(pid=105597) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 09:04:27.245928 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loaded extension: net-mtu-writable
May 09 09:04:27.245993 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading extension file: network_availability_zone.py {{(pid=105597) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 09:04:27.246194 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Ext name="Network Availability Zone" alias="network_availability_zone" description="Availability zone support for network." updated="2015-01-01T10:00:00-00:00" {{(pid=105597) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 09:04:27.246326 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loaded extension: network_availability_zone
May 09 09:04:27.246484 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading extension file: network_ha.py {{(pid=105597) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 09:04:27.246682 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Ext name="Network HA creation flag" alias="network_ha" description="Network high availability creation flag." updated="2023-04-27T10:00:00-00:00" {{(pid=105597) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 09:04:27.246841 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Extension network_ha not supported by any of loaded plugins
May 09 09:04:27.246910 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading extension file: network_ip_availability.py {{(pid=105597) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 09:04:27.247109 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Ext name="Network IP Availability" alias="network-ip-availability" description="Provides IP availability data for each network and subnet." updated="2015-09-24T00:00:00-00:00" {{(pid=105597) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 09:04:27.247243 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loaded extension: network-ip-availability
May 09 09:04:27.247309 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading extension file: network_ip_availability_details.py {{(pid=105597) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 09:04:27.247514 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Ext name="Network IP Availability Details Extension" alias="network-ip-availability-details" description="Add ip_availability_details attribute to network-ip-availabilities resource." updated="2025-10-31T00:00:00-00:00" {{(pid=105597) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 09:04:27.247648 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loaded extension: network-ip-availability-details
May 09 09:04:27.247713 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading extension file: network_segment_range.py {{(pid=105597) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 09:04:27.247985 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Ext name="Neutron Network Segment Range" alias="network-segment-range" description="Provides support for the network segment range management" updated="2018-11-29T00:00:00-00:00" {{(pid=105597) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 09:04:27.248122 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Extension network-segment-range not supported by any of loaded plugins
May 09 09:04:27.248190 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading extension file: pagination.py {{(pid=105597) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 09:04:27.248376 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Ext name="Pagination support" alias="pagination" description="Extension that indicates that pagination is enabled." updated="2016-06-12T00:00:00-00:00" {{(pid=105597) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 09:04:27.248443 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loaded extension: pagination
May 09 09:04:27.248504 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading extension file: port_device_profile.py {{(pid=105597) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 09:04:27.248686 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Ext name="Port device profile" alias="port-device-profile" description="Expose the port device profile (Cyborg)" updated="2020-12-17T10:00:00-00:00" {{(pid=105597) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 09:04:27.248842 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loaded extension: port-device-profile
May 09 09:04:27.248910 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading extension file: port_hardware_offload_type.py {{(pid=105597) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 09:04:27.249088 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Ext name="Port hardware offload" alias="port-hardware-offload-type" description="Expose port hardware offload extension" updated="2023-05-09T10:00:00-00:00" {{(pid=105597) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 09:04:27.249221 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Extension port-hardware-offload-type not supported by any of loaded plugins
May 09 09:04:27.249284 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading extension file: port_hint_ovs_tx_steering.py {{(pid=105597) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 09:04:27.249462 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Ext name="Port hint: Open vSwitch Tx steering" alias="port-hint-ovs-tx-steering" description="Enable Port hint: Open vSwitch Tx steering" updated="2023-01-01T00:00:00-00:00" {{(pid=105597) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 09:04:27.249594 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Extension port-hint-ovs-tx-steering not supported by any of loaded plugins
May 09 09:04:27.249664 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading extension file: port_hints.py {{(pid=105597) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 09:04:27.249852 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Ext name="Port hints" alias="port-hints" description="Backend specific port hints to allow backend specific tuning" updated="2023-01-01T00:00:00-00:00" {{(pid=105597) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 09:04:27.249983 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Extension port-hints not supported by any of loaded plugins
May 09 09:04:27.250046 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading extension file: port_mac_address_override.py {{(pid=105597) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 09:04:27.250223 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Ext name="Neutron Port MAC address override" alias="port-mac-override" description="Allow overriding the MAC address of a direct-physical Port via the active binding profile" updated="2022-03-18T10:00:00-00:00" {{(pid=105597) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 09:04:27.250403 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Extension port-mac-override not supported by any of loaded plugins
May 09 09:04:27.250472 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading extension file: port_mac_address_regenerate.py {{(pid=105597) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 09:04:27.250644 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Ext name="Neutron Port MAC address regenerate" alias="port-mac-address-regenerate" description="Network port MAC address regenerate" updated="2018-05-03T10:00:00-00:00" {{(pid=105597) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 09:04:27.250795 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loaded extension: port-mac-address-regenerate
May 09 09:04:27.250866 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading extension file: port_numa_affinity_policy.py {{(pid=105597) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 09:04:27.251042 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Ext name="Port NUMA affinity policy" alias="port-numa-affinity-policy" description="Expose the port NUMA affinity policy" updated="2020-07-08T10:00:00-00:00" {{(pid=105597) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 09:04:27.251173 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loaded extension: port-numa-affinity-policy
May 09 09:04:27.251241 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading extension file: port_numa_affinity_policy_socket.py {{(pid=105597) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 09:04:27.251416 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Ext name="Port NUMA affinity policy "socket"" alias="port-numa-affinity-policy-socket" description="Adds "socket" to the supported port NUMA affinity policies" updated="2024-02-09T10:00:00-00:00" {{(pid=105597) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 09:04:27.251548 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loaded extension: port-numa-affinity-policy-socket
May 09 09:04:27.251620 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading extension file: port_resource_request.py {{(pid=105597) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 09:04:27.251811 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Ext name="Port Resource Request" alias="port-resource-request" description="Expose resource request to Port" updated="2018-05-08T10:00:00-00:00" {{(pid=105597) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 09:04:27.251942 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Extension port-resource-request not supported by any of loaded plugins
May 09 09:04:27.252011 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading extension file: port_resource_request_groups.py {{(pid=105597) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 09:04:27.252197 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Ext name="Port Resource Request Groups" alias="port-resource-request-groups" description="Support requesting multiple groups of resources and traits from the same RP subtree in resource_request" updated="2021-08-02T10:00:00-00:00" {{(pid=105597) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 09:04:27.252327 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Extension port-resource-request-groups not supported by any of loaded plugins
May 09 09:04:27.252397 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading extension file: port_trusted_vif.py {{(pid=105597) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 09:04:27.252569 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Ext name="Port trusted vif" alias="port-trusted-vif" description="Expose port 'trusted' attribute in the API" updated="2024-07-10T10:00:00-00:00" {{(pid=105597) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 09:04:27.252699 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Extension port-trusted-vif not supported by any of loaded plugins
May 09 09:04:27.252784 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading extension file: portbindings.py {{(pid=105597) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 09:04:27.252959 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Ext name="Port Binding" alias="binding" description="Expose port bindings of a virtual port to external application" updated="2014-02-03T10:00:00-00:00" {{(pid=105597) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 09:04:27.253096 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loaded extension: binding
May 09 09:04:27.253176 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading extension file: portbindings_extended.py {{(pid=105597) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 09:04:27.253358 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Ext name="Port Bindings Extended" alias="binding-extended" description="Expose port bindings of a virtual port to external application" updated="2017-07-17T10:00:00-00:00" {{(pid=105597) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 09:04:27.253487 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loaded extension: binding-extended
May 09 09:04:27.253556 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading extension file: portsecurity.py {{(pid=105597) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 09:04:27.253729 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Ext name="Port Security" alias="port-security" description="Provides port security" updated="2012-07-23T10:00:00-00:00" {{(pid=105597) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 09:04:27.254034 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loaded extension: port-security
May 09 09:04:27.254100 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading extension file: project_id.py {{(pid=105597) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 09:04:27.254277 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Ext name="project_id field enabled" alias="project-id" description="Extension that indicates that project_id field is enabled." updated="2016-09-09T09:09:09-09:09" {{(pid=105597) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 09:04:27.254345 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loaded extension: project-id
May 09 09:04:27.254459 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading extension file: providernet.py {{(pid=105597) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 09:04:27.254662 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Ext name="Provider Network" alias="provider" description="Expose mapping of virtual networks to physical networks" updated="2012-09-07T10:00:00-00:00" {{(pid=105597) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 09:04:27.254872 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loaded extension: provider
May 09 09:04:27.254964 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading extension file: qinq.py {{(pid=105597) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 09:04:27.255238 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Ext name="QinQ networks" alias="qinq" description="Expose network 'qinq' attribute in the API. This allows to configure network to allow vlan in vlan configuration using 0x8a88 ethertype (QinQ)." updated="2024-12-02T10:00:00-00:00" {{(pid=105597) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 09:04:27.255423 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loaded extension: qinq
May 09 09:04:27.255530 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading extension file: qos.py {{(pid=105597) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 09:04:27.256469 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Ext name="Quality of Service" alias="qos" description="The Quality of Service extension." updated="2015-06-08T10:00:00-00:00" {{(pid=105597) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 09:04:27.256661 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Extension qos not supported by any of loaded plugins
May 09 09:04:27.256783 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading extension file: qos_bw_limit_direction.py {{(pid=105597) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 09:04:27.257071 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Ext name="Direction for QoS bandwidth limit rule" alias="qos-bw-limit-direction" description="Allow to configure QoS bandwidth limit rule with specific direction: ingress or egress" updated="2017-04-10T10:00:00-00:00" {{(pid=105597) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 09:04:27.257257 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Extension qos-bw-limit-direction not supported by any of loaded plugins
May 09 09:04:27.257356 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading extension file: qos_bw_minimum_ingress.py {{(pid=105597) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 09:04:27.257602 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Ext name="Ingress direction for QoS minimum bandwidth rule" alias="qos-bw-minimum-ingress" description="Allow to configure QoS minumum bandwidth rule with ingress direction." updated="2018-07-09T10:00:00-00:00" {{(pid=105597) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 09:04:27.257813 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Extension qos-bw-minimum-ingress not supported by any of loaded plugins
May 09 09:04:27.257905 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading extension file: qos_default.py {{(pid=105597) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 09:04:27.258154 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Ext name="QoS default policy" alias="qos-default" description="Expose the QoS default policy per project" updated="2017-041-06T10:00:00-00:00" {{(pid=105597) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 09:04:27.258339 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Extension qos-default not supported by any of loaded plugins
May 09 09:04:27.258552 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading extension file: qos_fip.py {{(pid=105597) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 09:04:27.258865 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Ext name="Floating IP QoS" alias="qos-fip" description="The floating IP Quality of Service extension" updated="2017-07-20T00:00:00-00:00" {{(pid=105597) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 09:04:27.259049 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Extension qos-fip not supported by any of loaded plugins
May 09 09:04:27.259146 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading extension file: qos_fip_network_policy.py {{(pid=105597) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 09:04:27.259392 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Ext name="QoS floating IP network policy ID" alias="qos-fip-network-policy" description="Adds a the QoS network ID to the floating IP definition" updated="2021-11-15T10:00:00-00:00" {{(pid=105597) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 09:04:27.259576 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Extension qos-fip-network-policy not supported by any of loaded plugins
May 09 09:04:27.259648 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading extension file: qos_gateway_ip.py {{(pid=105597) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 09:04:27.259845 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Ext name="Router gateway IP QoS" alias="qos-gateway-ip" description="The Router gateway IP Quality of Service extension" updated="2018-02-24T00:00:00-00:00" {{(pid=105597) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 09:04:27.259997 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Extension qos-gateway-ip not supported by any of loaded plugins
May 09 09:04:27.260075 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading extension file: qos_port_network_policy.py {{(pid=105597) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 09:04:27.260260 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Ext name="QoS port network policy ID" alias="qos-port-network-policy" description="Adds a the QoS network ID to the port definition" updated="2019-11-01T10:00:00-00:00" {{(pid=105597) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 09:04:27.260394 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Extension qos-port-network-policy not supported by any of loaded plugins
May 09 09:04:27.260458 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading extension file: qos_pps_minimum_rule.py {{(pid=105597) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 09:04:27.260644 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Ext name="QoS minimum packet rate rule" alias="qos-pps-minimum" description="Allow to configure QoS minimum packet rate rule." updated="2021-07-14T10:00:00-00:00" {{(pid=105597) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 09:04:27.260794 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Extension qos-pps-minimum not supported by any of loaded plugins
May 09 09:04:27.260864 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading extension file: qos_pps_minimum_rule_alias.py {{(pid=105597) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 09:04:27.261050 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Ext name="QoS minimum packet rate rule alias" alias="qos-pps-minimum-rule-alias" description="API to enable GET, PUT and DELETE operations on QoS minimum packet rate rule without specifying policy ID" updated="2021-10-22T10:00:00-00:00" {{(pid=105597) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 09:04:27.261180 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Extension qos-pps-minimum-rule-alias not supported by any of loaded plugins
May 09 09:04:27.261295 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading extension file: qos_pps_rule.py {{(pid=105597) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 09:04:27.261441 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Ext name="QoS Rule Type Packet per Second Extension" alias="qos-pps" description="Add QoS Rule Type Packet per Second" updated="2021-05-12T10:00:00-00:00" {{(pid=105597) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 09:04:27.261568 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Extension qos-pps not supported by any of loaded plugins
May 09 09:04:27.261640 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading extension file: qos_rule_type_details.py {{(pid=105597) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 09:04:27.261831 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Ext name="Details of QoS rule types" alias="qos-rule-type-details" description="Expose details about QoS rule types supported by loaded backend drivers" updated="2017-06-22T10:00:00-00:00" {{(pid=105597) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 09:04:27.261961 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Extension qos-rule-type-details not supported by any of loaded plugins
May 09 09:04:27.262041 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading extension file: qos_rule_type_filter.py {{(pid=105597) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 09:04:27.262229 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Ext name="Allow to filter the list of QoS rule types" alias="qos-rule-type-filter" description="Allows to filter the QoS rule type list adding two new flags. "all_rules" prints all implemented QoS rule types."all_supported" prints all supported QoS rule types by the loaded mechanism drivers" updated="2022-02-02T10:00:00-00:00" {{(pid=105597) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 09:04:27.262402 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Extension qos-rule-type-filter not supported by any of loaded plugins
May 09 09:04:27.262486 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading extension file: qos_rules_alias.py {{(pid=105597) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 09:04:27.262681 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Ext name="Quality of Service rules alias API" alias="qos-rules-alias" description="API to enable GET, PUT and DELETE operations on QoS policy rules without specifying policy ID" updated="2018-10-07T10:00:00-00:00" {{(pid=105597) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 09:04:27.262830 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Extension qos-rules-alias not supported by any of loaded plugins
May 09 09:04:27.262901 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading extension file: quota_check_limit.py {{(pid=105597) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 09:04:27.263075 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Ext name="Quota engine limit check" alias="quota-check-limit" description="Support for checking the resource usage before applying a new quota limit" updated="2021-09-08T16:00:00-00:00" {{(pid=105597) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 09:04:27.263205 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loaded extension: quota-check-limit
May 09 09:04:27.263275 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading extension file: quota_check_limit_default.py {{(pid=105597) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 09:04:27.263465 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Ext name="Quota engine limit check by default" alias="quota-check-limit-default" description="By default, the Neutron quota engine checks the resource usage before applying a new quota limit" updated="2024-08-21T16:00:00-00:00" {{(pid=105597) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 09:04:27.263595 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loaded extension: quota-check-limit-default
May 09 09:04:27.263664 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading extension file: quotasv2.py {{(pid=105597) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 09:04:27.264050 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Ext name="Quota management support" alias="quotas" description="Expose functions for quotas management per project" updated="2012-07-29T10:00:00-00:00" {{(pid=105597) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 09:04:27.264260 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loaded extension: quotas
May 09 09:04:27.264328 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading extension file: quotasv2_detail.py {{(pid=105597) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 09:04:27.264603 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Ext name="Quota details management support" alias="quota_details" description="Expose functions for quotas usage statistics per project" updated="2017-02-10T10:00:00-00:00" {{(pid=105597) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 09:04:27.264678 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loaded extension: quota_details
May 09 09:04:27.264730 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading extension file: rbac.py {{(pid=105597) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 09:04:27.265322 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Ext name="RBAC Policies" alias="rbac-policies" description="Allows creation and modification of policies that control project access to resources." updated="2015-06-17T12:15:12-00:00" {{(pid=105597) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 09:04:27.265458 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loaded extension: rbac-policies
May 09 09:04:27.265531 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading extension file: rbac_address_group.py {{(pid=105597) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 09:04:27.265728 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Ext name="Add address_group type to RBAC" alias="rbac-address-group" description="Add address_group type to network RBAC" updated="2021-01-20T00:00:00-00:00" {{(pid=105597) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 09:04:27.265885 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Extension rbac-address-group not supported by any of loaded plugins
May 09 09:04:27.265932 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading extension file: rbac_address_scope.py {{(pid=105597) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 09:04:27.266143 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Ext name="Add address_scope type to RBAC" alias="rbac-address-scope" description="Add address_scope type to RBAC" updated="2020-02-12T00:00:00-00:00" {{(pid=105597) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 09:04:27.266255 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loaded extension: rbac-address-scope
May 09 09:04:27.266326 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading extension file: rbac_security_groups.py {{(pid=105597) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 09:04:27.266551 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Ext name="Add security_group type to network RBAC" alias="rbac-security-groups" description="Add security_group type to network RBAC" updated="2019-02-14T00:00:00-00:00" {{(pid=105597) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 09:04:27.266678 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loaded extension: rbac-security-groups
May 09 09:04:27.266752 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading extension file: rbac_subnetpool.py {{(pid=105597) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 09:04:27.266948 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Ext name="Add subnetpool type to RBAC" alias="rbac-subnetpool" description="Add subnetpool type to RBAC" updated="2020-02-05T00:00:00-00:00" {{(pid=105597) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 09:04:27.267081 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Extension rbac-subnetpool not supported by any of loaded plugins
May 09 09:04:27.267151 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading extension file: revisionifmatch.py {{(pid=105597) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 09:04:27.267326 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Ext name="If-Match constraints based on revision_number" alias="revision-if-match" description="Extension indicating that If-Match based on revision_number is supported." updated="2016-12-11T00:00:00-00:00" {{(pid=105597) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 09:04:27.267459 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loaded extension: revision-if-match
May 09 09:04:27.267530 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading extension file: revisions.py {{(pid=105597) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 09:04:27.267736 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Ext name="Resource revision numbers" alias="standard-attr-revisions" description="This extension will display the revision number of neutron resources." updated="2016-04-11T10:00:00-00:00" {{(pid=105597) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 09:04:27.267889 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loaded extension: standard-attr-revisions
May 09 09:04:27.267959 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading extension file: router_availability_zone.py {{(pid=105597) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 09:04:27.268160 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Ext name="Router Availability Zone" alias="router_availability_zone" description="Availability zone support for router." updated="2015-01-01T10:00:00-00:00" {{(pid=105597) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 09:04:27.268295 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loaded extension: router_availability_zone
May 09 09:04:27.268362 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading extension file: router_enable_snat.py {{(pid=105597) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 09:04:27.268536 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Ext name="Exposes the router "enable_snat" read-only configuration flag" alias="router-enable-snat" description="Exposes the router "enable_snat" read-only configuration flag" updated="2025-09-18T18:00:00-00:00" {{(pid=105597) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 09:04:27.268715 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loaded extension: router-enable-snat
May 09 09:04:27.268837 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading extension file: routerservicetype.py {{(pid=105597) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 09:04:27.269087 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Ext name="Router Service Type" alias="router-service-type" description="Provides router service type" updated="2013-01-29T00:00:00-00:00" {{(pid=105597) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 09:04:27.269259 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Extension router-service-type not supported by any of loaded plugins
May 09 09:04:27.269354 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading extension file: security_groups_default_rules.py {{(pid=105597) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 09:04:27.269839 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Ext name="Default rules for security groups" alias="security-groups-default-rules" description="Configure set of security group rules used as default rules for every new security group" updated="2022-12-19T10:00:00-00:00" {{(pid=105597) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 09:04:27.270028 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loaded extension: security-groups-default-rules
May 09 09:04:27.270124 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading extension file: security_groups_normalized_cidr.py {{(pid=105597) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 09:04:27.270436 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Ext name="Normalized CIDR field for security group rules" alias="security-groups-normalized-cidr" description="Add new field with normalized remote_ip_prefix cidr in SG rule" updated="2020-07-28T10:00:00-00:00" {{(pid=105597) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 09:04:27.270659 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loaded extension: security-groups-normalized-cidr
May 09 09:04:27.270806 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading extension file: security_groups_port_filtering.py {{(pid=105597) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 09:04:27.271061 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Ext name="Port filtering on security groups" alias="port-security-groups-filtering" description="Provides security groups filtering when listing ports" updated="2018-01-09T09:00:00-00:00" {{(pid=105597) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 09:04:27.271246 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Extension port-security-groups-filtering not supported by any of loaded plugins
May 09 09:04:27.271337 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading extension file: security_groups_remote_address_group.py {{(pid=105597) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 09:04:27.271601 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Ext name="Remote address group id field for security group rules" alias="security-groups-remote-address-group" description="Add new field of remote address group id in SG rules" updated="2020-08-25T10:00:00-00:00" {{(pid=105597) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 09:04:27.271799 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loaded extension: security-groups-remote-address-group
May 09 09:04:27.271871 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading extension file: security_groups_rules_belongs_to_default_sg.py {{(pid=105597) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 09:04:27.272050 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Ext name="Security group rule belongs to the project's default security group" alias="security-groups-rules-belongs-to-default-sg" description="Flag to determine if the security group rule belongs to the project's default security group" updated="2023-05-23T10:00:00-00:00" {{(pid=105597) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 09:04:27.272182 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loaded extension: security-groups-rules-belongs-to-default-sg
May 09 09:04:27.272252 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading extension file: security_groups_shared_filtering.py {{(pid=105597) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 09:04:27.272424 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Ext name="Security group filtering on the shared field" alias="security-groups-shared-filtering" description="Support filtering security groups on the shared field" updated="2021-10-05T09:00:00-00:00" {{(pid=105597) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 09:04:27.272555 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loaded extension: security-groups-shared-filtering
May 09 09:04:27.272626 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading extension file: securitygroup.py {{(pid=105597) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 09:04:27.273973 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Ext name="security-group" alias="security-group" description="The security groups extension." updated="2012-10-05T10:00:00-00:00" {{(pid=105597) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 09:04:27.274113 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loaded extension: security-group
May 09 09:04:27.274166 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading extension file: segment.py {{(pid=105597) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 09:04:27.274446 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Ext name="Segment" alias="segment" description="Segments extension." updated="2016-02-24T17:00:00-00:00" {{(pid=105597) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 09:04:27.274586 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Extension segment not supported by any of loaded plugins
May 09 09:04:27.274647 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading extension file: segments_peer_subnet_host_routes.py {{(pid=105597) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 09:04:27.274855 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Ext name="Segments peer-subnet host routes" alias="segments-peer-subnet-host-routes" description="Add host routes to subnets on a routed network (segments)" updated="2018-06-12T10:00:00-00:00" {{(pid=105597) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 09:04:27.274993 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Extension segments-peer-subnet-host-routes not supported by any of loaded plugins
May 09 09:04:27.275065 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading extension file: servicetype.py {{(pid=105597) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 09:04:27.275259 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Ext name="Neutron Service Type Management" alias="service-type" description="API for retrieving service providers for Neutron advanced services" updated="2013-01-20T00:00:00-00:00" {{(pid=105597) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 09:04:27.275386 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loaded extension: service-type
May 09 09:04:27.275457 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading extension file: sorting.py {{(pid=105597) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 09:04:27.275646 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Ext name="Sorting support" alias="sorting" description="Extension that indicates that sorting is enabled." updated="2016-06-12T00:00:00-00:00" {{(pid=105597) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 09:04:27.275709 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loaded extension: sorting
May 09 09:04:27.275799 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading extension file: standard_attr_segment.py {{(pid=105597) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 09:04:27.275976 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Ext name="Standard Attribute Segment Extension" alias="standard-attr-segment" description="Add standard attributes to Segment resource" updated="2018-04-09T10:00:00-00:00" {{(pid=105597) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 09:04:27.276109 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Extension standard-attr-segment not supported by any of loaded plugins
May 09 09:04:27.276180 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading extension file: standardattrdescription.py {{(pid=105597) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 09:04:27.276375 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Ext name="standard-attr-description" alias="standard-attr-description" description="Extension to add descriptions to standard attributes" updated="2016-02-10T10:00:00-00:00" {{(pid=105597) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 09:04:27.276504 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loaded extension: standard-attr-description
May 09 09:04:27.276575 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading extension file: stateful_security_group.py {{(pid=105597) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 09:04:27.276813 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Ext name="Stateful security group" alias="stateful-security-group" description="Indicates if the security group is stateful or not" updated="2019-11-26T09:00:00-00:00" {{(pid=105597) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 09:04:27.276941 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loaded extension: stateful-security-group
May 09 09:04:27.277020 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading extension file: stdattrs_common.py {{(pid=105597) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 09:04:27.277177 np0000173033 neutron-ovn-maintenance-worker[105597]: WARNING neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Did not find expected name "Stdattrs_common" in /opt/stack/neutron/neutron/extensions/stdattrs_common.py
May 09 09:04:27.277248 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading extension file: subnet_dns_publish_fixed_ip.py {{(pid=105597) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 09:04:27.277419 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Ext name="Subnet DNS publish fixed IP" alias="subnet-dns-publish-fixed-ip" description="Support choosing to publish DNS records for IPs from a subnet" updated="2019-05-24T10:00:00-00:00" {{(pid=105597) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 09:04:27.277554 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loaded extension: subnet-dns-publish-fixed-ip
May 09 09:04:27.277626 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading extension file: subnet_external_network.py {{(pid=105597) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 09:04:27.277818 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Ext name="Subnet belongs to an external network" alias="subnet-external-network" description="Informs if the subnet belongs to an external network" updated="2024-02-05T18:00:00-00:00" {{(pid=105597) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 09:04:27.277954 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loaded extension: subnet-external-network
May 09 09:04:27.278022 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading extension file: subnet_onboard.py {{(pid=105597) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 09:04:27.278210 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Ext name="Subnet Onboard" alias="subnet_onboard" description="Provides support for onboarding subnets into subnet pools" updated="2018-12-18T09:00:00-00:00" {{(pid=105597) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 09:04:27.278497 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Extension subnet_onboard not supported by any of loaded plugins
May 09 09:04:27.278578 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading extension file: subnet_segmentid_writable.py {{(pid=105597) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 09:04:27.278806 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Ext name="Subnet SegmentID (writable)" alias="subnet-segmentid-writable" description="Provides a writable segment_id attribute for a subnet resource." updated="2018-03-12T00:00:00-00:00" {{(pid=105597) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 09:04:27.278912 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Extension subnet-segmentid-writable not supported by any of loaded plugins
May 09 09:04:27.278972 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading extension file: subnet_service_types.py {{(pid=105597) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 09:04:27.279157 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Ext name="Subnet service types" alias="subnet-service-types" description="Provides ability to set the subnet service_types field" updated="2016-03-15T18:00:00-00:00" {{(pid=105597) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 09:04:27.279293 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loaded extension: subnet-service-types
May 09 09:04:27.279354 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading extension file: subnetallocation.py {{(pid=105597) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 09:04:27.279551 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Ext name="Subnet Allocation" alias="subnet_allocation" description="Enables allocation of subnets from a subnet pool" updated="2015-03-30T10:00:00-00:00" {{(pid=105597) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 09:04:27.279680 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loaded extension: subnet_allocation
May 09 09:04:27.279755 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading extension file: subnetpool_prefix_ops.py {{(pid=105597) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 09:04:27.279971 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Ext name="Subnet Pool Prefix Operations" alias="subnetpool-prefix-ops" description="Provides support for adjusting the prefix list of subnet pools" updated="2019-02-08T10:00:00-00:00" {{(pid=105597) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 09:04:27.280104 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loaded extension: subnetpool-prefix-ops
May 09 09:04:27.280172 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading extension file: tag_creation.py {{(pid=105597) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 09:04:27.280351 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Ext name="Tag creation extension" alias="tag-creation" description="Allow to create multiple tags for a resource" updated="2024-23-07T10:00:00-00:00" {{(pid=105597) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 09:04:27.280482 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loaded extension: tag-creation
May 09 09:04:27.280553 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading extension file: tag_ports_during_bulk_creation.py {{(pid=105597) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 09:04:27.280730 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Ext name="Tag Ports During Bulk Creation" alias="tag-ports-during-bulk-creation" description="Allow to tag ports during bulk creation" updated="2019-12-29T19:00:00-00:00" {{(pid=105597) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 09:04:27.280882 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Extension tag-ports-during-bulk-creation not supported by any of loaded plugins
May 09 09:04:27.280969 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading extension file: tagging.py {{(pid=105597) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 09:04:27.281665 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Ext name="Tag support for resources with standard attribute: port, subnet, subnetpool, network, security_group, router, floatingip, policy, trunk, network_segment_range" alias="standard-attr-tag" description="Enables to set tag on resources with standard attribute." updated="2017-01-01T00:00:00-00:00" {{(pid=105597) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 09:04:27.281826 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loaded extension: standard-attr-tag
May 09 09:04:27.281889 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading extension file: timestamp.py {{(pid=105597) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 09:04:27.282104 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Ext name="Resource timestamps" alias="standard-attr-timestamp" description="Adds created_at and updated_at fields to all Neutron resources that have Neutron standard attributes." updated="2016-09-12T10:00:00-00:00" {{(pid=105597) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 09:04:27.282230 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loaded extension: standard-attr-timestamp
May 09 09:04:27.282282 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading extension file: trunk.py {{(pid=105597) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 09:04:27.282529 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Ext name="Trunk Extension" alias="trunk" description="Provides support for trunk ports" updated="2016-01-01T10:00:00-00:00" {{(pid=105597) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 09:04:27.282654 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Extension trunk not supported by any of loaded plugins
May 09 09:04:27.282725 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading extension file: trunk_details.py {{(pid=105597) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 09:04:27.282927 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Ext name="Trunk port details" alias="trunk-details" description="Expose trunk port details" updated="2016-01-01T10:00:00-00:00" {{(pid=105597) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 09:04:27.283054 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Extension trunk-details not supported by any of loaded plugins
May 09 09:04:27.283124 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading extension file: uplink_status_propagation.py {{(pid=105597) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 09:04:27.283300 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Ext name="Uplink status propagation extension" alias="uplink-status-propagation" description="Provides ability to propagate port uplink status." updated="2018-05-31T18:00:00-00:00" {{(pid=105597) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 09:04:27.283431 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Extension uplink-status-propagation not supported by any of loaded plugins
May 09 09:04:27.283501 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading extension file: uplink_status_propagation_updatable.py {{(pid=105597) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 09:04:27.283676 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Ext name="Uplink status propagation updatable extension" alias="uplink-status-propagation-updatable" description="Provides ability to update the port uplink status." updated="2024-09-03T18:00:00-00:00" {{(pid=105597) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 09:04:27.283824 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Extension uplink-status-propagation-updatable not supported by any of loaded plugins
May 09 09:04:27.283889 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loading extension file: vlantransparent.py {{(pid=105597) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
May 09 09:04:27.284089 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Ext name="Vlantransparent" alias="vlan-transparent" description="Provides Vlan Transparent Networks" updated="2015-03-23T09:00:00-00:00" {{(pid=105597) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
May 09 09:04:27.284209 np0000173033 neutron-ovn-maintenance-worker[105597]: INFO neutron.api.extensions [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Loaded extension: vlan-transparent
May 09 09:04:27.286558 np0000173033 neutron-ovn-maintenance-worker[105597]: WARNING oslo_service.backend._threading.service [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Service MaintenanceWorker is not picklable with spawn; falling back to fork. Please make the service spawn-safe to avoid this fallback.
May 09 09:04:27.286868 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Full set of CONF: {{(pid=105597) _load_service_manager_options /opt/stack/data/venv/lib/python3.12/site-packages/cotyledon/oslo_config_glue.py:59}}
May 09 09:04:27.286929 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] ******************************************************************************** {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2979}}
May 09 09:04:27.286988 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Configuration options gathered from: {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2980}}
May 09 09:04:27.287063 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] command line args: ['--config-file', '/etc/neutron/neutron.conf', '--config-file', '/etc/neutron/plugins/ml2/ml2_conf.ini'] {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2981}}
May 09 09:04:27.287160 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] config files: ['/etc/neutron/neutron.conf', '/etc/neutron/plugins/ml2/ml2_conf.ini'] {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2982}}
May 09 09:04:27.287220 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] ================================================================================ {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2987}}
May 09 09:04:27.287331 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] agent_down_time                = 75 {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.287413 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] allow_automatic_dhcp_failover  = True {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.287506 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] allow_bulk                     = True {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.287671 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] allowed_conntrack_helpers      = [{'amanda': 'tcp'}, {'ftp': 'tcp'}, {'h323': 'udp'}, {'h323': 'tcp'}, {'irc': 'tcp'}, {'netbios-ns': 'udp'}, {'pptp': 'tcp'}, {'sane': 'tcp'}, {'sip': 'udp'}, {'sip': 'tcp'}, {'snmp': 'udp'}, {'tftp': 'udp'}] {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.287760 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] api_extensions_path            =  {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.287863 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] api_paste_config               = api-paste.ini {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.287958 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] api_workers                    = 4 {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.288048 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] auth_strategy                  = keystone {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.288137 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] base_mac                       = fa:16:3e:00:00:00 {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.288239 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] bind_host                      = 0.0.0.0 {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.288329 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] bind_port                      = 9696 {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.288416 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] client_socket_timeout          = 900 {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.288508 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] config_dir                     = [] {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.288575 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] config_file                    = ['/etc/neutron/neutron.conf', '/etc/neutron/plugins/ml2/ml2_conf.ini'] {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.288660 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] config_source                  = [] {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.288753 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] control_exchange               = neutron {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.288857 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] core_plugin                    = ml2 {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.288945 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] debug                          = True {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.289029 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] default_availability_zones     = [] {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.289182 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] default_log_levels             = ['amqp=WARN', 'boto=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'OFPHandler=INFO', 'OfctlService=INFO', 'os_ken.base.app_manager=INFO', 'os_ken.controller.controller=INFO'] {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.289262 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] dhcp_agent_notification        = False {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.289349 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] dhcp_agents_per_network        = 1 {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.289432 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] dhcp_lease_duration            = 86400 {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.289522 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] dhcp_load_type                 = networks {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.289609 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] dns_domain                     = openstacklocal {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.289698 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] dvr_base_mac                   = fa:16:3f:00:00:00 {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.289805 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] enable_default_route_bfd       = False {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.289884 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] enable_default_route_ecmp      = False {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.289968 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] enable_dvr                     = True {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.290051 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] enable_new_agents              = True {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.290134 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] enable_services_on_agents_with_admin_state_down = False {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.290218 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] enable_snat_by_default         = True {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.290301 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] enable_traditional_dhcp        = True {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.290422 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] external_dns_driver            = None {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.290497 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] filter_validation              = True {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.290591 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] global_physnet_mtu             = 1430 {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.290656 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] graceful_shutdown_timeout      = 60 {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.290807 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] host                           = np0000173033 {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.290896 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] host_dvr_for_dhcp              = True {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.290981 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] http_retries                   = 3 {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.291070 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] instance_format                = [instance: %(uuid)s]  {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.291156 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] instance_uuid_format           = [instance: %(uuid)s]  {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.291244 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] ipam_driver                    = internal {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.291329 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] ipv6_pd_enabled                = False {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.291418 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] l3_ha                          = False {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.291500 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] l3_ha_net_cidr                 = 169.254.192.0/18 {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.291589 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] l3_ha_network_physical_name    =  {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.291675 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] l3_ha_network_type             =  {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.291782 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] log_color                      = False {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.291871 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] log_config_append              = None {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.291960 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] log_date_format                = %Y-%m-%d %H:%M:%S {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.292045 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] log_dir                        = None {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.292127 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] log_file                       = None {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.292180 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] log_options                    = True {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.292271 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] log_rotate_interval            = 1 {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.292357 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] log_rotate_interval_type       = days {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.292445 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] log_rotation_type              = none {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.292517 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] logging_context_format_string  = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.292568 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] logging_debug_format_suffix    = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.292661 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] logging_default_format_string  = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.292768 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] logging_exception_prefix       = ERROR %(name)s %(instance)s {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.292823 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] logging_user_identity_format   = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.292910 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] max_allowed_address_pair       = 10 {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.292992 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] max_dns_nameservers            = 5 {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.293075 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] max_header_line                = 16384 {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.293158 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] max_l3_agents_per_router       = 3 {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.293241 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] max_logfile_count              = 30 {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.293323 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] max_logfile_size_mb            = 200 {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.293406 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] max_routes                     = 30 {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.293488 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] max_subnet_host_routes         = 20 {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.293574 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] my_ip                          = 199.204.45.23 {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.293660 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] my_ipv6                        = 2604:e100:1:0:f816:3eff:fe38:46d9 {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.293912 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] network_auto_schedule          = True {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.293995 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] network_link_prefix            = None {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.294085 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] network_scheduler_driver       = neutron.scheduler.dhcp_agent_scheduler.WeightScheduler {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.294174 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] notify_nova_on_port_data_changes = True {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.294260 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] notify_nova_on_port_status_changes = True {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.294349 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] pagination_max_limit           = -1 {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.294448 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] periodic_fuzzy_delay           = 5 {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.294539 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] periodic_interval              = 40 {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.294622 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] publish_errors                 = False {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.294706 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] rate_limit_burst               = 0 {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.294811 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] rate_limit_except_level        = CRITICAL {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.294896 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] rate_limit_interval            = 0 {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.294982 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] router_distributed             = False {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.295068 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] rpc_resources_processing_step  = 20 {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.295153 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] rpc_response_max_timeout       = 600 {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.295234 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] rpc_response_timeout           = 60 {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.295321 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] rpc_state_report_workers       = 0 {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.295402 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] rpc_workers                    = None {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.295484 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] send_events_interval           = 2 {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.295580 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] service_plugins                = ['ovn-router'] {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.295669 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] setproctitle                   = on {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.295753 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] shell_completion               = None {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.295857 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] state_path                     = /opt/stack/data/neutron {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.295946 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] syslog_log_facility            = LOG_USER {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.296031 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] tcp_keepidle                   = 600 {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.296118 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] transport_url                  = **** {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.296200 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] use_journal                    = False {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.296283 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] use_json                       = False {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.296365 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] use_stderr                     = False {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.296456 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] use_syslog                     = False {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.296532 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] vlan_qinq                      = None {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.296615 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] vlan_transparent               = None {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.296698 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] wsgi_default_pool_size         = 100 {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.296796 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] wsgi_keep_alive                = True {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.296886 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] wsgi_log_format                = %(client_ip)s "%(request_line)s" status: %(status_code)s  len: %(body_length)s time: %(wall_seconds).7f {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.296969 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] wsgi_server_debug              = False {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.297070 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] oslo_concurrency.disable_process_locking = False {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.297165 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] oslo_concurrency.lock_path     = /opt/stack/data/neutron/lock {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.297265 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] profiler.connection_string     = messaging:// {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.297356 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] profiler.enabled               = False {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.297448 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] profiler.es_doc_type           = notification {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.297536 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] profiler.es_scroll_size        = 10000 {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.297626 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] profiler.es_scroll_time        = 2m {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.297713 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] profiler.filter_error_trace    = False {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.297822 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] profiler.hmac_keys             = **** {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.297912 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] profiler.sentinel_service_name = mymaster {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.298007 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] profiler.socket_timeout        = 0.1 {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.298095 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] profiler.trace_requests        = False {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.298181 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] profiler.trace_sqlalchemy      = False {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.298277 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] profiler_jaeger.process_tags   = {} {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.298363 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] profiler_jaeger.service_name_prefix = None {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.298487 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] profiler_otlp.service_name_prefix = None {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.298581 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.298671 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] oslo_messaging_metrics.metrics_enabled = False {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.298778 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] oslo_messaging_metrics.metrics_process_name =  {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.298871 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.298957 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.299054 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] oslo_policy.enforce_new_defaults = True {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.299235 np0000173033 neutron-ovn-maintenance-worker[105597]: WARNING oslo_config.cfg [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Deprecated: Option "enforce_scope" from group "oslo_policy" is deprecated for removal (This configuration was added temporarily to facilitate a smooth transition to the new RBAC. OpenStack will always enforce scope checks. This configuration option is deprecated and will be removed in the 2025.2 cycle.).  Its value may be silently ignored in the future.
May 09 09:04:27.299331 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] oslo_policy.enforce_scope      = True {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.299427 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] oslo_policy.policy_default_rule = default {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.299525 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] oslo_policy.policy_dirs        = ['policy.d'] {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.299611 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] oslo_policy.policy_file        = /etc/neutron/policy.json {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.299706 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.299806 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.299892 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] oslo_policy.remote_ssl_client_crt_file = None {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.299976 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] oslo_policy.remote_ssl_client_key_file = None {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.300061 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.300160 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] oslo_policy.remote_timeout     = 60.0 {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.300291 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.300388 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] service_providers.service_provider = [] {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.300488 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] nova.auth_section              = None {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.300595 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] nova.auth_type                 = password {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.300688 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] nova.auth_url                  = https://199.204.45.23/identity {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.300791 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] nova.cafile                    = None {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.300874 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] nova.certfile                  = None {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.300963 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] nova.collect_timing            = False {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.301046 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] nova.default_domain_id         = None {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.301129 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] nova.default_domain_name       = None {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.301212 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] nova.domain_id                 = None {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.301294 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] nova.domain_name               = None {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.301383 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] nova.endpoint_type             = public {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.301469 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] nova.insecure                  = False {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.301551 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] nova.keyfile                   = None {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.301643 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] nova.password                  = **** {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.301726 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] nova.project_domain_id         = None {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.301830 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] nova.project_domain_name       = Default {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.301917 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] nova.project_id                = None {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.302008 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] nova.project_name              = service {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.302095 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] nova.region_name               = RegionOne {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.302181 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] nova.split_loggers             = False {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.302262 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] nova.system_scope              = None {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.302345 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] nova.timeout                   = None {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.302466 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] nova.trust_id                  = None {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.302568 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] nova.user_domain_id            = None {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.302658 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] nova.user_domain_name          = Default {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.302751 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] nova.user_id                   = None {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.302852 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] nova.username                  = nova {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.302943 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] placement.auth_section         = None {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.303032 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] placement.auth_type            = password {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.303125 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] placement.cafile               = None {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.303214 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] placement.certfile             = None {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.303319 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] placement.collect_timing       = False {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.303413 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] placement.endpoint_type        = public {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.303503 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] placement.insecure             = False {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.303594 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] placement.keyfile              = None {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.303684 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] placement.region_name          = None {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.303795 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] placement.split_loggers        = False {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.303883 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] placement.timeout              = None {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.303976 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] ironic.auth_section            = None {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.304066 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] ironic.auth_type               = None {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.304155 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] ironic.cafile                  = None {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.304243 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] ironic.certfile                = None {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.304336 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] ironic.collect_timing          = False {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.304434 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] ironic.connect_retries         = None {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.304521 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] ironic.connect_retry_delay     = None {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.304621 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] ironic.enable_notifications    = False {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.304710 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] ironic.endpoint_override       = None {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.304821 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] ironic.insecure                = False {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.304911 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] ironic.interface               = None {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.305000 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] ironic.keyfile                 = None {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.305091 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] ironic.max_version             = None {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.305181 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] ironic.min_version             = None {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.305272 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] ironic.region_name             = None {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.305364 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] ironic.retriable_status_codes  = None {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.305452 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] ironic.service_name            = None {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.305540 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] ironic.service_type            = None {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.305635 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] ironic.split_loggers           = False {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.305726 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] ironic.status_code_retries     = None {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.305831 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] ironic.status_code_retry_delay = None {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.305926 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] ironic.timeout                 = None {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.306020 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] ironic.valid_interfaces        = None {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.306111 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] ironic.version                 = None {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.306202 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] cli_script.dry_run             = False {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.306299 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.306447 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.306582 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.306670 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.306781 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.306869 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.306956 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] oslo_messaging_rabbit.heartbeat_in_pthread = False {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.307043 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.307129 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] oslo_messaging_rabbit.heartbeat_timeout_threshold = 60 {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.307219 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] oslo_messaging_rabbit.hostname = np0000173033 {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.307305 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] oslo_messaging_rabbit.kombu_compression = None {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.307397 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.307486 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.307572 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.307658 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] oslo_messaging_rabbit.kombu_reconnect_splay = 0.0 {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.307755 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] oslo_messaging_rabbit.processname = neutron-ovn-maintenance-worker {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.307851 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.307936 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.308024 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.308106 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.308191 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.308277 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] oslo_messaging_rabbit.rabbit_quorum_max_memory_bytes = 0 {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.308360 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] oslo_messaging_rabbit.rabbit_quorum_max_memory_length = 0 {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.308444 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.308529 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.308614 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.308698 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] oslo_messaging_rabbit.rabbit_stream_fanout = False {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.308806 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.308892 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] oslo_messaging_rabbit.rabbit_transient_quorum_queue = False {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.308975 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.309059 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] oslo_messaging_rabbit.ssl      = False {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.309150 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] oslo_messaging_rabbit.ssl_ca_file =  {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.309235 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] oslo_messaging_rabbit.ssl_cert_file =  {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.309320 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.309408 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] oslo_messaging_rabbit.ssl_key_file =  {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.309512 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] oslo_messaging_rabbit.ssl_version =  {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.309611 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] oslo_messaging_rabbit.use_queue_manager = False {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.309660 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] oslo_messaging_notifications.driver = [] {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.309755 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] oslo_messaging_notifications.retry = -1 {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.309856 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] oslo_messaging_notifications.topics = ['notifications'] {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.309941 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] oslo_messaging_notifications.transport_url = **** {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.310043 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] oslo_reports.file_event_handler = None {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.310109 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] oslo_reports.file_event_handler_interval = 1 {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.310192 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] oslo_reports.log_dir           = None {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.310285 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.310428 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] QUOTAS.default_quota           = -1 {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.310495 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] QUOTAS.quota_driver            = neutron.db.quota.driver_nolock.DbQuotaNoLockDriver {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.310580 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] QUOTAS.quota_floatingip        = 50 {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.310663 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] QUOTAS.quota_network           = 100 {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.310757 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] QUOTAS.quota_port              = 500 {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.310849 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] QUOTAS.quota_rbac_policy       = 10 {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.310934 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] QUOTAS.quota_router            = 10 {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.311019 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] QUOTAS.quota_security_group    = 10 {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.311103 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] QUOTAS.quota_security_group_rule = 100 {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.311186 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] QUOTAS.quota_subnet            = 100 {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.311271 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] QUOTAS.track_quota_usage       = True {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.311365 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] privsep.capabilities           = [21, 12, 1, 2, 19] {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.311450 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] privsep.group                  = None {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.311677 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] privsep.helper_command         = None {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.311677 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] privsep.log_daemon_traceback   = False {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.311762 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] privsep.logger_name            = oslo_privsep.daemon {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.311803 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] privsep.thread_pool_size       = 16 {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.311886 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] privsep.user                   = None {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.311974 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] privsep_dhcp_release.capabilities = [21, 12] {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.312058 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] privsep_dhcp_release.group     = None {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.312151 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] privsep_dhcp_release.helper_command = None {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.312233 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] privsep_dhcp_release.log_daemon_traceback = False {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.312306 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] privsep_dhcp_release.logger_name = oslo_privsep.daemon {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.312389 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] privsep_dhcp_release.thread_pool_size = 16 {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.312470 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] privsep_dhcp_release.user      = None {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.312570 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] privsep_ovs_vsctl.capabilities = [21, 12] {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.312640 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] privsep_ovs_vsctl.group        = None {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.312721 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] privsep_ovs_vsctl.helper_command = None {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.312851 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] privsep_ovs_vsctl.log_daemon_traceback = False {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.312947 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] privsep_ovs_vsctl.logger_name  = oslo_privsep.daemon {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.312981 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] privsep_ovs_vsctl.thread_pool_size = 16 {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.313071 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] privsep_ovs_vsctl.user         = None {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.313179 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] privsep_namespace.capabilities = [21] {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.313240 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] privsep_namespace.group        = None {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.313320 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] privsep_namespace.helper_command = None {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.313403 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] privsep_namespace.log_daemon_traceback = False {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.313489 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] privsep_namespace.logger_name  = oslo_privsep.daemon {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.313572 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] privsep_namespace.thread_pool_size = 16 {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.313651 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] privsep_namespace.user         = None {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.313738 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] privsep_conntrack.capabilities = [12] {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.313839 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] privsep_conntrack.group        = None {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.313920 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] privsep_conntrack.helper_command = None {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.314002 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] privsep_conntrack.log_daemon_traceback = False {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.314087 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] privsep_conntrack.logger_name  = oslo_privsep.daemon {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.314172 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] privsep_conntrack.thread_pool_size = 16 {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.314251 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] privsep_conntrack.user         = None {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.314342 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] privsep_link.capabilities      = [12, 21] {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.314445 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] privsep_link.group             = None {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.314526 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] privsep_link.helper_command    = None {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.314610 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] privsep_link.log_daemon_traceback = False {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.314694 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] privsep_link.logger_name       = oslo_privsep.daemon {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.314796 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] privsep_link.thread_pool_size  = 16 {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.314878 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] privsep_link.user              = None {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.314976 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] AGENT.root_helper              = sudo /opt/stack/data/venv/bin/neutron-rootwrap /etc/neutron/rootwrap.conf {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.315061 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] AGENT.root_helper_daemon       = sudo /opt/stack/data/venv/bin/neutron-rootwrap-daemon /etc/neutron/rootwrap.conf {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.315146 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] AGENT.use_helper_for_ns_read   = True {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.315316 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] SECURITYGROUP.enable_ipset     = True {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.315316 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] SECURITYGROUP.enable_security_group = True {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.315407 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] SECURITYGROUP.firewall_driver  = None {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.315486 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] SECURITYGROUP.permitted_ethertypes = [] {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.315574 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] experimental.ipv6_pd_enabled   = False {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.315674 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] ml2.extension_drivers          = ['port_security', 'qos'] {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.315871 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] ml2.external_network_type      = None {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.315871 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] ml2.mechanism_drivers          = ['ovn'] {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.315959 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] ml2.overlay_ip_version         = 4 {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.316036 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] ml2.path_mtu                   = 0 {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.316124 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] ml2.physical_network_mtus      = {} {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.316218 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] ml2.project_network_types      = ['geneve'] {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.316308 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] ml2.tunnelled_network_rp_name  = rp_tunnelled {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.316403 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] ml2.type_drivers               = ['local', 'flat', 'vlan', 'geneve'] {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.316501 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] ml2_type_flat.flat_networks    = ['public'] {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.316603 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] ml2_type_geneve.max_header_size = 38 {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.316688 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] ml2_type_geneve.vni_ranges     = ['1:65536'] {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.316797 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] ml2_type_vlan.network_vlan_ranges = ['public'] {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.316887 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] network_log.burst_limit        = 25 {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.316971 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] network_log.local_output_log_base = None {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.317056 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] network_log.rate_limit         = 100 {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.317146 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] database.asyncio_connection    = **** {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.317237 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] database.asyncio_slave_connection = **** {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.317319 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] database.backend               = sqlalchemy {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.317408 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] database.connection            = **** {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.317493 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] database.connection_debug      = 0 {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.317603 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] database.connection_parameters =  {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.317760 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] database.connection_recycle_time = 3600 {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.317760 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] database.connection_trace      = False {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.317858 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] database.db_inc_retry_interval = True {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.317940 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] database.db_max_retries        = 20 {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.318024 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] database.db_max_retry_interval = 10 {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.318108 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] database.db_retry_interval     = 1 {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.318192 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] database.max_overflow          = 50 {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.318277 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] database.max_pool_size         = 5 {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.318361 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] database.max_retries           = 10 {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.318527 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] database.mysql_sql_mode        = TRADITIONAL {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.318617 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] database.mysql_wsrep_sync_wait = None {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.318701 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] database.pool_timeout          = None {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.318802 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] database.retry_interval        = 10 {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.318887 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] database.slave_connection      = **** {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.318975 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] database.sqlite_synchronous    = True {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.319057 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] database.synchronous_reader    = True {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.319141 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] database.use_db_reconnect      = False {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.319235 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] ovn.bfd_min_rx                 = 1000 {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.319320 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] ovn.bfd_min_tx                 = 100 {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.319407 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] ovn.bfd_mult                   = 3 {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.319490 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] ovn.broadcast_arps_to_all_routers = True {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.319573 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] ovn.dhcp_default_lease_time    = 43200 {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.319659 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] ovn.disable_ovn_dhcp_for_baremetal_ports = False {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.319754 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] ovn.dns_records_ovn_owned      = False {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.319853 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] ovn.dns_servers                = [] {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.319936 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] ovn.enable_distributed_floating_ip = False {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.320023 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] ovn.fdb_age_threshold          = 300 {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.320112 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] ovn.ha_failover_strategy       = normal {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.320201 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] ovn.live_migration_activation_strategy = rarp {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.320286 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] ovn.localnet_learn_fdb         = False {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.320372 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] ovn.mac_binding_age_threshold  = 300 {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.320460 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] ovn.neutron_sync_mode          = log {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.320547 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] ovn.ovn_dhcp4_global_options   = {} {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.320638 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] ovn.ovn_dhcp6_global_options   = {} {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.320727 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] ovn.ovn_l3_scheduler           = leastloaded {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.320833 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] ovn.ovn_metadata_enabled       = True {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.320923 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] ovn.ovn_nb_ca_cert             = /opt/stack/data/CA/int-ca/ca-chain.pem {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.321010 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] ovn.ovn_nb_certificate         = /opt/stack/data/CA/int-ca/devstack-cert.crt {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.321105 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] ovn.ovn_nb_connection          = ['ssl:199.204.45.23:6641'] {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.321193 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] ovn.ovn_nb_private_key         = /opt/stack/data/CA/int-ca/private/devstack-cert.key {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.321277 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] ovn.ovn_router_indirect_snat   = False {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.321364 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] ovn.ovn_sb_ca_cert             = /opt/stack/data/CA/int-ca/ca-chain.pem {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.321451 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] ovn.ovn_sb_certificate         = /opt/stack/data/CA/int-ca/devstack-cert.crt {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.321563 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] ovn.ovn_sb_connection          = ['ssl:199.204.45.23:6642'] {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.321625 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] ovn.ovn_sb_private_key         = /opt/stack/data/CA/int-ca/private/devstack-cert.key {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.321710 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] ovn.ovs_create_tap             = False {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.321813 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] ovn.ovsdb_connection_timeout   = 180 {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.321903 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] ovn.ovsdb_log_level            = INFO {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.321989 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] ovn.ovsdb_probe_interval       = 60000 {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.322073 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] ovn.ovsdb_retry_max_interval   = 180 {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.322157 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] ovn.stateless_nat_enabled      = False {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.322243 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] ovn.vhost_sock_dir             = /var/run/openvswitch {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.322332 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] OVS.bridge_mac_table_size      = 50000 {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.322439 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] OVS.igmp_flood                 = False {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.322524 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] OVS.igmp_flood_reports         = True {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.322608 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] OVS.igmp_flood_unregistered    = False {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.322695 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] OVS.igmp_snooping_enable       = False {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.322796 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] OVS.ovsdb_timeout              = 10 {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.322888 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] ovn_nb_global.fdb_removal_limit = 0 {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.322972 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] ovn_nb_global.ignore_lsp_down  = False {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.323060 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] ovn_nb_global.mac_binding_removal_limit = 0 {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.323127 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG cotyledon.oslo_config_glue [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] ******************************************************************************** {{(pid=105597) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3013}}
May 09 09:04:27.323213 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG oslo_service.backend._threading.service [None req-1b8b33d0-1bde-4d67-a742-2f4c76889867 None None] Selected the multiprocessing context: <multiprocessing.context.ForkContext object at 0x73c88c16aa50> and updated it in Cotyledon ServiceManager: <multiprocessing.context.ForkContext object at 0x73c88c16aa50> {{(pid=105597) launch_service /opt/stack/data/venv/lib/python3.12/site-packages/oslo_service/backend/_threading/service.py:394}}
May 09 09:04:27.354316 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] Full set of CONF: {{(pid=106314) _load_service_options /opt/stack/data/venv/lib/python3.12/site-packages/cotyledon/oslo_config_glue.py:65}}
May 09 09:04:27.355114 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] ******************************************************************************** {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2979}}
May 09 09:04:27.355114 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] Configuration options gathered from: {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2980}}
May 09 09:04:27.355114 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] command line args: ['--config-file', '/etc/neutron/neutron.conf', '--config-file', '/etc/neutron/plugins/ml2/ml2_conf.ini'] {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2981}}
May 09 09:04:27.355114 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] config files: ['/etc/neutron/neutron.conf', '/etc/neutron/plugins/ml2/ml2_conf.ini'] {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2982}}
May 09 09:04:27.355114 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] ================================================================================ {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2987}}
May 09 09:04:27.355114 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] agent_down_time                = 75 {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.355114 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] allow_automatic_dhcp_failover  = True {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.355490 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] allow_bulk                     = True {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.355490 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] allowed_conntrack_helpers      = [{'amanda': 'tcp'}, {'ftp': 'tcp'}, {'h323': 'udp'}, {'h323': 'tcp'}, {'irc': 'tcp'}, {'netbios-ns': 'udp'}, {'pptp': 'tcp'}, {'sane': 'tcp'}, {'sip': 'udp'}, {'sip': 'tcp'}, {'snmp': 'udp'}, {'tftp': 'udp'}] {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.355490 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] api_extensions_path            =  {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.355654 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] api_paste_config               = api-paste.ini {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.355778 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] api_workers                    = 4 {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.355873 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] auth_strategy                  = keystone {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.355973 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] base_mac                       = fa:16:3e:00:00:00 {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.356213 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] bind_host                      = 0.0.0.0 {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.356352 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] bind_port                      = 9696 {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.356418 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] client_socket_timeout          = 900 {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.356560 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] config_dir                     = [] {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.356648 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] config_file                    = ['/etc/neutron/neutron.conf', '/etc/neutron/plugins/ml2/ml2_conf.ini'] {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.356736 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] config_source                  = [] {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.356879 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] control_exchange               = neutron {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.356980 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] core_plugin                    = ml2 {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.357091 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] debug                          = True {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.357198 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] default_availability_zones     = [] {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.357425 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] default_log_levels             = ['amqp=WARN', 'boto=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'OFPHandler=INFO', 'OfctlService=INFO', 'os_ken.base.app_manager=INFO', 'os_ken.controller.controller=INFO'] {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.357563 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] dhcp_agent_notification        = False {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.357645 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] dhcp_agents_per_network        = 1 {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.357706 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] dhcp_lease_duration            = 86400 {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.357842 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] dhcp_load_type                 = networks {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.357932 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] dns_domain                     = openstacklocal {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.358045 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] dvr_base_mac                   = fa:16:3f:00:00:00 {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.358152 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] enable_default_route_bfd       = False {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.358230 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] enable_default_route_ecmp      = False {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.358342 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] enable_dvr                     = True {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.358470 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] enable_new_agents              = True {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.358570 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] enable_services_on_agents_with_admin_state_down = False {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.358673 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] enable_snat_by_default         = True {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.358796 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] enable_traditional_dhcp        = True {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.358882 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] external_dns_driver            = None {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.358942 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] fatal_deprecations             = False {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.359040 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] filter_validation              = True {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.359101 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] global_physnet_mtu             = 1430 {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.359208 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] graceful_shutdown_timeout      = 60 {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.359494 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] host                           = np0000173033 {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.359579 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] host_dvr_for_dhcp              = True {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.359705 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] http_retries                   = 3 {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.359837 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] instance_format                = [instance: %(uuid)s]  {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.359937 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] instance_uuid_format           = [instance: %(uuid)s]  {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.360041 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] ipam_driver                    = internal {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.360131 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] ipv6_pd_enabled                = False {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.360228 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] l3_ha                          = False {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.360338 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] l3_ha_net_cidr                 = 169.254.192.0/18 {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.360437 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] l3_ha_network_physical_name    =  {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.360526 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] l3_ha_network_type             =  {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.360616 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] log_color                      = False {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.360716 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] log_config_append              = None {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.360841 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] log_date_format                = %Y-%m-%d %H:%M:%S {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.360918 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] log_dir                        = None {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.361028 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] log_file                       = None {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.361130 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] log_options                    = True {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.361130 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] log_rotate_interval            = 1 {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.361242 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] log_rotate_interval_type       = days {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.361308 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] log_rotation_type              = none {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.361459 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] logging_context_format_string  = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.361459 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] logging_debug_format_suffix    = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.361790 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] logging_default_format_string  = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.361790 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] logging_exception_prefix       = ERROR %(name)s %(instance)s {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.361790 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] logging_user_identity_format   = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.361790 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] max_allowed_address_pair       = 10 {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.362018 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] max_dns_nameservers            = 5 {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.362018 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] max_header_line                = 16384 {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.362018 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] max_l3_agents_per_router       = 3 {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.362167 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] max_logfile_count              = 30 {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.362167 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] max_logfile_size_mb            = 200 {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.362337 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] max_routes                     = 30 {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.362478 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] max_subnet_host_routes         = 20 {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.362568 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] my_ip                          = 199.204.45.23 {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.362666 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] my_ipv6                        = 2604:e100:1:0:f816:3eff:fe38:46d9 {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.362761 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] network_auto_schedule          = True {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.362874 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] network_link_prefix            = None {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.362982 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] network_scheduler_driver       = neutron.scheduler.dhcp_agent_scheduler.WeightScheduler {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.363075 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] notify_nova_on_port_data_changes = True {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.363196 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] notify_nova_on_port_status_changes = True {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.363305 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] pagination_max_limit           = -1 {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.363393 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] periodic_fuzzy_delay           = 5 {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.363488 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] periodic_interval              = 40 {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.363571 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] publish_errors                 = False {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.363634 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] rate_limit_burst               = 0 {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.363698 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] rate_limit_except_level        = CRITICAL {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.363773 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] rate_limit_interval            = 0 {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.363842 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] router_distributed             = False {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.363910 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] rpc_resources_processing_step  = 20 {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.363981 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] rpc_response_max_timeout       = 600 {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.364050 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] rpc_response_timeout           = 60 {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.364113 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] rpc_state_report_workers       = 0 {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.364173 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] rpc_workers                    = None {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.364242 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] send_events_interval           = 2 {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.364324 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] service_plugins                = ['ovn-router'] {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.364390 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] setproctitle                   = on {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.364464 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] shell_completion               = None {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.364536 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] state_path                     = /opt/stack/data/neutron {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.364608 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] syslog_log_facility            = LOG_USER {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.364676 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] tcp_keepidle                   = 600 {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.364754 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] transport_url                  = **** {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.364831 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] use_journal                    = False {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.365082 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] use_json                       = False {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.365185 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] use_stderr                     = False {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.365267 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] use_syslog                     = False {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.365325 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] vlan_qinq                      = None {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.365384 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] vlan_transparent               = None {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.365445 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] wsgi_default_pool_size         = 100 {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.365507 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] wsgi_keep_alive                = True {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.365598 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] wsgi_log_format                = %(client_ip)s "%(request_line)s" status: %(status_code)s  len: %(body_length)s time: %(wall_seconds).7f {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.365655 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] wsgi_server_debug              = False {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
May 09 09:04:27.365806 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] oslo_concurrency.disable_process_locking = False {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.365909 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] oslo_concurrency.lock_path     = /opt/stack/data/neutron/lock {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.366034 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] profiler.connection_string     = messaging:// {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.366121 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] profiler.enabled               = False {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.366203 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] profiler.es_doc_type           = notification {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.366278 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] profiler.es_scroll_size        = 10000 {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.366354 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] profiler.es_scroll_time        = 2m {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.366457 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] profiler.filter_error_trace    = False {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.366535 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] profiler.hmac_keys             = **** {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.366611 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] profiler.sentinel_service_name = mymaster {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.366696 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] profiler.socket_timeout        = 0.1 {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.366787 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] profiler.trace_requests        = False {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.366861 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] profiler.trace_sqlalchemy      = False {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.366976 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] profiler_jaeger.process_tags   = {} {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.367042 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] profiler_jaeger.service_name_prefix = None {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.367106 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] profiler_otlp.service_name_prefix = None {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.367202 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.367272 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.367343 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_metrics.metrics_process_name =  {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.367422 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.367497 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.367605 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] oslo_policy.enforce_new_defaults = True {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.367656 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] oslo_policy.enforce_scope      = True {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.367707 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] oslo_policy.policy_default_rule = default {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.367769 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] oslo_policy.policy_dirs        = ['policy.d'] {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.367815 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] oslo_policy.policy_file        = /etc/neutron/policy.json {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.367875 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.367934 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.367977 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.368091 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.368091 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.368091 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] oslo_policy.remote_timeout     = 60.0 {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.368165 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.368224 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] service_providers.service_provider = [] {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.368290 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] nova.auth_section              = None {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.368337 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] nova.auth_type                 = password {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.368381 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] nova.auth_url                  = https://199.204.45.23/identity {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.368417 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] nova.cafile                    = None {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.368471 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] nova.certfile                  = None {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.368518 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] nova.collect_timing            = False {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.368568 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] nova.default_domain_id         = None {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.368617 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] nova.default_domain_name       = None {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.368646 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] nova.domain_id                 = None {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.368691 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] nova.domain_name               = None {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.368731 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] nova.endpoint_type             = public {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.368795 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] nova.insecure                  = False {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.368834 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] nova.keyfile                   = None {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.368891 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] nova.password                  = **** {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.368939 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] nova.project_domain_id         = None {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.368981 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] nova.project_domain_name       = Default {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.369027 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] nova.project_id                = None {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.369071 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] nova.project_name              = service {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.369106 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] nova.region_name               = RegionOne {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.369153 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] nova.split_loggers             = False {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.369188 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] nova.system_scope              = None {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.369220 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] nova.timeout                   = None {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.369273 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] nova.trust_id                  = None {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.369304 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] nova.user_domain_id            = None {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.369346 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] nova.user_domain_name          = Default {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.369393 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] nova.user_id                   = None {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.369423 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] nova.username                  = nova {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.369487 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] placement.auth_section         = None {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.369534 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] placement.auth_type            = password {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.369574 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] placement.cafile               = None {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.369608 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] placement.certfile             = None {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.369643 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] placement.collect_timing       = False {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.369689 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] placement.endpoint_type        = public {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.369719 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] placement.insecure             = False {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.369783 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] placement.keyfile              = None {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.369824 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] placement.region_name          = None {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.369868 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] placement.split_loggers        = False {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.369918 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] placement.timeout              = None {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.369975 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] ironic.auth_section            = None {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.370014 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] ironic.auth_type               = None {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.370049 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] ironic.cafile                  = None {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.370082 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] ironic.certfile                = None {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.370132 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] ironic.collect_timing          = False {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.370168 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] ironic.connect_retries         = None {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.370224 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] ironic.connect_retry_delay     = None {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.370270 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] ironic.enable_notifications    = False {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.370305 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] ironic.endpoint_override       = None {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.370334 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] ironic.insecure                = False {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.370400 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] ironic.interface               = None {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.370529 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] ironic.keyfile                 = None {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.370586 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] ironic.max_version             = None {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.370616 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] ironic.min_version             = None {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.370663 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] ironic.region_name             = None {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.370704 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] ironic.retriable_status_codes  = None {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.370757 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] ironic.service_name            = None {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.370805 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] ironic.service_type            = None {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.370846 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] ironic.split_loggers           = False {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.370924 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] ironic.status_code_retries     = None {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.370985 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] ironic.status_code_retry_delay = None {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.371034 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] ironic.timeout                 = None {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.371075 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] ironic.valid_interfaces        = None {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.371115 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] ironic.version                 = None {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.371161 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] cli_script.dry_run             = False {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.371228 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.371277 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.371325 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.371375 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.371428 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.371459 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.371522 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.heartbeat_in_pthread = False {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.371568 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.371616 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 60 {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.371652 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.hostname = np0000173033 {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.371693 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.371757 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.371813 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.371852 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.371907 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.kombu_reconnect_splay = 0.0 {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.371959 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.processname = neutron-ovn-maintenance-worker {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.372001 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.372047 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.372095 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.372125 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.372168 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.372198 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_quorum_max_memory_bytes = 0 {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.372252 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_quorum_max_memory_length = 0 {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.372283 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.372332 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.372362 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.372402 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_stream_fanout = False {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.372447 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.372478 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_transient_quorum_queue = False {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.372521 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.372562 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.ssl      = False {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.372598 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.ssl_ca_file =  {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.372639 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.ssl_cert_file =  {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.372669 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.372716 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.ssl_key_file =  {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.372770 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.ssl_version =  {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.372811 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.use_queue_manager = False {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.372866 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_notifications.driver = [] {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.372929 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_notifications.retry = -1 {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.372984 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.373026 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_notifications.transport_url = **** {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.373082 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] oslo_reports.file_event_handler = None {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.373116 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] oslo_reports.file_event_handler_interval = 1 {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.373163 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] oslo_reports.log_dir           = None {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.373232 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.373301 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] QUOTAS.default_quota           = -1 {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.373351 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] QUOTAS.quota_driver            = neutron.db.quota.driver_nolock.DbQuotaNoLockDriver {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.373406 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] QUOTAS.quota_floatingip        = 50 {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.373436 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] QUOTAS.quota_network           = 100 {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.373490 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] QUOTAS.quota_port              = 500 {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.373526 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] QUOTAS.quota_rbac_policy       = 10 {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.373572 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] QUOTAS.quota_router            = 10 {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.373619 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] QUOTAS.quota_security_group    = 10 {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.373667 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] QUOTAS.quota_security_group_rule = 100 {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.373697 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] QUOTAS.quota_subnet            = 100 {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.373753 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] QUOTAS.track_quota_usage       = True {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.373837 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] privsep.capabilities           = [21, 12, 1, 2, 19] {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.373901 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] privsep.group                  = None {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.373951 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] privsep.helper_command         = None {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.374001 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] privsep.log_daemon_traceback   = False {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.374049 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] privsep.logger_name            = oslo_privsep.daemon {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.374088 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] privsep.thread_pool_size       = 16 {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.374140 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] privsep.user                   = None {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.374193 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] privsep_dhcp_release.capabilities = [21, 12] {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.374232 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] privsep_dhcp_release.group     = None {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.374271 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] privsep_dhcp_release.helper_command = None {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.374318 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] privsep_dhcp_release.log_daemon_traceback = False {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.374357 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] privsep_dhcp_release.logger_name = oslo_privsep.daemon {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.374386 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] privsep_dhcp_release.thread_pool_size = 16 {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.374489 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] privsep_dhcp_release.user      = None {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.374489 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] privsep_ovs_vsctl.capabilities = [21, 12] {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.374562 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] privsep_ovs_vsctl.group        = None {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.374562 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] privsep_ovs_vsctl.helper_command = None {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.374614 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] privsep_ovs_vsctl.log_daemon_traceback = False {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.374643 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] privsep_ovs_vsctl.logger_name  = oslo_privsep.daemon {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.374687 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] privsep_ovs_vsctl.thread_pool_size = 16 {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.374727 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] privsep_ovs_vsctl.user         = None {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.374796 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] privsep_namespace.capabilities = [21] {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.374836 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] privsep_namespace.group        = None {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.374886 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] privsep_namespace.helper_command = None {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.374937 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] privsep_namespace.log_daemon_traceback = False {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.374973 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] privsep_namespace.logger_name  = oslo_privsep.daemon {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.375014 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] privsep_namespace.thread_pool_size = 16 {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.375055 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] privsep_namespace.user         = None {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.375114 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] privsep_conntrack.capabilities = [12] {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.375151 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] privsep_conntrack.group        = None {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.375199 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] privsep_conntrack.helper_command = None {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.375229 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] privsep_conntrack.log_daemon_traceback = False {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.375267 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] privsep_conntrack.logger_name  = oslo_privsep.daemon {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.375313 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] privsep_conntrack.thread_pool_size = 16 {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.375343 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] privsep_conntrack.user         = None {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.375402 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] privsep_link.capabilities      = [12, 21] {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.375448 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] privsep_link.group             = None {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.375485 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] privsep_link.helper_command    = None {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.375519 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] privsep_link.log_daemon_traceback = False {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.375572 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] privsep_link.logger_name       = oslo_privsep.daemon {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.375602 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] privsep_link.thread_pool_size  = 16 {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.375642 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] privsep_link.user              = None {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.375698 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] AGENT.root_helper              = sudo /opt/stack/data/venv/bin/neutron-rootwrap /etc/neutron/rootwrap.conf {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.375755 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] AGENT.root_helper_daemon       = sudo /opt/stack/data/venv/bin/neutron-rootwrap-daemon /etc/neutron/rootwrap.conf {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.375806 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] AGENT.use_helper_for_ns_read   = True {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.375866 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] SECURITYGROUP.enable_ipset     = True {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.375929 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] SECURITYGROUP.enable_security_group = True {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.375986 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] SECURITYGROUP.firewall_driver  = None {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.376018 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] SECURITYGROUP.permitted_ethertypes = [] {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.376082 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] experimental.ipv6_pd_enabled   = False {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.376143 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] ml2.extension_drivers          = ['port_security', 'qos'] {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.376180 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] ml2.external_network_type      = None {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.376222 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] ml2.mechanism_drivers          = ['ovn'] {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.376261 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] ml2.overlay_ip_version         = 4 {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.376308 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] ml2.path_mtu                   = 0 {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.376343 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] ml2.physical_network_mtus      = {} {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.376379 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] ml2.project_network_types      = ['geneve'] {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.376427 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] ml2.tunnelled_network_rp_name  = rp_tunnelled {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.376462 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] ml2.type_drivers               = ['local', 'flat', 'vlan', 'geneve'] {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.376525 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] ml2_type_flat.flat_networks    = ['public'] {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.376575 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] ml2_type_geneve.max_header_size = 38 {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.376622 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] ml2_type_geneve.vni_ranges     = ['1:65536'] {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.376678 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] ml2_type_vlan.network_vlan_ranges = ['public'] {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.376740 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] network_log.burst_limit        = 25 {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.376809 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] network_log.local_output_log_base = None {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.376845 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] network_log.rate_limit         = 100 {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.376901 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] database.asyncio_connection    = **** {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.376954 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] database.asyncio_slave_connection = **** {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.376996 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] database.backend               = sqlalchemy {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.377043 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] database.connection            = **** {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.377091 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] database.connection_debug      = 0 {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.377126 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] database.connection_parameters =  {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.377177 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] database.connection_recycle_time = 3600 {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.377213 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] database.connection_trace      = False {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.377250 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] database.db_inc_retry_interval = True {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.377292 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] database.db_max_retries        = 20 {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.377322 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] database.db_max_retry_interval = 10 {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.377367 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] database.db_retry_interval     = 1 {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.377400 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] database.max_overflow          = 50 {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.377440 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] database.max_pool_size         = 5 {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.377471 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] database.max_retries           = 10 {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.377519 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] database.mysql_sql_mode        = TRADITIONAL {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.377568 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] database.mysql_wsrep_sync_wait = None {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.377605 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] database.pool_timeout          = None {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.377653 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] database.retry_interval        = 10 {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.377688 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] database.slave_connection      = **** {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.377717 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] database.sqlite_synchronous    = True {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.377778 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] database.synchronous_reader    = True {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.377813 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] database.use_db_reconnect      = False {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.377887 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] ovn.bfd_min_rx                 = 1000 {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.377943 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] ovn.bfd_min_tx                 = 100 {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.377999 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] ovn.bfd_mult                   = 3 {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.378052 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] ovn.broadcast_arps_to_all_routers = True {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.378086 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] ovn.dhcp_default_lease_time    = 43200 {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.378136 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] ovn.disable_ovn_dhcp_for_baremetal_ports = False {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.378183 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] ovn.dns_records_ovn_owned      = False {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.378213 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] ovn.dns_servers                = [] {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.378257 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] ovn.enable_distributed_floating_ip = False {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.378305 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] ovn.fdb_age_threshold          = 300 {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.378336 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] ovn.ha_failover_strategy       = normal {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.378387 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] ovn.live_migration_activation_strategy = rarp {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.378464 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] ovn.localnet_learn_fdb         = False {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.378510 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] ovn.mac_binding_age_threshold  = 300 {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.378569 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] ovn.neutron_sync_mode          = log {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.378609 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_dhcp4_global_options   = {} {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.378650 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_dhcp6_global_options   = {} {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.378699 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_l3_scheduler           = leastloaded {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.378735 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_metadata_enabled       = True {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.378799 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_nb_ca_cert             = /opt/stack/data/CA/int-ca/ca-chain.pem {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.378849 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_nb_certificate         = /opt/stack/data/CA/int-ca/devstack-cert.crt {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.378882 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_nb_connection          = ['ssl:199.204.45.23:6641'] {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.378929 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_nb_private_key         = /opt/stack/data/CA/int-ca/private/devstack-cert.key {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.378965 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_router_indirect_snat   = False {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.379016 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_sb_ca_cert             = /opt/stack/data/CA/int-ca/ca-chain.pem {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.379052 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_sb_certificate         = /opt/stack/data/CA/int-ca/devstack-cert.crt {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.379081 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_sb_connection          = ['ssl:199.204.45.23:6642'] {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.379127 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_sb_private_key         = /opt/stack/data/CA/int-ca/private/devstack-cert.key {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.379161 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovs_create_tap             = False {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.379194 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovsdb_connection_timeout   = 180 {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.379235 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovsdb_log_level            = INFO {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.379276 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovsdb_probe_interval       = 60000 {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.379323 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovsdb_retry_max_interval   = 180 {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.379353 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] ovn.stateless_nat_enabled      = False {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.379407 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] ovn.vhost_sock_dir             = /var/run/openvswitch {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.379464 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] OVS.bridge_mac_table_size      = 50000 {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.379499 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] OVS.igmp_flood                 = False {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.379550 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] OVS.igmp_flood_reports         = True {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.379605 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] OVS.igmp_flood_unregistered    = False {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.379636 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] OVS.igmp_snooping_enable       = False {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.379669 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] OVS.ovsdb_timeout              = 10 {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.379733 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] ovn_nb_global.fdb_removal_limit = 0 {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.379800 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] ovn_nb_global.ignore_lsp_down  = False {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.379850 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] ovn_nb_global.mac_binding_removal_limit = 0 {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
May 09 09:04:27.379881 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon.oslo_config_glue [-] ******************************************************************************** {{(pid=106314) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3013}}
May 09 09:04:27.380045 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG cotyledon._service_worker [-] Run service ServiceWrapper(0) [106314] {{(pid=106314) wait_forever /opt/stack/data/venv/lib/python3.12/site-packages/cotyledon/_service_worker.py:218}}
May 09 09:04:27.381429 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron_lib.callbacks.manager [-] Publish callbacks ['neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver.post_fork_initialize-819484', 'neutron.services.ovn_l3.plugin.OVNL3RouterPlugin._post_fork_initialize-1877484'] for process (None), after_init {{(pid=106314) _notify_loop /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:184}}
May 09 09:04:27.383504 np0000173033 neutron-ovn-maintenance-worker[106314]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Getting OvsdbNbOvnIdl for MaintenanceWorker with retry
May 09 09:04:27.437967 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl [-] Created lookup_table index Logical_Switch.name {{(pid=106314) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:87}}
May 09 09:04:27.437967 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl [-] Created lookup_table index Logical_Router.name {{(pid=106314) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:87}}
May 09 09:04:27.438192 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl [-] Created lookup_table index Load_Balancer.name {{(pid=106314) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:87}}
May 09 09:04:27.438192 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Copp.name {{(pid=106314) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
May 09 09:04:27.438192 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Meter.name {{(pid=106314) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
May 09 09:04:27.438192 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Load_Balancer_Group.name {{(pid=106314) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
May 09 09:04:27.438409 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index HA_Chassis_Group.name {{(pid=106314) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
May 09 09:04:27.438409 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Address_Set.name {{(pid=106314) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
May 09 09:04:27.438409 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Connection.target {{(pid=106314) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
May 09 09:04:27.438409 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Gateway_Chassis.name {{(pid=106314) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
May 09 09:04:27.438409 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Mirror.name {{(pid=106314) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
May 09 09:04:27.438699 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Chassis_Template_Var.chassis {{(pid=106314) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
May 09 09:04:27.438699 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Logical_Router_Port.name {{(pid=106314) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
May 09 09:04:27.438699 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Port_Group.name {{(pid=106314) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
May 09 09:04:27.438699 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Logical_Switch_Port.name {{(pid=106314) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
May 09 09:04:27.441789 np0000173033 neutron-ovn-maintenance-worker[106314]: INFO ovsdbapp.backend.ovs_idl.vlog [-] ssl:199.204.45.23:6641: connecting...
May 09 09:04:27.447837 np0000173033 neutron-ovn-maintenance-worker[106314]: INFO ovsdbapp.backend.ovs_idl.vlog [-] ssl:199.204.45.23:6641: connected
May 09 09:04:27.476133 np0000173033 neutron-ovn-maintenance-worker[106314]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Getting OvsdbSbOvnIdl for MaintenanceWorker with retry
May 09 09:04:27.507950 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl [-] Created lookup_table index Chassis.name {{(pid=106314) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:87}}
May 09 09:04:27.508088 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl [-] Created lookup_table index Port_Binding.logical_port {{(pid=106314) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:87}}
May 09 09:04:27.508183 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Chassis_Private.name {{(pid=106314) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
May 09 09:04:27.508183 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Datapath_Binding.tunnel_key {{(pid=106314) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
May 09 09:04:27.510717 np0000173033 neutron-ovn-maintenance-worker[106314]: INFO ovsdbapp.backend.ovs_idl.vlog [-] ssl:199.204.45.23:6642: connecting...
May 09 09:04:27.516579 np0000173033 neutron-ovn-maintenance-worker[106314]: INFO ovsdbapp.backend.ovs_idl.vlog [-] ssl:199.204.45.23:6642: connected
May 09 09:04:27.553293 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method PortForwardingPlugin._check_floatingip_request of <neutron.services.portforwarding.pf_plugin.PortForwardingPlugin object at 0x73c886fce450>> floatingip precommit_update 55550000, False {{(pid=106314) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:27.553464 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method PortForwardingPlugin._check_floatingip_request of <neutron.services.portforwarding.pf_plugin.PortForwardingPlugin object at 0x73c886fce450>> floatingip precommit_delete 55550000, False {{(pid=106314) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:27.553557 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method PortForwardingPlugin._check_port_has_port_forwarding of <neutron.services.portforwarding.pf_plugin.PortForwardingPlugin object at 0x73c886fce450>> floatingip before_create 55550000, False {{(pid=106314) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:27.553619 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method PortForwardingPlugin._check_port_has_port_forwarding of <neutron.services.portforwarding.pf_plugin.PortForwardingPlugin object at 0x73c886fce450>> floatingip before_update 55550000, False {{(pid=106314) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:27.553675 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method PortForwardingPlugin._process_port_request_handler of <neutron.services.portforwarding.pf_plugin.PortForwardingPlugin object at 0x73c886fce450>> port after_update 55550000, False {{(pid=106314) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:27.553733 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method PortForwardingPlugin._process_port_request_handler of <neutron.services.portforwarding.pf_plugin.PortForwardingPlugin object at 0x73c886fce450>> port precommit_delete 55550000, False {{(pid=106314) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:27.554006 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron_lib.callbacks.manager [-] Publish callbacks ['neutron.services.portforwarding.drivers.ovn.driver.OVNPortForwarding.register-52738'] for port_forwarding_plugin (None), after_init {{(pid=106314) _notify_loop /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:184}}
May 09 09:04:27.554077 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method OVNPortForwarding._handle_notification of <neutron.services.portforwarding.drivers.ovn.driver.OVNPortForwarding object at 0x73c88705c4a0>> port_forwarding after_create 55550000, False {{(pid=106314) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:27.554147 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method OVNPortForwarding._handle_lb_on_ls of <neutron.services.portforwarding.drivers.ovn.driver.OVNPortForwarding object at 0x73c88705c4a0>> router_interface after_create 55550000, False {{(pid=106314) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:27.554205 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method OVNPortForwarding._handle_notification of <neutron.services.portforwarding.drivers.ovn.driver.OVNPortForwarding object at 0x73c88705c4a0>> port_forwarding after_update 55550000, False {{(pid=106314) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:27.554272 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method OVNPortForwarding._handle_notification of <neutron.services.portforwarding.drivers.ovn.driver.OVNPortForwarding object at 0x73c88705c4a0>> port_forwarding after_delete 55550000, False {{(pid=106314) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:27.554332 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method OVNPortForwarding._handle_lb_on_ls of <neutron.services.portforwarding.drivers.ovn.driver.OVNPortForwarding object at 0x73c88705c4a0>> router_interface after_delete 55550000, False {{(pid=106314) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:27.554447 np0000173033 neutron-ovn-maintenance-worker[106314]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.extensions.qos [-] Starting OVNClientQosExtension
May 09 09:04:27.554565 np0000173033 neutron-ovn-maintenance-worker[106314]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.extensions.placement [-] Starting OVNClientPlacementExtension
May 09 09:04:27.554923 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Plugin._prevent_segment_delete_with_subnet_associated of <neutron.services.segments.plugin.Plugin object at 0x73c886fce840>> segment before_delete 55550000, False {{(pid=106314) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:27.554993 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Plugin._validate_auto_address_subnet_delete of <neutron.services.segments.plugin.Plugin object at 0x73c886fce840>> subnet precommit_delete_associations 55550000, False {{(pid=106314) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:27.555281 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method NovaSegmentNotifier._notify_host_addition_to_aggregate of <neutron.services.segments.plugin.NovaSegmentNotifier object at 0x73c886fce660>> segment_host_mapping after_create 55550000, False {{(pid=106314) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:27.555349 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method NovaSegmentNotifier._notify_host_removal_from_aggregate of <neutron.services.segments.plugin.NovaSegmentNotifier object at 0x73c886fce660>> segment_host_mapping after_delete 55550000, False {{(pid=106314) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:27.555410 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method NovaSegmentNotifier._notify_port_created_or_deleted of <neutron.services.segments.plugin.NovaSegmentNotifier object at 0x73c886fce660>> port after_create 55550000, False {{(pid=106314) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:27.555472 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method NovaSegmentNotifier._notify_port_created_or_deleted of <neutron.services.segments.plugin.NovaSegmentNotifier object at 0x73c886fce660>> port after_delete 55550000, False {{(pid=106314) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:27.555529 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method NovaSegmentNotifier._notify_port_updated of <neutron.services.segments.plugin.NovaSegmentNotifier object at 0x73c886fce660>> port after_update 55550000, False {{(pid=106314) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:27.555587 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method NovaSegmentNotifier._notify_segment_deleted of <neutron.services.segments.plugin.NovaSegmentNotifier object at 0x73c886fce660>> segment after_delete 55550000, False {{(pid=106314) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:27.555654 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method NovaSegmentNotifier._notify_subnet_created of <neutron.services.segments.plugin.NovaSegmentNotifier object at 0x73c886fce660>> subnet after_create 55550000, False {{(pid=106314) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:27.555734 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method NovaSegmentNotifier._notify_subnet_deleted of <neutron.services.segments.plugin.NovaSegmentNotifier object at 0x73c886fce660>> subnet after_delete 55550000, False {{(pid=106314) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:27.555734 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method NovaSegmentNotifier._notify_subnet_updated of <neutron.services.segments.plugin.NovaSegmentNotifier object at 0x73c886fce660>> subnet after_update 55550000, False {{(pid=106314) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:27.609389 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method SegmentHostRoutes.host_routes_after_create of <neutron.services.segments.plugin.SegmentHostRoutes object at 0x73c88700be90>> subnet after_create 55550000, False {{(pid=106314) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:27.609593 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method SegmentHostRoutes.host_routes_after_delete of <neutron.services.segments.plugin.SegmentHostRoutes object at 0x73c88700be90>> subnet after_delete 55550000, False {{(pid=106314) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:27.609593 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method SegmentHostRoutes.host_routes_before_create of <neutron.services.segments.plugin.SegmentHostRoutes object at 0x73c88700be90>> subnet before_create 55550000, False {{(pid=106314) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:27.609593 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method SegmentHostRoutes.host_routes_before_update of <neutron.services.segments.plugin.SegmentHostRoutes object at 0x73c88700be90>> subnet before_update 55550000, False {{(pid=106314) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:27.611983 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method LoggingPlugin._clean_logs_by_resource_id of <neutron.services.logapi.logging_plugin.LoggingPlugin object at 0x73c88700bec0>> security_group after_delete 55550000, False {{(pid=106314) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:27.612071 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method LoggingPlugin._clean_logs_by_target_id of <neutron.services.logapi.logging_plugin.LoggingPlugin object at 0x73c88700bec0>> port after_delete 55550000, False {{(pid=106314) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:27.612185 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron_lib.callbacks.manager [-] Publish callbacks ['neutron.services.logapi.drivers.ovn.driver.DriverBase._register-1749023'] for logging-plugin (None), after_init {{(pid=106314) _notify_loop /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:184}}
May 09 09:04:27.612341 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method SecurityGroupRuleCallBack.handle_event of <neutron.services.logapi.common.sg_callback.SecurityGroupRuleCallBack object at 0x73c88700bf50>> security_group_rule after_create 55550000, False {{(pid=106314) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:27.612445 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method SecurityGroupRuleCallBack.handle_event of <neutron.services.logapi.common.sg_callback.SecurityGroupRuleCallBack object at 0x73c88700bf50>> security_group_rule after_update 55550000, False {{(pid=106314) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:27.612445 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method SecurityGroupRuleCallBack.handle_event of <neutron.services.logapi.common.sg_callback.SecurityGroupRuleCallBack object at 0x73c88700bf50>> security_group_rule after_delete 55550000, False {{(pid=106314) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
May 09 09:04:37.072398 np0000173033 neutron-ovn-maintenance-worker[105597]: DEBUG dbcounter [-] [105597] Writing DB stats neutron:SELECT=4 {{(pid=105597) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 09 09:04:37.613811 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [-] OVN-Northbound DB sync process started @ 2026-05-09 09:04:37.613261 {{(pid=106314) do_sync /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:99}}
May 09 09:04:37.616391 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-f51a1909-4349-4393-b943-37635515e3d7 None None] OVN-NB Sync port groups started @ 2026-05-09 09:04:37.615903 {{(pid=106314) sync_port_groups /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:132}}
May 09 09:04:37.639988 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG dbcounter [-] [106314] Writer thread running {{(pid=106314) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:102}}
May 09 09:04:37.652971 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-f51a1909-4349-4393-b943-37635515e3d7 None None] OVN-NB Sync port groups completed @ 2026-05-09 09:04:37.652601 {{(pid=106314) sync_port_groups /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:196}}
May 09 09:04:37.653117 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-f51a1909-4349-4393-b943-37635515e3d7 None None] OVN-NB Sync networks, ports and DHCP options started @ 2026-05-09 09:04:37.652940 {{(pid=106314) sync_networks_ports_and_dhcp_opts /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1152}}
May 09 09:04:37.935225 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.services.segments.db [None req-f51a1909-4349-4393-b943-37635515e3d7 None None] neutron.services.segments.plugin.Plugin method get_segments called with arguments (<neutron_lib.context.Context object at 0x73c885ea76b0>,) {'filters': {'network_id': ['f8a38fcb-e8a2-421a-a340-bf7f2f5cc72b'], 'is_dynamic': False}} {{(pid=106314) wrapper /opt/stack/data/venv/lib/python3.12/site-packages/oslo_log/helpers.py:77}}
May 09 09:04:37.943030 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-f51a1909-4349-4393-b943-37635515e3d7 None None] OVN-NB Sync metadata ports started {{(pid=106314) _sync_metadata_ports /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1097}}
May 09 09:04:38.002072 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-f51a1909-4349-4393-b943-37635515e3d7 None None] OVN-NB Sync metadata ports completed {{(pid=106314) _sync_metadata_ports /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1149}}
May 09 09:04:38.002308 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-f51a1909-4349-4393-b943-37635515e3d7 None None] OVN-NB Sync DHCP options for Neutron subnets started {{(pid=106314) _sync_subnet_dhcp_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:949}}
May 09 09:04:38.057435 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-f51a1909-4349-4393-b943-37635515e3d7 None None] OVN-NB Sync DHCP options for Neutron subnets completed @ 2026-05-09 09:04:38.057179 {{(pid=106314) _sync_subnet_dhcp_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1022}}
May 09 09:04:38.058214 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] There are no commands to commit {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:78}}
May 09 09:04:38.058514 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-f51a1909-4349-4393-b943-37635515e3d7 None None] OVN-NB Sync DHCP options for Neutron ports with extra dhcp options assigned started {{(pid=106314) _sync_port_dhcp_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1027}}
May 09 09:04:38.058656 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-f51a1909-4349-4393-b943-37635515e3d7 None None] OVN-NB Sync DHCP options for Neutron ports with extra dhcp options assigned completed {{(pid=106314) _sync_port_dhcp_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1086}}
May 09 09:04:38.058656 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-f51a1909-4349-4393-b943-37635515e3d7 None None] OVN-NB Sync networks, ports and DHCP options completed @ 2026-05-09 09:04:38.058564 {{(pid=106314) sync_networks_ports_and_dhcp_opts /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1321}}
May 09 09:04:38.058862 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-f51a1909-4349-4393-b943-37635515e3d7 None None] OVN-NB Sync ACLs started @ 2026-05-09 09:04:38.058670 {{(pid=106314) sync_acls /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:239}}
May 09 09:04:38.106004 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-f51a1909-4349-4393-b943-37635515e3d7 None None] OVN-NB Sync ACLs completed @ 2026-05-09 09:04:38.105671 {{(pid=106314) sync_acls /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:431}}
May 09 09:04:38.106159 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-f51a1909-4349-4393-b943-37635515e3d7 None None] OVN-NB Sync routers and router ports started @ 2026-05-09 09:04:38.106007 {{(pid=106314) sync_routers_and_rports /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:736}}
May 09 09:04:38.155343 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] There are no commands to commit {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:78}}
May 09 09:04:38.155662 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-f51a1909-4349-4393-b943-37635515e3d7 None None] OVN-NB Sync routers and router ports completed @ 2026-05-09 09:04:38.155411 {{(pid=106314) sync_routers_and_rports /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:944}}
May 09 09:04:38.155784 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-f51a1909-4349-4393-b943-37635515e3d7 None None] OVN-NB Sync port QoS policies started @ 2026-05-09 09:04:38.155660 {{(pid=106314) sync_port_qos_policies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1369}}
May 09 09:04:38.155877 np0000173033 neutron-ovn-maintenance-worker[106314]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.extensions.qos [None req-f51a1909-4349-4393-b943-37635515e3d7 None None] Starting OVNClientQosExtension
May 09 09:04:38.169210 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-f51a1909-4349-4393-b943-37635515e3d7 None None] OVN-NB Sync port QoS policies completed @ 2026-05-09 09:04:38.169058 {{(pid=106314) sync_port_qos_policies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1392}}
May 09 09:04:38.169326 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-f51a1909-4349-4393-b943-37635515e3d7 None None] OVN-NB Sync Floating IP QoS policies started @ 2026-05-09 09:04:38.169202 {{(pid=106314) sync_fip_qos_policies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1397}}
May 09 09:04:38.169372 np0000173033 neutron-ovn-maintenance-worker[106314]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.extensions.qos [None req-f51a1909-4349-4393-b943-37635515e3d7 None None] Starting OVNClientQosExtension
May 09 09:04:38.191329 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-f51a1909-4349-4393-b943-37635515e3d7 None None] OVN-NB Sync Floating IP QoS policies completed @ 2026-05-09 09:04:38.191085 {{(pid=106314) sync_fip_qos_policies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1420}}
May 09 09:04:38.191421 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-f51a1909-4349-4393-b943-37635515e3d7 None None] OVN-NB Sync Floating IP NAT rules started @ 2026-05-09 09:04:38.191324 {{(pid=106314) sync_fip_dnat_rules /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1425}}
May 09 09:04:38.191646 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-f51a1909-4349-4393-b943-37635515e3d7 None None] OVN-NB Sync Floating IP NAT rules completed @ 2026-05-09 09:04:38.191545 {{(pid=106314) sync_fip_dnat_rules /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1451}}
May 09 09:04:38.191730 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-f51a1909-4349-4393-b943-37635515e3d7 None None] OVN-Northbound DB sync process completed @ 2026-05-09 09:04:38.191641 {{(pid=106314) do_sync /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:112}}
May 09 09:04:47.615175 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [-] OVN-Southbound DB sync process started @ 2026-05-09 09:04:47.614566 {{(pid=106314) do_sync /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1480}}
May 09 09:04:47.616363 np0000173033 neutron-ovn-maintenance-worker[106314]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.extensions.qos [-] Starting OVNClientQosExtension
May 09 09:04:47.617680 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-32cb0c34-4c52-4c60-9364-5469adec1ed9 None None] OVN-SB Sync hostname and physical networks started @ 2026-05-09 09:04:47.617251 {{(pid=106314) sync_hostname_and_physical_networks /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1511}}
May 09 09:04:47.622501 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.add_provnet_ext_id_to_localnet_ports {{(pid=106314) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
May 09 09:04:47.622636 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_baremetal_ports_dhcp_options {{(pid=106314) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
May 09 09:04:47.622636 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_dhcp_options_consistency {{(pid=106314) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
May 09 09:04:47.623078 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_fair_meter_consistency {{(pid=106314) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
May 09 09:04:47.623194 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_fdb_aging_settings {{(pid=106314) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
May 09 09:04:47.623435 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_for_igmp_snoop_support {{(pid=106314) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
May 09 09:04:47.623435 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_for_inconsistencies {{(pid=106314) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
May 09 09:04:47.623656 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_global_dhcp_opts {{(pid=106314) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
May 09 09:04:47.623656 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_localnet_port_has_learn_fdb {{(pid=106314) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
May 09 09:04:47.623656 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_network_broadcast_arps_to_all_routers {{(pid=106314) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
May 09 09:04:47.623957 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_provider_distributed_ports {{(pid=106314) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
May 09 09:04:47.623957 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_redirect_type_router_gateway_ports {{(pid=106314) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
May 09 09:04:47.623957 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip {{(pid=106314) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
May 09 09:04:47.624297 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.cleanup_old_hash_ring_nodes {{(pid=106314) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
May 09 09:04:47.624297 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.configure_nb_global {{(pid=106314) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
May 09 09:04:47.624297 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.migrate_lrp_gateway_chassis_to_ha_chassis_group {{(pid=106314) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
May 09 09:04:47.624297 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.set_fip_distributed_flag {{(pid=106314) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
May 09 09:04:47.624454 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.set_network_type_and_physnet {{(pid=106314) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
May 09 09:04:47.624495 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.set_ovn_owned_dns_option {{(pid=106314) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
May 09 09:04:47.624611 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.update_ha_failover {{(pid=106314) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
May 09 09:04:47.624675 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.update_lrouter_ports_ext_ids_name_prefix {{(pid=106314) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
May 09 09:04:47.624719 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.update_mac_aging_settings {{(pid=106314) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
May 09 09:04:47.624860 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.update_router_distributed_flag {{(pid=106314) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
May 09 09:04:47.625309 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.update_router_static_routes {{(pid=106314) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
May 09 09:04:47.625378 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.update_security_group_with_address_group {{(pid=106314) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
May 09 09:04:47.625485 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.update_virtual_port_parent_hostname {{(pid=106314) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
May 09 09:04:47.627353 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.add_provnet_ext_id_to_localnet_ports' {{(pid=106314) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
May 09 09:04:47.627738 np0000173033 neutron-ovn-maintenance-worker[106314]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver [-] Maintenance task thread has started
May 09 09:04:47.627895 np0000173033 neutron-ovn-maintenance-worker[106314]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver [-] MaintenanceWorker process has finished the post initialization
May 09 09:04:47.629027 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_baremetal_ports_dhcp_options' {{(pid=106314) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
May 09 09:04:47.629362 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_dhcp_options_consistency' {{(pid=106314) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
May 09 09:04:47.629362 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fair_meter_consistency' {{(pid=106314) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
May 09 09:04:47.629362 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
May 09 09:04:47.629471 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_for_igmp_snoop_support' {{(pid=106314) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
May 09 09:04:47.629516 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_for_inconsistencies' {{(pid=106314) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
May 09 09:04:47.629886 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_global_dhcp_opts' {{(pid=106314) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
May 09 09:04:47.631002 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_localnet_port_has_learn_fdb' {{(pid=106314) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
May 09 09:04:47.638415 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.agent.neutron_agent [None req-32cb0c34-4c52-4c60-9364-5469adec1ed9 None None] Agents populated in the agent cache: {{(pid=106314) populate /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/agent/neutron_agent.py:310}}
May 09 09:04:47.638662 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_network_broadcast_arps_to_all_routers' {{(pid=106314) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
May 09 09:04:47.643541 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.agent.neutron_agent [None req-32cb0c34-4c52-4c60-9364-5469adec1ed9 None None] - Agent type: OVN Controller Gateway agent, hostname: np0000173033, name: 674084fc-c085-45f6-9e8a-02cd5b8dd8a3 {{(pid=106314) populate /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/agent/neutron_agent.py:312}}
May 09 09:04:47.643893 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_provider_distributed_ports' {{(pid=106314) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
May 09 09:04:47.644187 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.agent.neutron_agent [None req-32cb0c34-4c52-4c60-9364-5469adec1ed9 None None] - Agent type: OVN Neutron agent, hostname: np0000173033, name: 674084fc-c085-45f6-9e8a-02cd5b8dd8a3 {{(pid=106314) populate /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/agent/neutron_agent.py:312}}
May 09 09:04:47.645774 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_redirect_type_router_gateway_ports' {{(pid=106314) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
May 09 09:04:47.646088 np0000173033 neutron-ovn-maintenance-worker[106314]: WARNING neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-32cb0c34-4c52-4c60-9364-5469adec1ed9 None None] New host np0000173033 found in OVN SB DB, but not in Neutron. Add its SegmentHostMapping in Neutron
May 09 09:04:47.646449 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
May 09 09:04:47.648702 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.cleanup_old_hash_ring_nodes' {{(pid=106314) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
May 09 09:04:47.648886 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.configure_nb_global' {{(pid=106314) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
May 09 09:04:47.649416 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.migrate_lrp_gateway_chassis_to_ha_chassis_group' {{(pid=106314) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
May 09 09:04:47.649577 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
May 09 09:04:47.654389 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_network_type_and_physnet' {{(pid=106314) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
May 09 09:04:47.654546 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_ovn_owned_dns_option' {{(pid=106314) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
May 09 09:04:47.654708 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
May 09 09:04:47.654873 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_lrouter_ports_ext_ids_name_prefix' {{(pid=106314) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
May 09 09:04:47.655008 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
May 09 09:04:47.655547 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_router_distributed_flag' {{(pid=106314) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
May 09 09:04:47.655706 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_router_static_routes' {{(pid=106314) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
May 09 09:04:47.657453 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_security_group_with_address_group' {{(pid=106314) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
May 09 09:04:47.657582 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_virtual_port_parent_hostname' {{(pid=106314) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
May 09 09:04:47.658825 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.add_provnet_ext_id_to_localnet_ports' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=106314) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
May 09 09:04:47.659330 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_baremetal_ports_dhcp_options {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:04:47.659839 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Sync baremetal port DHCP options with configuration. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:04:47.719824 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG oslo_concurrency.lockutils [None req-32cb0c34-4c52-4c60-9364-5469adec1ed9 None None] Acquiring lock "update_segment_host_mapping" by "neutron.services.segments.db.update_segment_host_mapping" {{(pid=106314) inner /opt/stack/data/venv/lib/python3.12/site-packages/oslo_concurrency/lockutils.py:506}}
May 09 09:04:47.720368 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG oslo_concurrency.lockutils [None req-32cb0c34-4c52-4c60-9364-5469adec1ed9 None None] Lock "update_segment_host_mapping" acquired by "neutron.services.segments.db.update_segment_host_mapping" :: waited 0.001s {{(pid=106314) inner /opt/stack/data/venv/lib/python3.12/site-packages/oslo_concurrency/lockutils.py:519}}
May 09 09:04:47.735302 np0000173033 neutron-ovn-maintenance-worker[106314]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_baremetal_ports_dhcp_options finished in 0.075 seconds
May 09 09:04:47.738733 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_baremetal_ports_dhcp_options' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=106314) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
May 09 09:04:47.739012 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_dhcp_options_consistency {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:04:47.739170 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Backfill network_id on DHCP_Options external_ids. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:04:47.760668 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron_lib.callbacks.manager [None req-32cb0c34-4c52-4c60-9364-5469adec1ed9 None None] Publish callbacks ['neutron.plugins.ml2.plugin.DhcpAgentSchedulerDbMixin.auto_schedule_new_network_segments-15696736', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_host_addition_to_aggregate-1455598'] for segment_host_mapping (None), after_create {{(pid=106314) _notify_loop /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:184}}
May 09 09:04:47.785639 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.services.segments.db [None req-32cb0c34-4c52-4c60-9364-5469adec1ed9 None None] Segments {'1163f525-70cd-4b84-ae9b-7d56378305e9'} mapped to the host np0000173033 {{(pid=106314) update_segment_host_mapping /opt/stack/neutron/neutron/services/segments/db.py:249}}
May 09 09:04:47.791884 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG oslo_concurrency.lockutils [None req-32cb0c34-4c52-4c60-9364-5469adec1ed9 None None] Lock "update_segment_host_mapping" "released" by "neutron.services.segments.db.update_segment_host_mapping" :: held 0.072s {{(pid=106314) inner /opt/stack/data/venv/lib/python3.12/site-packages/oslo_concurrency/lockutils.py:538}}
May 09 09:04:47.792015 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-32cb0c34-4c52-4c60-9364-5469adec1ed9 None None] OVN-SB Sync hostname and physical networks completed @ 2026-05-09 09:04:47.791859 {{(pid=106314) sync_hostname_and_physical_networks /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1538}}
May 09 09:04:47.811301 np0000173033 neutron-ovn-maintenance-worker[106314]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_dhcp_options_consistency finished in 0.072 seconds
May 09 09:04:47.818253 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_dhcp_options_consistency' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=106314) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
May 09 09:04:47.820965 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fair_meter_consistency {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:04:47.821078 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Refresh ACL logging fair meter after configuration reload. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:04:47.821443 np0000173033 neutron-ovn-maintenance-worker[106314]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fair_meter_consistency finished in 0.000 seconds
May 09 09:04:47.821585 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fair_meter_consistency' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=106314) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
May 09 09:04:47.821774 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:04:47.822005 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:04:47.828209 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:04:47.918769 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:04:47.919384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:04:47.924990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:04:47.924990 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:04:47.924990 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:04:47.924990 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:04:47.924990 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:04:47.924990 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:04:47.924990 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:04:47.924990 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:04:47.931923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.103 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:04:47.931923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:04:47.931923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:04:47.931923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:04:47.931923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:04:47.931923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:04:47.931923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:04:47.931923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:04:47.931923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:04:47.931923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:04:47.931923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:04:47.931923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:04:47.932566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:04:47.932566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:04:47.932566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:04:47.932566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:04:47.932566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:04:47.932566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:04:47.932566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:04:47.932566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:04:47.932566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:04:47.932566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:04:47.932566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:04:47.932566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:04:47.932566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:04:47.933234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:04:47.933234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:04:47.934533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call immediate 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:04:47.934533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:04:47.934533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:04:47.934533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:04:47.934533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:04:47.934533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:04:47.934533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:04:47.934533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:04:47.934533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:04:47.934533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:04:47.934533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:04:47.934533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:04:47.934533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:04:47.934533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:04:47.934533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:04:47.934533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:04:47.935460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:04:47.935460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:04:47.935460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:04:47.935460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:04:47.935460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:04:47.935460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:04:47.935460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:04:47.935460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:04:47.935460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:04:47.935460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:04:47.935460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:04:47.935460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:04:47.935460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:04:47.935460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:04:47.935460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:04:47.935460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:04:47.935460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:04:47.935460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:04:47.935460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:04:47.936309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:04:47.936309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:04:47.936309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:04:47.936309 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_for_igmp_snoop_support {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:04:47.941233 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Check for IGMP snoop support. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:04:47.941848 np0000173033 neutron-ovn-maintenance-worker[106314]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_for_igmp_snoop_support finished in 0.000 seconds
May 09 09:04:47.942027 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_for_igmp_snoop_support' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=106314) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
May 09 09:04:47.942208 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_for_inconsistencies {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:04:47.942321 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Checking Neutron and OVN revision consistency. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:04:47.988598 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Maintenance task: No inconsistencies found. Skipping {{(pid=106314) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:437}}
May 09 09:04:47.989430 np0000173033 neutron-ovn-maintenance-worker[106314]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_for_inconsistencies finished in 0.047 seconds
May 09 09:04:47.992074 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_global_dhcp_opts {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:04:47.992408 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Check global DHCP options consistency. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:04:47.992719 np0000173033 neutron-ovn-maintenance-worker[106314]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_global_dhcp_opts finished in 0.000 seconds
May 09 09:04:47.992977 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_global_dhcp_opts' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=106314) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
May 09 09:04:47.993205 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_localnet_port_has_learn_fdb {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:04:47.995844 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Ensure localnet ports have learn-fdb set per configuration. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:04:47.997321 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] There are no commands to commit {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:78}}
May 09 09:04:48.001006 np0000173033 neutron-ovn-maintenance-worker[106314]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_localnet_port_has_learn_fdb finished in 0.005 seconds
May 09 09:04:48.001327 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_localnet_port_has_learn_fdb' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=106314) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
May 09 09:04:48.001572 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_network_broadcast_arps_to_all_routers {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:04:48.001772 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Sync broadcast-arps-to-all-routers on external networks. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:04:48.120685 np0000173033 neutron-ovn-maintenance-worker[106314]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_network_broadcast_arps_to_all_routers finished in 0.114 seconds
May 09 09:04:48.120995 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_network_broadcast_arps_to_all_routers' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=106314) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
May 09 09:04:48.121191 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_provider_distributed_ports {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:04:48.121357 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Align reside-on-redirect-chassis on VLAN/FLAT distributed ports. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:04:48.215261 np0000173033 neutron-ovn-maintenance-worker[106314]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_provider_distributed_ports finished in 0.094 seconds
May 09 09:04:48.215261 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_provider_distributed_ports' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=106314) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
May 09 09:04:48.215261 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_redirect_type_router_gateway_ports {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:04:48.215261 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Align router gateway redirect-type for provider VLAN/FLAT networks. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:04:48.325187 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] There are no commands to commit {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:78}}
May 09 09:04:48.326782 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-32cb0c34-4c52-4c60-9364-5469adec1ed9 None None] OVN-Southbound DB sync process completed @ 2026-05-09 09:04:48.326508 {{(pid=106314) do_sync /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1488}}
May 09 09:04:48.339784 np0000173033 neutron-ovn-maintenance-worker[106314]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_redirect_type_router_gateway_ports finished in 0.124 seconds
May 09 09:04:48.340127 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_redirect_type_router_gateway_ports' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=106314) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
May 09 09:04:48.340127 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:04:48.340272 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:04:48.341634 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:04:48.342035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:04:48.342919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:04:48.342919 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:04:48.342919 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:04:48.342919 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:04:48.342919 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:04:48.342919 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:04:48.342919 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:04:48.342919 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:04:48.343862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:04:48.343862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:04:48.343862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:04:48.343862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:04:48.343862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:04:48.343862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:04:48.343862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:04:48.343862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:04:48.343862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:04:48.343862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:04:48.343862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:04:48.343862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:04:48.344681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:04:48.344681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:04:48.344681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:04:48.344681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:04:48.344681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:04:48.344681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:04:48.344681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:04:48.344681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:04:48.344681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:04:48.344681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:04:48.344681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:04:48.344681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:04:48.344681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:04:48.345551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:04:48.345551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:04:48.345551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call immediate 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:04:48.345551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:04:48.345551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:04:48.345551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:04:48.345551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:04:48.345551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:04:48.345551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:04:48.345551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:04:48.345551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:04:48.345551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:04:48.345551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:04:48.345551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:04:48.345551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:04:48.346612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:04:48.346612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:04:48.346612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:04:48.346612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:04:48.346612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:04:48.346612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:04:48.346612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:04:48.346612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:04:48.346612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:04:48.346612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:04:48.346612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:04:48.346612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:04:48.346612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:04:48.346612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:04:48.346612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:04:48.346612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:04:48.346612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:04:48.346612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:04:48.346612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:04:48.347788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:04:48.347788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:04:48.347788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:04:48.347788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:04:48.347788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:04:48.347788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:04:48.347788 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: cleanup_old_hash_ring_nodes {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:04:48.347788 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Purge stale hash ring nodes older than five days. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:04:48.350168 np0000173033 neutron-ovn-maintenance-worker[106314]: INFO neutron.db.ovn_hash_ring_db [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Cleaned up Hash Ring nodes older than 5 days
May 09 09:04:48.350308 np0000173033 neutron-ovn-maintenance-worker[106314]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task cleanup_old_hash_ring_nodes finished in 0.005 seconds
May 09 09:04:48.350572 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: configure_nb_global {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:04:48.350661 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply ovn_nb_global settings to NB_Global options. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:04:48.351009 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'ignore_lsp_down': 'false', 'fdb_removal_limit': '0', 'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:04:48.351618 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'ignore_lsp_down': 'false', 'fdb_removal_limit': '0', 'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:04:48.352083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:04:48.353018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:04:48.353018 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:04:48.353018 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:04:48.353018 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:04:48.353018 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:04:48.353018 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:04:48.353018 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:04:48.353018 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:04:48.354159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:04:48.354159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:04:48.354159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:04:48.354159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:04:48.354159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:04:48.354159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:04:48.354159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:04:48.354159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:04:48.354159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:04:48.354159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:04:48.354159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:04:48.354159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:04:48.354159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:04:48.354159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:04:48.354159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:04:48.354996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:04:48.354996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:04:48.354996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:04:48.354996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:04:48.354996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:04:48.354996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:04:48.354996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:04:48.354996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:04:48.354996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:04:48.355567 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task configure_nb_global failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:04:48.355567 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:04:48.355567 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:04:48.355567 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:04:48.355567 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:04:48.355567 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1038, in configure_nb_global
May 09 09:04:48.355567 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:04:48.355567 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:04:48.355567 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:04:48.355567 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:04:48.355567 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:04:48.356152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:04:48.356152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:04:48.356152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:04:48.356152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:04:48.356152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:04:48.356152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:04:48.356152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:04:48.356152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:04:48.356152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:04:48.356152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:04:48.356152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:04:48.356152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:04:48.356152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:04:48.356152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:04:48.356836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:04:48.356836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:04:48.356836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:04:48.356836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:04:48.357046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call immediate 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.configure_nb_global' (it runs every 86400.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:04:48.357046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:04:48.357046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:04:48.357046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:04:48.357046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:04:48.357046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:04:48.357046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:04:48.357046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:04:48.357046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:04:48.357046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:04:48.357046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:04:48.357046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:04:48.357046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:04:48.357046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:04:48.357046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:04:48.357046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:04:48.357791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1038, in configure_nb_global
May 09 09:04:48.357791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:04:48.357791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:04:48.357791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:04:48.357791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:04:48.357791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:04:48.357791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:04:48.357791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:04:48.357791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:04:48.357791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:04:48.357791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:04:48.357791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:04:48.357791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:04:48.357791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:04:48.357791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:04:48.357791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:04:48.357791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:04:48.357791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:04:48.357791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:04:48.358902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:04:48.358902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:04:48.358902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:04:48.358902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:04:48.358902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:04:48.358902 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] There are no commands to commit {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:78}}
May 09 09:04:48.358902 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.migrate_lrp_gateway_chassis_to_ha_chassis_group' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=106314) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
May 09 09:04:48.358902 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:04:48.358902 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:04:48.359426 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:04:48.359909 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:04:48.360208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:04:48.361139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:04:48.361139 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:04:48.361139 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:04:48.361139 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:04:48.361139 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:04:48.361139 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:04:48.361139 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:04:48.361139 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:04:48.362457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:04:48.362457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:04:48.362457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:04:48.362457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:04:48.362457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:04:48.362457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:04:48.362457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:04:48.362457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:04:48.362457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:04:48.362457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:04:48.362457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:04:48.362457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:04:48.362457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:04:48.362457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:04:48.362457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:04:48.362910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:04:48.362910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:04:48.362910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:04:48.362910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:04:48.362910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:04:48.362910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:04:48.362910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:04:48.362910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:04:48.362910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:04:48.363835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:04:48.363835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:04:48.363835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:04:48.363835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:04:48.363835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:04:48.363835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:04:48.363835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:04:48.363835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:04:48.363835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:04:48.363835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:04:48.363835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:04:48.363835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:04:48.364205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:04:48.364205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:04:48.364205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:04:48.364205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:04:48.364205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:04:48.364205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:04:48.364205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:04:48.364205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:04:48.364205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:04:48.364205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:04:48.364205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:04:48.364205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:04:48.364205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:04:48.364205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:04:48.364602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:04:48.364602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:04:48.364602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:04:48.364602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:04:48.365506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call immediate 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:04:48.365506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:04:48.365506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:04:48.365506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:04:48.365506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:04:48.365506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:04:48.365506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:04:48.365506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:04:48.365506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:04:48.365506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:04:48.365506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:04:48.365506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:04:48.365506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:04:48.365506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:04:48.365506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:04:48.365506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:04:48.365982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:04:48.365982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:04:48.365982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:04:48.365982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:04:48.365982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:04:48.365982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:04:48.365982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:04:48.365982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:04:48.365982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:04:48.365982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:04:48.365982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:04:48.365982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:04:48.365982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:04:48.365982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:04:48.365982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:04:48.365982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:04:48.365982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:04:48.365982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:04:48.366496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:04:48.366496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:04:48.366496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:04:48.366496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:04:48.366496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:04:48.366496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:04:48.366496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:04:48.366496 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_network_type_and_physnet {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:04:48.366496 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set network type and physnet external_ids on logical switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:04:48.373154 np0000173033 neutron-ovn-maintenance-worker[106314]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_network_type_and_physnet finished in 0.007 seconds
May 09 09:04:48.373269 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_network_type_and_physnet' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=106314) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
May 09 09:04:48.373402 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_ovn_owned_dns_option {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:04:48.373485 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply ovn_owned option on DNS records per configuration. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:04:48.373706 np0000173033 neutron-ovn-maintenance-worker[106314]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_ovn_owned_dns_option finished in 0.000 seconds
May 09 09:04:48.373797 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_ovn_owned_dns_option' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=106314) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
May 09 09:04:48.373912 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:04:48.373993 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:04:48.374159 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:04:48.374856 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:04:48.375278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:04:48.376051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:04:48.376051 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:04:48.376051 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:04:48.376051 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:04:48.376051 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:04:48.376051 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:04:48.376051 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:04:48.376051 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:04:48.376702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:04:48.376702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:04:48.376702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:04:48.376702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:04:48.376702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:04:48.376702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:04:48.376702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:04:48.376702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:04:48.376702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:04:48.376702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:04:48.376702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:04:48.376702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:04:48.376702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:04:48.376702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:04:48.376702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:04:48.377224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:04:48.377224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:04:48.377224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:04:48.377224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:04:48.377224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:04:48.377224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:04:48.377224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:04:48.377224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:04:48.377224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:04:48.377514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:04:48.377514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:04:48.377514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:04:48.377514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:04:48.377514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:04:48.377514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:04:48.377514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:04:48.377514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:04:48.377514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:04:48.377514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:04:48.377514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:04:48.378059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:04:48.378059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:04:48.378059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:04:48.378059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:04:48.378059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:04:48.378059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:04:48.378059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:04:48.378059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:04:48.378059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:04:48.378059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:04:48.378059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:04:48.378059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:04:48.378059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:04:48.378059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:04:48.378474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:04:48.378474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:04:48.378474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:04:48.378474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:04:48.378616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call immediate 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:04:48.378616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:04:48.378616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:04:48.378616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:04:48.378616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:04:48.378616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:04:48.378616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:04:48.378616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:04:48.378616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:04:48.378616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:04:48.378616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:04:48.378616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:04:48.378616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:04:48.378616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:04:48.378616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:04:48.378616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:04:48.378616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:04:48.379210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:04:48.379210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:04:48.379210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:04:48.379210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:04:48.379210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:04:48.379210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:04:48.379210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:04:48.379210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:04:48.379210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:04:48.379210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:04:48.379210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:04:48.379210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:04:48.379210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:04:48.379210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:04:48.379210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:04:48.379210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:04:48.379210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:04:48.379210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:04:48.379210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:04:48.380067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:04:48.380067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:04:48.380067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:04:48.380067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:04:48.380067 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_lrouter_ports_ext_ids_name_prefix {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:04:48.380067 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: ('Check missing prefix router_name', 'in external_ids of LRPs.') {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:04:48.380067 np0000173033 neutron-ovn-maintenance-worker[106314]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_lrouter_ports_ext_ids_name_prefix finished in 0.000 seconds
May 09 09:04:48.380067 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_lrouter_ports_ext_ids_name_prefix' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=106314) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
May 09 09:04:48.380808 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:04:48.380808 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:04:48.380808 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:04:48.381201 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:04:48.381635 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:04:48.381874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:04:48.382595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:04:48.382595 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:04:48.382595 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:04:48.382595 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:04:48.382595 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:04:48.382595 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:04:48.382595 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:04:48.382595 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:04:48.383430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:04:48.383430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:04:48.383430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:04:48.383430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:04:48.383430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:04:48.383430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:04:48.383430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:04:48.383430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:04:48.383430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:04:48.383430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:04:48.383430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:04:48.383430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:04:48.383874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:04:48.383874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:04:48.383874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:04:48.383874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:04:48.383874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:04:48.383874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:04:48.383874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:04:48.383874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:04:48.383874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:04:48.383874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:04:48.383874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:04:48.383874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:04:48.383874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:04:48.384342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:04:48.384342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:04:48.384583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call immediate 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:04:48.384583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:04:48.384583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:04:48.384583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:04:48.384583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:04:48.384583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:04:48.384583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:04:48.384583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:04:48.384583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:04:48.384583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:04:48.384583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:04:48.384583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:04:48.384583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:04:48.384583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:04:48.384583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:04:48.384583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:04:48.385097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:04:48.385097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:04:48.385097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:04:48.385097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:04:48.385097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:04:48.385097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:04:48.385097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:04:48.385097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:04:48.385097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:04:48.385097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:04:48.385097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:04:48.385097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:04:48.385097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:04:48.385097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:04:48.385097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:04:48.385097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:04:48.385097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:04:48.385097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:04:48.385097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:04:48.385666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:04:48.385666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:04:48.385666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:04:48.385666 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_router_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:04:48.385666 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Sync router.distributed with distributed floating IP config. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:04:48.389306 np0000173033 neutron-ovn-maintenance-worker[106314]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_router_distributed_flag finished in 0.004 seconds
May 09 09:04:48.389471 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_router_distributed_flag' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=106314) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
May 09 09:04:48.389608 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_router_static_routes {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:04:48.389712 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Mark Neutron-owned static routes in OVN external_ids. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:04:48.390656 np0000173033 neutron-ovn-maintenance-worker[106314]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_router_static_routes finished in 0.001 seconds
May 09 09:04:48.390819 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_router_static_routes' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=106314) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
May 09 09:04:48.390976 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_security_group_with_address_group {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:04:48.391078 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Repair Address_Sets and ACLs for address group rules. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:04:48.403340 np0000173033 neutron-ovn-maintenance-worker[106314]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_security_group_with_address_group finished in 0.012 seconds
May 09 09:04:48.403710 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_security_group_with_address_group' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=106314) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
May 09 09:04:48.405370 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_virtual_port_parent_hostname' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=106314) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
May 09 09:04:49.893723 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched CREATE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(uuid=d99935a6-0176-4236-a9d6-81a74523bba7, enabled=[], external_ids={'neutron:is_ext_gw': 'False', 'neutron:network_name': 'neutron-f8a38fcb-e8a2-421a-a340-bf7f2f5cc72b', 'neutron:revision_number': '1', 'neutron:router_name': 'neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1', 'neutron:subnet_ids': 'bbd6957a-953f-4fc1-8761-5d93128564d9'}, gateway_chassis=[], ha_chassis_group=[], ipv6_prefix=[], ipv6_ra_configs={'address_mode': 'slaac', 'mtu': '1372', 'send_periodic': 'true'}, mac='fa:16:3e:75:8a:e2', name='lrp-b97f77b4-9a3e-462a-89dd-07c6397abd6e', networks=['fdcf:9de2:99bf::1/64'], options={}, peer=[], status={}) old= {{(pid=106314) matches /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}}
May 09 09:04:49.895318 np0000173033 neutron-ovn-maintenance-worker[106314]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.extensions.qos [-] Starting OVNClientQosExtension
May 09 09:04:49.938847 np0000173033 neutron-ovn-maintenance-worker[106314]: INFO neutron.common.ovn.utils [-] HA Chassis Group neutron-f8a38fcb-e8a2-421a-a340-bf7f2f5cc72b synchronized; highest priority chassis 674084fc-c085-45f6-9e8a-02cd5b8dd8a3
May 09 09:04:49.939681 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=HA_Chassis_Group, record=neutron-f8a38fcb-e8a2-421a-a340-bf7f2f5cc72b, col_values=(('external_ids', {'neutron:availability_zone_hints': '', 'neutron:network_id': 'f8a38fcb-e8a2-421a-a340-bf7f2f5cc72b', 'neutron:router_id': 'a766a392-b08a-4e6f-abb6-a919e8e71db1'}),), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:04:49.940148 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): HAChassisGroupAddChassisCommand(_result=None, hcg=neutron-f8a38fcb-e8a2-421a-a340-bf7f2f5cc72b, chassis=674084fc-c085-45f6-9e8a-02cd5b8dd8a3, priority=1, columns={}) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:04:49.940459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:04:49.941280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
May 09 09:04:49.941280 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:04:49.941280 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:04:49.941280 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:04:49.941280 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:04:49.941280 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:04:49.941280 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:04:49.941280 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:04:49.943292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.event [-] Unexpected exception in notify_loop: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:04:49.943292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.event Traceback (most recent call last):
May 09 09:04:49.943292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/event.py", line 185, in notify_loop
May 09 09:04:49.943292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.event     match.run(event, row, updates)
May 09 09:04:49.943292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.event   File "/opt/stack/neutron/neutron/services/ovn_l3/ovsdb_monitor.py", line 86, in run
May 09 09:04:49.943292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.event     self.l3_plugin._ovn_client.link_network_ha_chassis_group(
May 09 09:04:49.943292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.event   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2416, in link_network_ha_chassis_group
May 09 09:04:49.943292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.event     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:04:49.943292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.event   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:04:49.943292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.event     next(self.gen)
May 09 09:04:49.943292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.event   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:04:49.943292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.event     with super().transaction(*args, **kwargs) as t:
May 09 09:04:49.943292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.event   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:04:49.943292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.event     next(self.gen)
May 09 09:04:49.943292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:04:49.943292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.event     with self.create_transaction(
May 09 09:04:49.943292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:04:49.943292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.event     self.result = self.commit()
May 09 09:04:49.943292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.event                   ^^^^^^^^^^^^^
May 09 09:04:49.943988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:04:49.943988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.event     raise result.ex
May 09 09:04:49.943988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:04:49.943988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.event     txn.results.put(txn.do_commit())
May 09 09:04:49.943988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.event                     ^^^^^^^^^^^^^^^
May 09 09:04:49.943988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:04:49.943988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.event     raise RuntimeError(msg)
May 09 09:04:49.943988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.event RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:04:49.943988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.event 
May 09 09:04:51.833237 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched CREATE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(uuid=43c33750-8195-467b-bb58-0c4bab844bb9, enabled=[], external_ids={'neutron:is_ext_gw': 'True', 'neutron:network_name': 'neutron-d18a9fac-2ebf-4d1e-a86c-0f33fba8eeda', 'neutron:revision_number': '8', 'neutron:router_name': 'neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1', 'neutron:subnet_ids': 'bbd6a8cc-80e6-4336-a70b-1c962c561590 088bbd0f-7e81-4a76-8399-24d15c984009'}, gateway_chassis=[], ha_chassis_group=900d3c10-79c7-465f-a88f-7d5002e346cb, ipv6_prefix=[], ipv6_ra_configs={}, mac='fa:16:3e:cd:93:fb', name='lrp-c729386b-b745-4cf6-b460-5cf7f6adbd23', networks=['172.24.5.120/24', '2001:db8::1/64'], options={'gateway_mtu': '1372', 'reside-on-redirect-chassis': 'true'}, peer=[], status={}) old= {{(pid=106314) matches /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}}
May 09 09:04:51.835068 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched DELETE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(uuid=af490d31-0757-4e81-b1d2-6f4595484e79, enabled=[], external_ids={'neutron:is_ext_gw': 'True', 'neutron:network_name': 'neutron-d18a9fac-2ebf-4d1e-a86c-0f33fba8eeda', 'neutron:revision_number': '1', 'neutron:router_name': 'neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1', 'neutron:subnet_ids': 'bbd6a8cc-80e6-4336-a70b-1c962c561590'}, gateway_chassis=[], ha_chassis_group=900d3c10-79c7-465f-a88f-7d5002e346cb, ipv6_prefix=[], ipv6_ra_configs={}, mac='fa:16:3e:cd:93:fb', name='lrp-c729386b-b745-4cf6-b460-5cf7f6adbd23', networks=['172.24.5.120/24'], options={'gateway_mtu': '1372', 'reside-on-redirect-chassis': 'true'}, peer=[], status={'hosting-chassis': '674084fc-c085-45f6-9e8a-02cd5b8dd8a3'}) old= {{(pid=106314) matches /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}}
May 09 09:04:51.877245 np0000173033 neutron-ovn-maintenance-worker[106314]: INFO neutron.common.ovn.utils [-] HA Chassis Group neutron-f8a38fcb-e8a2-421a-a340-bf7f2f5cc72b synchronized; highest priority chassis 674084fc-c085-45f6-9e8a-02cd5b8dd8a3
May 09 09:04:51.877858 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=HA_Chassis_Group, record=neutron-f8a38fcb-e8a2-421a-a340-bf7f2f5cc72b, col_values=(('external_ids', {'neutron:availability_zone_hints': '', 'neutron:network_id': 'f8a38fcb-e8a2-421a-a340-bf7f2f5cc72b', 'neutron:router_id': 'a766a392-b08a-4e6f-abb6-a919e8e71db1'}),), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:04:51.878207 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): HAChassisGroupAddChassisCommand(_result=None, hcg=neutron-f8a38fcb-e8a2-421a-a340-bf7f2f5cc72b, chassis=674084fc-c085-45f6-9e8a-02cd5b8dd8a3, priority=1, columns={}) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:04:51.878460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:04:51.879166 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
May 09 09:04:51.879166 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:04:51.879166 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:04:51.879166 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:04:51.879166 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:04:51.879166 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:04:51.879166 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:04:51.879166 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:04:51.880343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.event [-] Unexpected exception in notify_loop: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:04:51.880343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.event Traceback (most recent call last):
May 09 09:04:51.880343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/event.py", line 185, in notify_loop
May 09 09:04:51.880343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.event     match.run(event, row, updates)
May 09 09:04:51.880343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.event   File "/opt/stack/neutron/neutron/services/ovn_l3/ovsdb_monitor.py", line 89, in run
May 09 09:04:51.880343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.event     self.l3_plugin._ovn_client.update_router_ha_chassis_group(
May 09 09:04:51.880343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.event   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1665, in update_router_ha_chassis_group
May 09 09:04:51.880343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.event     self.link_network_ha_chassis_group(context, network_id, router_id)
May 09 09:04:51.880343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.event   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2416, in link_network_ha_chassis_group
May 09 09:04:51.880343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.event     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:04:51.880343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.event   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:04:51.880343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.event     next(self.gen)
May 09 09:04:51.880343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.event   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:04:51.880343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.event     with super().transaction(*args, **kwargs) as t:
May 09 09:04:51.880343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.event   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:04:51.880343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.event     next(self.gen)
May 09 09:04:51.880343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:04:51.880343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.event     with self.create_transaction(
May 09 09:04:51.881092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:04:51.881092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.event     self.result = self.commit()
May 09 09:04:51.881092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.event                   ^^^^^^^^^^^^^
May 09 09:04:51.881092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:04:51.881092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.event     raise result.ex
May 09 09:04:51.881092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:04:51.881092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.event     txn.results.put(txn.do_commit())
May 09 09:04:51.881092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.event                     ^^^^^^^^^^^^^^^
May 09 09:04:51.881092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:04:51.881092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.event     raise RuntimeError(msg)
May 09 09:04:51.881092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.event RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:04:51.881092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.event 
May 09 09:04:51.908888 np0000173033 neutron-ovn-maintenance-worker[106314]: INFO neutron.common.ovn.utils [-] HA Chassis Group neutron-f8a38fcb-e8a2-421a-a340-bf7f2f5cc72b synchronized; highest priority chassis 674084fc-c085-45f6-9e8a-02cd5b8dd8a3
May 09 09:04:51.909437 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=HA_Chassis_Group, record=neutron-f8a38fcb-e8a2-421a-a340-bf7f2f5cc72b, col_values=(('external_ids', {'neutron:availability_zone_hints': '', 'neutron:network_id': 'f8a38fcb-e8a2-421a-a340-bf7f2f5cc72b', 'neutron:router_id': 'a766a392-b08a-4e6f-abb6-a919e8e71db1'}),), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:04:51.909851 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): HAChassisGroupAddChassisCommand(_result=None, hcg=neutron-f8a38fcb-e8a2-421a-a340-bf7f2f5cc72b, chassis=674084fc-c085-45f6-9e8a-02cd5b8dd8a3, priority=1, columns={}) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:04:51.910121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:04:51.910781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
May 09 09:04:51.910781 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:04:51.910781 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:04:51.910781 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:04:51.910781 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:04:51.910781 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:04:51.910781 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:04:51.910781 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:04:51.911624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.event [-] Unexpected exception in notify_loop: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:04:51.911624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.event Traceback (most recent call last):
May 09 09:04:51.911624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/event.py", line 185, in notify_loop
May 09 09:04:51.911624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.event     match.run(event, row, updates)
May 09 09:04:51.911624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.event   File "/opt/stack/neutron/neutron/services/ovn_l3/ovsdb_monitor.py", line 81, in run
May 09 09:04:51.911624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.event     self.l3_plugin._ovn_client.update_router_ha_chassis_group(
May 09 09:04:51.911624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.event   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1665, in update_router_ha_chassis_group
May 09 09:04:51.911624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.event     self.link_network_ha_chassis_group(context, network_id, router_id)
May 09 09:04:51.911624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.event   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2416, in link_network_ha_chassis_group
May 09 09:04:51.911624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.event     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:04:51.911624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.event   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:04:51.911624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.event     next(self.gen)
May 09 09:04:51.911624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.event   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:04:51.911624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.event     with super().transaction(*args, **kwargs) as t:
May 09 09:04:51.911624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.event   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:04:51.911624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.event     next(self.gen)
May 09 09:04:51.911624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:04:51.911624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.event     with self.create_transaction(
May 09 09:04:51.912620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:04:51.912620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.event     self.result = self.commit()
May 09 09:04:51.912620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.event                   ^^^^^^^^^^^^^
May 09 09:04:51.912620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:04:51.912620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.event     raise result.ex
May 09 09:04:51.912620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:04:51.912620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.event     txn.results.put(txn.do_commit())
May 09 09:04:51.912620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.event                     ^^^^^^^^^^^^^^^
May 09 09:04:51.912620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:04:51.912620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.event     raise RuntimeError(msg)
May 09 09:04:51.912620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.event RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:04:51.912620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.event 
May 09 09:04:52.822199 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:04:52.823076 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:04:52.823183 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:04:52.824214 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:04:52.883326 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:04:52.884638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:04:52.884638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:04:52.884638 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:04:52.884638 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:04:52.884638 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:04:52.884638 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:04:52.884638 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:04:52.884638 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:04:52.884638 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:04:52.885696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.061 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:04:52.885696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:04:52.885696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:04:52.885696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:04:52.885696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:04:52.885696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:04:52.885696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:04:52.885696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:04:52.885696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:04:52.885696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:04:52.885696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:04:52.885696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:04:52.886377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:04:52.886377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:04:52.886377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:04:52.886377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:04:52.886377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:04:52.886377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:04:52.886377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:04:52.886377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:04:52.886377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:04:52.886377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:04:52.886377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:04:52.886377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:04:52.886377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:04:52.887142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:04:52.887142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:04:52.887142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:04:52.887142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:04:52.887142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:04:52.887142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:04:52.887142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:04:52.887142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:04:52.887142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:04:52.887142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:04:52.887142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:04:52.887142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:04:52.887142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:04:52.887142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:04:52.887142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:04:52.888108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:04:52.888108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:04:52.888108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:04:52.888108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:04:52.888108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:04:52.888108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:04:52.888108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:04:52.888108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:04:52.888108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:04:52.888108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:04:52.888108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:04:52.888108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:04:52.888108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:04:52.888108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:04:52.888108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:04:52.888108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:04:52.888108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:04:52.888108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:04:52.888108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:04:52.889091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:04:52.889091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:04:52.889091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:04:52.889091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:04:52.889091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:04:52.889091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:04:53.340642 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:04:53.341553 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:04:53.341779 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:04:53.343022 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:04:53.343442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:04:53.344418 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:04:53.344418 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:04:53.344418 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:04:53.344418 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:04:53.344418 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:04:53.344418 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:04:53.344418 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:04:53.344418 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:04:53.346172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:04:53.346172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:04:53.346172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:04:53.346172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:04:53.346172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:04:53.346172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:04:53.346172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:04:53.346172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:04:53.346172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:04:53.346172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:04:53.346172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:04:53.346172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:04:53.346726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:04:53.346726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:04:53.346726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:04:53.346726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:04:53.346726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:04:53.346726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:04:53.346726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:04:53.346726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:04:53.346726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:04:53.346726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:04:53.346726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:04:53.346726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:04:53.346726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:04:53.347353 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:04:53.347353 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:04:53.348102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:04:53.348102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:04:53.348102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:04:53.348102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:04:53.348102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:04:53.348102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:04:53.348102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:04:53.348102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:04:53.348102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:04:53.348102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:04:53.348102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:04:53.348102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:04:53.348102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:04:53.348102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:04:53.348102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:04:53.348102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:04:53.349052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:04:53.349052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:04:53.349052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:04:53.349052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:04:53.349052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:04:53.349052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:04:53.349052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:04:53.349052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:04:53.349052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:04:53.349052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:04:53.349052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:04:53.349052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:04:53.349052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:04:53.349052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:04:53.349052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:04:53.349052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:04:53.349052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:04:53.349052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:04:53.349052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:04:53.349891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:04:53.349891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:04:53.349891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:04:53.358815 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:04:53.359157 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:04:53.359271 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:04:53.359440 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:04:53.360088 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:04:53.360412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:04:53.361251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:04:53.361251 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:04:53.361251 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:04:53.361251 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:04:53.361251 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:04:53.361251 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:04:53.361251 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:04:53.361251 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:04:53.362393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:04:53.362393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:04:53.362393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:04:53.362393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:04:53.362393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:04:53.362393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:04:53.362393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:04:53.362393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:04:53.362393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:04:53.362393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:04:53.362393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:04:53.362393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:04:53.362393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:04:53.362393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:04:53.362393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:04:53.362897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:04:53.362897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:04:53.362897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:04:53.362897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:04:53.362897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:04:53.362897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:04:53.362897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:04:53.362897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:04:53.362897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:04:53.363374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:04:53.363374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:04:53.363374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:04:53.363374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:04:53.363374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:04:53.363374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:04:53.363374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:04:53.363374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:04:53.363374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:04:53.363374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:04:53.363374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:04:53.363374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:04:53.363712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:04:53.363712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:04:53.363712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:04:53.363712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:04:53.363712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:04:53.363712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:04:53.363712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:04:53.363712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:04:53.363712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:04:53.363712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:04:53.363712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:04:53.363712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:04:53.363712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:04:53.363712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:04:53.364095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:04:53.364095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:04:53.364095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:04:53.364095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:04:53.364517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:04:53.364517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:04:53.364517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:04:53.364517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:04:53.364517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:04:53.364517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:04:53.364517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:04:53.364517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:04:53.364517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:04:53.364517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:04:53.364517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:04:53.364517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:04:53.364517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:04:53.364517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:04:53.364517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:04:53.364517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:04:53.364979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:04:53.364979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:04:53.364979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:04:53.364979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:04:53.364979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:04:53.364979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:04:53.364979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:04:53.364979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:04:53.364979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:04:53.364979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:04:53.364979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:04:53.364979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:04:53.364979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:04:53.364979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:04:53.364979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:04:53.364979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:04:53.364979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:04:53.364979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:04:53.365452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:04:53.365452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:04:53.365452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:04:53.365452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:04:53.365452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:04:53.365452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:04:53.365452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:04:53.374126 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:04:53.374388 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:04:53.374523 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:04:53.374699 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:04:53.375280 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:04:53.375647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:04:53.376471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:04:53.376471 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:04:53.376471 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:04:53.376471 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:04:53.376471 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:04:53.376471 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:04:53.376471 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:04:53.376471 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:04:53.377417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:04:53.377417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:04:53.377417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:04:53.377417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:04:53.377417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:04:53.377417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:04:53.377417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:04:53.377417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:04:53.377417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:04:53.377417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:04:53.377417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:04:53.377417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:04:53.377417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:04:53.377417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:04:53.377417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:04:53.377893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:04:53.377893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:04:53.377893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:04:53.377893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:04:53.377893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:04:53.377893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:04:53.377893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:04:53.377893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:04:53.377893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:04:53.378466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:04:53.378466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:04:53.378466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:04:53.378466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:04:53.378466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:04:53.378466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:04:53.378466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:04:53.378466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:04:53.378466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:04:53.378466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:04:53.378466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:04:53.378864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:04:53.378864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:04:53.378864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:04:53.378864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:04:53.378864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:04:53.378864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:04:53.378864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:04:53.378864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:04:53.378864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:04:53.378864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:04:53.378864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:04:53.378864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:04:53.378864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:04:53.378864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:04:53.379246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:04:53.379246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:04:53.379246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:04:53.379246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:04:53.379778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:04:53.379778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:04:53.379778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:04:53.379778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:04:53.379778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:04:53.379778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:04:53.379778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:04:53.379778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:04:53.379778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:04:53.379778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:04:53.379778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:04:53.379778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:04:53.379778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:04:53.379778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:04:53.379778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:04:53.379778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:04:53.379778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:04:53.380515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:04:53.380515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:04:53.380515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:04:53.380515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:04:53.380515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:04:53.380515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:04:53.380515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:04:53.380515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:04:53.380515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:04:53.380515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:04:53.380515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:04:53.380515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:04:53.380515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:04:53.380515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:04:53.380515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:04:53.380515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:04:53.380515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:04:53.380515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:04:53.380515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:04:53.381364 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:04:53.381364 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:04:53.381364 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:04:53.381364 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:04:53.381364 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:04:53.381364 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:04:53.381364 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:04:53.381364 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:04:53.381855 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:04:53.381855 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:04:53.382102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:04:53.383028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:04:53.383028 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:04:53.383028 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:04:53.383028 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:04:53.383028 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:04:53.383028 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:04:53.383028 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:04:53.383028 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:04:53.384018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:04:53.384018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:04:53.384018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:04:53.384018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:04:53.384018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:04:53.384018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:04:53.384018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:04:53.384018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:04:53.384018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:04:53.384018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:04:53.384018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:04:53.384018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:04:53.384528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:04:53.384528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:04:53.384528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:04:53.384528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:04:53.384528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:04:53.384528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:04:53.384528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:04:53.384528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:04:53.384528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:04:53.384528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:04:53.384528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:04:53.384528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:04:53.384528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:04:53.385062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:04:53.385062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:04:53.385163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:04:53.385163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:04:53.385163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:04:53.385163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:04:53.385163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:04:53.385163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:04:53.385163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:04:53.385163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:04:53.385163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:04:53.385163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:04:53.385163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:04:53.385163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:04:53.385163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:04:53.385163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:04:53.385163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:04:53.385163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:04:53.385927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:04:53.385927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:04:53.385927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:04:53.385927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:04:53.385927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:04:53.385927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:04:53.385927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:04:53.385927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:04:53.385927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:04:53.385927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:04:53.385927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:04:53.385927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:04:53.385927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:04:53.385927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:04:53.385927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:04:53.385927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:04:53.385927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:04:53.385927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:04:53.385927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:04:53.386765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:04:53.386765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:04:53.386765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:04:57.823923 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:04:57.824691 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:04:57.824806 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:04:57.826016 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:04:58.249135 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:04:58.249617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:04:58.250592 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:04:58.250592 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:04:58.250592 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:04:58.250592 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:04:58.250592 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:04:58.250592 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:04:58.250592 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:04:58.250592 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:04:58.251506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.426 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:04:58.251506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:04:58.251506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:04:58.251506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:04:58.251506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:04:58.251506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:04:58.251506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:04:58.251506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:04:58.251506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:04:58.251506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:04:58.251506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:04:58.251506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:04:58.251905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:04:58.251905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:04:58.251905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:04:58.251905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:04:58.251905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:04:58.251905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:04:58.251905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:04:58.251905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:04:58.251905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:04:58.251905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:04:58.251905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:04:58.251905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:04:58.251905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:04:58.252552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:04:58.252552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:04:58.252552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:04:58.252552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:04:58.252552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:04:58.252552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:04:58.252552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:04:58.252552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:04:58.252552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:04:58.252552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:04:58.252552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:04:58.252552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:04:58.252552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:04:58.252552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:04:58.252552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:04:58.253219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:04:58.253219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:04:58.253219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:04:58.253219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:04:58.253219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:04:58.253219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:04:58.253219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:04:58.253219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:04:58.253219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:04:58.253219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:04:58.253219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:04:58.253219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:04:58.253219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:04:58.253219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:04:58.253219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:04:58.253219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:04:58.253219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:04:58.253219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:04:58.253219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:04:58.253919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:04:58.253919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:04:58.253919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:04:58.253919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:04:58.253919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:04:58.253919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:04:58.341356 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:04:58.342290 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:04:58.342458 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:04:58.343782 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:04:58.344071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:04:58.344853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:04:58.344853 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:04:58.344853 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:04:58.344853 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:04:58.344853 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:04:58.344853 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:04:58.344853 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:04:58.344853 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:04:58.346497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:04:58.346497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:04:58.346497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:04:58.346497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:04:58.346497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:04:58.346497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:04:58.346497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:04:58.346497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:04:58.346497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:04:58.346497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:04:58.346497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:04:58.346497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:04:58.346826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:04:58.346826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:04:58.346826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:04:58.346826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:04:58.346826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:04:58.346826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:04:58.346826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:04:58.346826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:04:58.346826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:04:58.346826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:04:58.346826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:04:58.346826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:04:58.346826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:04:58.347190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:04:58.347190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:04:58.348321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:04:58.348321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:04:58.348321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:04:58.348321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:04:58.348321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:04:58.348321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:04:58.348321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:04:58.348321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:04:58.348321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:04:58.348321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:04:58.348321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:04:58.348321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:04:58.348321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:04:58.348321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:04:58.348321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:04:58.348321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:04:58.348640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:04:58.348640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:04:58.348640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:04:58.348640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:04:58.348640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:04:58.348640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:04:58.348640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:04:58.348640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:04:58.348640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:04:58.348640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:04:58.348640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:04:58.348640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:04:58.348640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:04:58.348640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:04:58.348640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:04:58.348640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:04:58.348640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:04:58.348640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:04:58.348640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:04:58.349156 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:04:58.349156 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:04:58.349156 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:04:58.359366 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:04:58.359817 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:04:58.360003 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:04:58.360197 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:04:58.360733 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:04:58.360962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:04:58.363830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:04:58.363830 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:04:58.363830 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:04:58.363830 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:04:58.363830 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:04:58.363830 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:04:58.363830 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:04:58.363830 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:04:58.365113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:04:58.365113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:04:58.365113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:04:58.365113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:04:58.365113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:04:58.365113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:04:58.365113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:04:58.365113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:04:58.365113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:04:58.365113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:04:58.365113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:04:58.365113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:04:58.365113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:04:58.365113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:04:58.365113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:04:58.365909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:04:58.365909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:04:58.365909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:04:58.365909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:04:58.365909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:04:58.365909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:04:58.365909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:04:58.365909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:04:58.365909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:04:58.366401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:04:58.366401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:04:58.366401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:04:58.366401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:04:58.366401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:04:58.366401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:04:58.366401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:04:58.366401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:04:58.366401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:04:58.366401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:04:58.366401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:04:58.366401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:04:58.367009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:04:58.367009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:04:58.367009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:04:58.367009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:04:58.367009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:04:58.367009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:04:58.367009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:04:58.367009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:04:58.367009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:04:58.367009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:04:58.367009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:04:58.367009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:04:58.367009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:04:58.367009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:04:58.367738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:04:58.367738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:04:58.367738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:04:58.367738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:04:58.368023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:04:58.368023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:04:58.368023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:04:58.368023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:04:58.368023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:04:58.368023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:04:58.368023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:04:58.368023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:04:58.368023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:04:58.368023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:04:58.368023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:04:58.368023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:04:58.368023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:04:58.368023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:04:58.368023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:04:58.368023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:04:58.368693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:04:58.368693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:04:58.368693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:04:58.368693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:04:58.368693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:04:58.368693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:04:58.368693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:04:58.368693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:04:58.368693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:04:58.368693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:04:58.368693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:04:58.368693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:04:58.368693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:04:58.368693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:04:58.368693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:04:58.368693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:04:58.368693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:04:58.368693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:04:58.369479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:04:58.369479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:04:58.369479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:04:58.369479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:04:58.369479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:04:58.369479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:04:58.369479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:04:58.374771 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:04:58.375124 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:04:58.375262 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:04:58.375262 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:04:58.377380 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:04:58.377933 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:04:58.378978 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:04:58.378978 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:04:58.378978 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:04:58.378978 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:04:58.378978 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:04:58.378978 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:04:58.378978 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:04:58.378978 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:04:58.380251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:04:58.380251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:04:58.380251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:04:58.380251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:04:58.380251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:04:58.380251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:04:58.380251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:04:58.380251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:04:58.380251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:04:58.380251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:04:58.380251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:04:58.380251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:04:58.380251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:04:58.380251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:04:58.380251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:04:58.381271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:04:58.381271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:04:58.381271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:04:58.381271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:04:58.381271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:04:58.381271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:04:58.381271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:04:58.381271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:04:58.381271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:04:58.381770 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:04:58.381770 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:04:58.381770 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:04:58.381770 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:04:58.381770 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:04:58.381770 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:04:58.381770 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:04:58.381770 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:04:58.381770 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:04:58.381770 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:04:58.381770 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:04:58.382231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:04:58.382231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:04:58.382231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:04:58.382231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:04:58.382231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:04:58.382231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:04:58.382231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:04:58.382231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:04:58.382231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:04:58.382231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:04:58.382231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:04:58.382231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:04:58.382231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:04:58.382231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:04:58.382725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:04:58.382725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:04:58.382725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:04:58.382725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:04:58.383574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:04:58.383574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:04:58.383574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:04:58.383574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:04:58.383574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:04:58.383574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:04:58.383574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:04:58.383574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:04:58.383574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:04:58.383574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:04:58.383574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:04:58.383574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:04:58.383574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:04:58.383574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:04:58.383574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:04:58.383574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:04:58.383574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:04:58.384313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:04:58.384313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:04:58.384313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:04:58.384313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:04:58.384313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:04:58.384313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:04:58.384313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:04:58.384313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:04:58.384313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:04:58.384313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:04:58.384313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:04:58.384313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:04:58.384313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:04:58.384313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:04:58.384313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:04:58.384313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:04:58.384313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:04:58.384313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:04:58.384313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:04:58.385382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:04:58.385382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:04:58.385382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:04:58.385382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:04:58.385382 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:04:58.385382 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:04:58.385382 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:04:58.385382 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:04:58.386580 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:04:58.388258 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:04:58.388450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:04:58.389225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:04:58.389225 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:04:58.389225 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:04:58.389225 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:04:58.389225 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:04:58.389225 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:04:58.389225 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:04:58.389225 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:04:58.390140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:04:58.390140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:04:58.390140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:04:58.390140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:04:58.390140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:04:58.390140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:04:58.390140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:04:58.390140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:04:58.390140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:04:58.390140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:04:58.390140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:04:58.390140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:04:58.390701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:04:58.390701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:04:58.390701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:04:58.390701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:04:58.390701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:04:58.390701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:04:58.390701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:04:58.390701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:04:58.390701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:04:58.390701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:04:58.390701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:04:58.390701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:04:58.390701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:04:58.391690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:04:58.391690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:04:58.391690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:04:58.391690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:04:58.391690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:04:58.391690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:04:58.391690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:04:58.391690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:04:58.391690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:04:58.391690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:04:58.391690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:04:58.391690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:04:58.391690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:04:58.391690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:04:58.391690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:04:58.393324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:04:58.393324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:04:58.393324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:04:58.393324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:04:58.393324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:04:58.393324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:04:58.393324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:04:58.393324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:04:58.393324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:04:58.393324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:04:58.393324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:04:58.393324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:04:58.393324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:04:58.393324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:04:58.393324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:04:58.393324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:04:58.393324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:04:58.393324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:04:58.393324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:04:58.394977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:04:58.394977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:04:58.394977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:04:58.394977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:04:58.394977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:04:58.394977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:05:02.824651 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:05:02.825587 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:05:02.825587 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:05:02.827109 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:05:02.863787 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:05:02.864197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:02.865156 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:05:02.865156 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:02.865156 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:05:02.865156 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:05:02.865156 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:02.865156 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:05:02.865156 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:02.865156 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:05:02.866258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:02.866258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:05:02.866258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:05:02.866258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:05:02.866258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:05:02.866258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:05:02.866258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:05:02.866258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:02.866258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:05:02.866258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:02.866258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:05:02.866258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:02.866716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:05:02.866716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:02.866716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:05:02.866716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:02.866716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:05:02.866716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:05:02.866716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:02.866716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:05:02.866716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:02.866716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:05:02.866716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:05:02.866716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:02.866716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:05:02.867166 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:02.867166 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:05:02.867622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:02.867622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:05:02.867622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:05:02.867622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:05:02.867622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:05:02.867622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:05:02.867622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:05:02.867622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:05:02.867622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:05:02.867622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:05:02.867622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:05:02.867622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:05:02.867622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:05:02.867622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:05:02.867622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:05:02.867622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:05:02.868273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:05:02.868273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:05:02.868273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:02.868273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:05:02.868273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:02.868273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:05:02.868273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:02.868273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:05:02.868273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:02.868273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:05:02.868273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:02.868273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:05:02.868273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:05:02.868273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:02.868273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:05:02.868273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:02.868273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:05:02.868273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:05:02.868273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:02.868905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:05:02.868905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:02.868905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:05:03.342952 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:05:03.344284 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:05:03.344495 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:05:03.345623 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:05:03.345876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:03.346807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:05:03.346807 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:03.346807 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:05:03.346807 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:05:03.346807 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:03.346807 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:05:03.346807 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:03.346807 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:05:03.348263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:03.348263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:05:03.348263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:05:03.348263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:05:03.348263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:05:03.348263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:05:03.348263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:05:03.348263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:03.348263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:05:03.348263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:03.348263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:05:03.348263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:03.348697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:05:03.348697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:03.348697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:05:03.348697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:03.348697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:05:03.348697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:05:03.348697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:03.348697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:05:03.348697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:03.348697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:05:03.348697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:05:03.348697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:03.348697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:05:03.349104 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:03.349104 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:05:03.350096 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:03.350096 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:05:03.350096 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:05:03.350096 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:05:03.350096 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:05:03.350096 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:05:03.350096 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:05:03.350096 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:05:03.350096 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:05:03.350096 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:05:03.350096 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:05:03.350096 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:05:03.350096 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:05:03.350096 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:05:03.350096 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:05:03.350096 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:05:03.350635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:05:03.350635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:05:03.350635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:03.350635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:05:03.350635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:03.350635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:05:03.350635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:03.350635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:05:03.350635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:03.350635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:05:03.350635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:03.350635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:05:03.350635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:05:03.350635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:03.350635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:05:03.350635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:03.350635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:05:03.350635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:05:03.350635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:03.351301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:05:03.351301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:03.351301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:05:03.359962 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:05:03.360327 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:05:03.360445 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:05:03.360562 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:05:03.361066 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:05:03.361251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:03.361719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:05:03.361719 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:03.361719 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:05:03.361719 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:05:03.361719 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:03.361719 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:05:03.361719 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:03.361719 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:05:03.362520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:03.362520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:05:03.362520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:05:03.362520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:05:03.362520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:03.362520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:05:03.362520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:03.362520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:05:03.362520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:03.362520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:05:03.362520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:03.362520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:05:03.362520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:03.362520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:05:03.362520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:05:03.362980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:03.362980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:05:03.362980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:03.362980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:05:03.362980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:05:03.362980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:03.362980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:05:03.362980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:03.362980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:05:03.363331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:03.363331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:05:03.363331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:05:03.363331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:05:03.363331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:05:03.363331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:05:03.363331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:05:03.363331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:05:03.363331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:05:03.363331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:05:03.363331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:03.363331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:05:03.363696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:03.363696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:05:03.363696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:03.363696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:05:03.363696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:03.363696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:05:03.363696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:03.363696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:05:03.363696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:05:03.363696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:03.363696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:05:03.363696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:03.363696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:05:03.363696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:05:03.364345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:03.364345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:05:03.364345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:03.364345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:05:03.364345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:03.364345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:05:03.364345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:05:03.364345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:05:03.364345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:05:03.364345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:05:03.364345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:05:03.364345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:05:03.364345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:05:03.364345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:05:03.364700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:05:03.364700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:05:03.364700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:05:03.364700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:05:03.364700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:05:03.364700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:05:03.364700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:05:03.364700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:05:03.364700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:05:03.364700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:05:03.364700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:05:03.364700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:03.364700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:05:03.364700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:03.364700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:05:03.364700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:03.364700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:05:03.364700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:03.364700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:05:03.365217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:03.365217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:05:03.365217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:05:03.365217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:03.365217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:05:03.365217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:03.365217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:05:03.365217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:05:03.365217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:03.365217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:05:03.365217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:03.365217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:05:03.375257 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:05:03.375686 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:05:03.375938 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:05:03.376157 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:05:03.376615 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:05:03.376903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:03.377450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:05:03.377450 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:03.377450 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:05:03.377450 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:05:03.377450 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:03.377450 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:05:03.377450 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:03.377450 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:05:03.379099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:03.379099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:05:03.379099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:05:03.379099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:05:03.379099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:03.379099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:05:03.379099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:03.379099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:05:03.379099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:03.379099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:05:03.379099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:03.379099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:05:03.379099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:03.379099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:05:03.379099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:05:03.379872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:03.379872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:05:03.379872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:03.379872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:05:03.379872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:05:03.379872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:03.379872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:05:03.379872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:03.379872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:05:03.380408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:03.380408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:05:03.380408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:05:03.380408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:05:03.380408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:05:03.380408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:05:03.380408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:05:03.380408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:05:03.380408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:05:03.380408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:03.380408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:05:03.380995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:03.380995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:05:03.380995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:03.380995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:05:03.380995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:03.380995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:05:03.380995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:03.380995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:05:03.380995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:05:03.380995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:03.380995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:05:03.380995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:03.380995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:05:03.380995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:05:03.381630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:03.381630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:05:03.381630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:03.381630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:05:03.382388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:03.382388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:05:03.382388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:05:03.382388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:05:03.382388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:05:03.382388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:05:03.382388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:05:03.382388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:05:03.382388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:05:03.382388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:05:03.382388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:05:03.382388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:05:03.382388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:05:03.382388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:05:03.382388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:05:03.382388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:05:03.382388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:05:03.383229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:05:03.383229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:05:03.383229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:05:03.383229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:03.383229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:05:03.383229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:03.383229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:05:03.383229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:03.383229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:05:03.383229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:03.383229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:05:03.383229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:03.383229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:05:03.383229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:05:03.383229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:03.383229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:05:03.383229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:03.383229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:05:03.383229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:05:03.384156 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:03.384156 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:05:03.384156 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:03.384156 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:05:03.384347 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:05:03.384548 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:05:03.384666 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:05:03.384866 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:05:03.385152 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:05:03.385354 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:05:03.385463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:03.386005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:05:03.386005 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:03.386005 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:05:03.386005 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:05:03.386005 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:03.386005 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:05:03.386005 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:03.386005 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:05:03.386801 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:03.386801 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:05:03.386801 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:05:03.386801 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:05:03.386801 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:05:03.386801 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:05:03.386801 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:05:03.386801 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:03.386801 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:05:03.386801 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:03.386801 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:05:03.386801 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:03.387157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:05:03.387157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:03.387157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:05:03.387157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:03.387157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:05:03.387157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:05:03.387157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:03.387157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:05:03.387157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:03.387157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:05:03.387157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:05:03.387157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:03.387157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:05:03.387504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:03.387504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:05:03.387761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:03.387761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:05:03.387761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:05:03.387761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:05:03.387761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:05:03.387761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:05:03.387761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:05:03.387761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:05:03.387761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:05:03.387761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:05:03.387761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:05:03.387761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:05:03.387761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:05:03.387761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:05:03.387761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:05:03.387761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:05:03.388201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:05:03.388201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:05:03.388201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:03.388201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:05:03.388201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:03.388201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:05:03.388201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:03.388201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:05:03.388201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:03.388201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:05:03.388201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:03.388201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:05:03.388201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:05:03.388201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:03.388201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:05:03.388201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:03.388201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:05:03.388201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:05:03.388201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:03.388694 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:05:03.388694 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:03.388694 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:05:07.855549 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:05:07.855549 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:05:07.855549 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:05:07.855549 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:05:07.870444 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:05:07.870444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:07.871159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:05:07.871159 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:07.871159 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:05:07.871159 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:05:07.871159 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:07.871159 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:05:07.871159 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:07.871159 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:05:07.872293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.044 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:07.872293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:05:07.872293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:05:07.872293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:05:07.872293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:05:07.872293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:05:07.872293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:05:07.872293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:07.872293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:05:07.872293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:07.872293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:05:07.872293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:07.872977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:05:07.872977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:07.872977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:05:07.872977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:07.872977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:05:07.872977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:05:07.872977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:07.872977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:05:07.872977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:07.872977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:05:07.872977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:05:07.872977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:07.872977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:05:07.873611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:07.873611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:05:07.873611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:07.873611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:05:07.873611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:05:07.873611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:05:07.873611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:05:07.873611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:05:07.873611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:05:07.873611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:05:07.873611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:05:07.873611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:05:07.873611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:05:07.873611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:05:07.873611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:05:07.874415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:05:07.874415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:05:07.874415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:05:07.874415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:05:07.874415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:05:07.874415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:07.874415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:05:07.874415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:07.874415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:05:07.874415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:07.874415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:05:07.874415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:07.874415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:05:07.874415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:07.874415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:05:07.874415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:05:07.874415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:07.874415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:05:07.874415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:07.875293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:05:07.875293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:05:07.875293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:07.875293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:05:07.875293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:07.875293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:05:08.347786 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:05:08.347786 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:05:08.347786 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:05:08.348781 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:05:08.349158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:08.350081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:05:08.350081 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:08.350081 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:05:08.350081 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:05:08.350081 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:08.350081 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:05:08.350081 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:08.350081 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:05:08.351103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:08.351103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:05:08.351103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:05:08.351103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:05:08.351103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:05:08.351103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:05:08.351103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:05:08.351103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:08.351103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:05:08.351103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:08.351103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:05:08.351103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:08.351742 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:05:08.351742 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:08.351742 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:05:08.351742 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:08.351742 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:05:08.351742 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:05:08.351742 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:08.351742 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:05:08.351742 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:08.351742 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:05:08.351742 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:05:08.351742 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:08.351742 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:05:08.352174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:08.352174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:05:08.352242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:08.352242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:05:08.352242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:05:08.352242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:05:08.352242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:05:08.352242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:05:08.352242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:05:08.352242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:05:08.352242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:05:08.352242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:05:08.352242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:05:08.352242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:05:08.352242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:05:08.352242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:05:08.352242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:05:08.352242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:05:08.352708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:05:08.352708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:05:08.352708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:08.352708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:05:08.352708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:08.352708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:05:08.352708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:08.352708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:05:08.352708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:08.352708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:05:08.352708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:08.352708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:05:08.352708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:05:08.352708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:08.352708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:05:08.352708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:08.352708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:05:08.352708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:05:08.352708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:08.353286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:05:08.353286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:08.353286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:05:08.360551 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:05:08.360879 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:05:08.360992 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:05:08.361097 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:05:08.361712 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:05:08.362118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:08.362840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:05:08.362840 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:08.362840 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:05:08.362840 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:05:08.362840 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:08.362840 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:05:08.362840 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:08.362840 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:05:08.363558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:08.363558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:05:08.363558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:05:08.363558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:05:08.363558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:08.363558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:05:08.363558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:08.363558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:05:08.363558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:08.363558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:05:08.363558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:08.363558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:05:08.363558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:08.363558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:05:08.363558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:05:08.364297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:08.364297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:05:08.364297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:08.364297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:05:08.364297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:05:08.364297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:08.364297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:05:08.364297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:08.364297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:05:08.364690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:08.364690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:05:08.364690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:05:08.364690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:05:08.364690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:05:08.364690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:05:08.364690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:05:08.364690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:05:08.364690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:05:08.364690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:05:08.364690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:08.364690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:05:08.365240 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:08.365240 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:05:08.365240 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:08.365240 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:05:08.365240 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:08.365240 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:05:08.365240 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:08.365240 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:05:08.365240 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:05:08.365240 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:08.365240 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:05:08.365240 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:08.365240 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:05:08.365240 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:05:08.365812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:08.365812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:05:08.365812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:08.365812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:05:08.365812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:08.365812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:05:08.365812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:05:08.365812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:05:08.365812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:05:08.365812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:05:08.365812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:05:08.365812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:05:08.365812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:05:08.365812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:05:08.366410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:05:08.366410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:05:08.366410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:05:08.366410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:05:08.366410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:05:08.366410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:05:08.366410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:05:08.366410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:05:08.366410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:05:08.366410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:05:08.366410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:05:08.366410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:08.366410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:05:08.366410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:08.366410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:05:08.366410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:08.366410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:05:08.366410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:08.366410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:05:08.367207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:08.367207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:05:08.367207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:05:08.367207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:08.367207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:05:08.367207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:08.367207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:05:08.367207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:05:08.367207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:08.367207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:05:08.367207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:08.367207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:05:08.375701 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:05:08.384666 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:05:08.385142 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:05:08.385360 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:05:08.385517 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:05:08.385999 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:05:08.386174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:08.386562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:05:08.386562 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:08.386562 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:05:08.386562 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:05:08.386562 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:08.386562 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:05:08.386562 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:08.386562 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:05:08.387313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:08.387313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:05:08.387313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:05:08.387313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:05:08.387313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:08.387313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:05:08.387313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:08.387313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:05:08.387313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:08.387313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:05:08.387313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:08.387313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:05:08.387313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:08.387313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:05:08.387313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:05:08.402769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:08.402769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:05:08.402769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:08.402769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:05:08.402769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:05:08.402769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:08.402769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:05:08.402769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:08.402769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:05:08.402769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:08.402769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:05:08.402769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:05:08.402769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:05:08.402769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:05:08.403225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:05:08.403225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:05:08.403225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:05:08.403225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:05:08.403225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:08.403225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:05:08.403225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:08.403225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:05:08.403225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:08.403225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:05:08.403225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:08.403225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:05:08.403225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:08.403605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:05:08.403605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:05:08.403605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:08.403605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:05:08.403605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:08.403605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:05:08.403605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:05:08.403605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:08.403605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:05:08.403605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:08.403605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:05:08.403605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:08.403992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:05:08.403992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:05:08.403992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:05:08.403992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:05:08.403992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:05:08.403992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:05:08.403992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:05:08.403992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:05:08.403992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:05:08.403992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:05:08.403992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:05:08.403992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:05:08.403992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:05:08.403992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:05:08.403992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:05:08.403992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:05:08.403992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:05:08.403992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:05:08.403992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:05:08.404436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:08.404436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:05:08.404436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:08.404436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:05:08.404436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:08.404436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:05:08.404436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:08.404436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:05:08.404436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:08.404436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:05:08.404436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:05:08.404436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:08.404436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:05:08.404436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:08.404436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:05:08.404436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:05:08.404436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:08.404436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:05:08.404436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:08.404436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:05:08.404960 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:05:08.404960 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:05:08.404960 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:05:08.404960 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:05:08.404960 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:05:08.404960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:08.405125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:05:08.405125 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:08.405125 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:05:08.405125 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:05:08.405125 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:08.405125 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:05:08.405125 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:08.405125 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:05:08.405125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:08.405125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:05:08.405125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:05:08.405125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:05:08.405125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:05:08.405125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:05:08.405125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:05:08.405534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:08.405534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:05:08.405534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:08.405534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:05:08.405534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:08.405534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:05:08.405534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:08.405534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:05:08.405534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:08.405534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:05:08.405534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:05:08.405534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:08.405534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:05:08.405534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:08.406092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:05:08.406092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:05:08.406092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:08.406092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:05:08.406092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:08.406092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:05:08.406092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:08.406092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:05:08.406092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:05:08.406092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:05:08.406092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:05:08.406092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:05:08.406092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:05:08.406092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:05:08.406476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:05:08.406476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:05:08.406476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:05:08.406476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:05:08.406476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:05:08.406476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:05:08.406476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:05:08.406476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:05:08.406476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:05:08.406476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:05:08.406476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:08.406476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:05:08.406476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:08.406476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:05:08.406476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:08.406476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:05:08.406476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:08.406476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:05:08.406476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:08.406476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:05:08.406945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:05:08.406945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:08.406945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:05:08.406945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:08.406945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:05:08.406945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:05:08.406945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:08.406945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:05:08.406945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:08.406945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:05:12.826435 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:05:12.827193 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:05:12.827509 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:05:12.828611 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:05:12.862507 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:05:12.863005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:12.864085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:05:12.864085 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:12.864085 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:05:12.864085 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:05:12.864085 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:12.864085 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:05:12.864085 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:12.864085 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:05:12.865182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:12.865182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:05:12.865182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:05:12.865182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:05:12.865182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:05:12.865182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:05:12.865182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:05:12.865182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:12.865182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:05:12.865182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:12.865182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:05:12.865182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:12.865850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:05:12.865850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:12.865850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:05:12.865850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:12.865850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:05:12.865850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:05:12.865850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:12.865850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:05:12.865850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:12.865850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:05:12.865850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:05:12.865850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:12.865850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:05:12.866509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:12.866509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:05:12.866509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:12.866509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:05:12.866509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:05:12.866509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:05:12.866509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:05:12.866509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:05:12.866509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:05:12.866509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:05:12.866509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:05:12.866509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:05:12.866509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:05:12.866509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:05:12.866509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:05:12.867243 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:05:12.867243 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:05:12.867243 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:05:12.867243 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:05:12.867243 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:05:12.867243 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:12.867243 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:05:12.867243 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:12.867243 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:05:12.867243 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:12.867243 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:05:12.867243 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:12.867243 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:05:12.867243 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:12.867243 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:05:12.867243 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:05:12.867243 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:12.867243 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:05:12.867243 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:12.868230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:05:12.868230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:05:12.868230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:12.868230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:05:12.868230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:12.868230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:05:13.347534 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:05:13.348469 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:05:13.348661 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:05:13.349890 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:05:13.350227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:13.351133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:05:13.351133 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:13.351133 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:05:13.351133 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:05:13.351133 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:13.351133 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:05:13.351133 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:13.351133 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:05:13.352769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:13.352769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:05:13.352769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:05:13.352769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:05:13.352769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:05:13.352769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:05:13.352769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:05:13.352769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:13.352769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:05:13.352769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:13.352769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:05:13.352769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:13.353382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:05:13.353382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:13.353382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:05:13.353382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:13.353382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:05:13.353382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:05:13.353382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:13.353382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:05:13.353382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:13.353382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:05:13.353382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:05:13.353382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:13.353382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:05:13.353991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:13.353991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:05:13.354439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:13.354439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:05:13.354439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:05:13.354439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:05:13.354439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:05:13.354439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:05:13.354439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:05:13.354439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:05:13.354439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:05:13.354439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:05:13.354439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:05:13.354439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:05:13.354439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:05:13.354439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:05:13.354439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:05:13.354439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:05:13.355196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:05:13.355196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:05:13.355196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:13.355196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:05:13.355196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:13.355196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:05:13.355196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:13.355196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:05:13.355196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:13.355196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:05:13.355196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:13.355196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:05:13.355196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:05:13.355196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:13.355196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:05:13.355196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:13.355196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:05:13.355196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:05:13.355196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:13.356111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:05:13.356111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:13.356111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:05:13.361075 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:05:13.361389 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:05:13.361507 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:05:13.361629 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:05:13.362177 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:05:13.362410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:13.363208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:05:13.363208 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:13.363208 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:05:13.363208 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:05:13.363208 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:13.363208 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:05:13.363208 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:13.363208 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:05:13.363984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:13.363984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:05:13.363984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:05:13.363984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:05:13.363984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:13.363984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:05:13.363984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:13.363984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:05:13.363984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:13.363984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:05:13.363984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:13.363984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:05:13.363984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:13.363984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:05:13.363984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:05:13.364768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:13.364768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:05:13.364768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:13.364768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:05:13.364768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:05:13.364768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:13.364768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:05:13.364768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:13.364768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:05:13.364768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:13.364768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:05:13.364768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:05:13.364768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:05:13.364768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:05:13.365485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:05:13.365485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:05:13.365485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:05:13.365485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:05:13.365485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:05:13.365485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:13.365485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:05:13.365485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:13.365485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:05:13.365485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:13.365485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:05:13.365485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:13.365485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:05:13.366152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:13.366152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:05:13.366152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:05:13.366152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:13.366152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:05:13.366152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:13.366152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:05:13.366152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:05:13.366152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:13.366152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:05:13.366152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:13.366152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:05:13.366794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:13.366794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:05:13.366794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:05:13.366794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:05:13.366794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:05:13.366794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:05:13.366794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:05:13.366794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:05:13.366794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:05:13.366794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:05:13.366794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:05:13.366794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:05:13.366794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:05:13.366794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:05:13.366794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:05:13.366794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:05:13.367629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:05:13.367629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:05:13.367629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:05:13.367629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:05:13.367629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:05:13.367629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:13.367629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:05:13.367629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:13.367629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:05:13.367629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:13.367629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:05:13.367629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:13.367629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:05:13.367629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:13.367629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:05:13.367629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:05:13.367629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:13.367629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:05:13.368615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:13.368615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:05:13.368615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:05:13.368615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:13.368615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:05:13.368615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:13.368615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:05:13.386161 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:05:13.386568 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:05:13.386850 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:05:13.387033 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:05:13.387727 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:05:13.388203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:13.389318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:05:13.389318 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:13.389318 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:05:13.389318 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:05:13.389318 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:13.389318 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:05:13.389318 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:13.389318 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:05:13.389769 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:05:13.390994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:13.390994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:05:13.390994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:05:13.390994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:05:13.390994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:13.390994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:05:13.390994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:13.390994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:05:13.390994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:13.390994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:05:13.390994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:13.390994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:05:13.390994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:13.390994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:05:13.390994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:05:13.391727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:13.391727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:05:13.391727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:13.391727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:05:13.391727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:05:13.391727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:13.391727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:05:13.391727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:13.391727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:05:13.392519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:13.392519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:05:13.392519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:05:13.392519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:05:13.392519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:05:13.392519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:05:13.392519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:05:13.392519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:05:13.392519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:05:13.392519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:13.392519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:05:13.393097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:13.393097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:05:13.393097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:13.393097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:05:13.393097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:13.393097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:05:13.393097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:13.393097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:05:13.393097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:05:13.393097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:13.393097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:05:13.393097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:13.393097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:05:13.393097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:05:13.393875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:13.393875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:05:13.393875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:13.393875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:05:13.394614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:13.394614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:05:13.394614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:05:13.394614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:05:13.394614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:05:13.394614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:05:13.394614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:05:13.394614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:05:13.394614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:05:13.394614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:05:13.394614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:05:13.394614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:05:13.394614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:05:13.394614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:05:13.394614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:05:13.394614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:05:13.394614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:05:13.395263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:05:13.395263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:05:13.395263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:05:13.395263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:13.395263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:05:13.395263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:13.395263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:05:13.395263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:13.395263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:05:13.395263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:13.395263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:05:13.395263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:13.395263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:05:13.395263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:05:13.395263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:13.395263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:05:13.395263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:13.395263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:05:13.395263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:05:13.395852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:13.395852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:05:13.395852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:13.395852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:05:13.395852 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:05:13.395852 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:05:13.395852 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:05:13.396068 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:05:13.396226 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:05:13.396352 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:13.397054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:05:13.397054 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:13.397054 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:05:13.397054 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:05:13.397054 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:13.397054 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:05:13.397054 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:13.397054 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:05:13.397952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:13.397952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:05:13.397952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:05:13.397952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:05:13.397952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:05:13.397952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:05:13.397952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:05:13.397952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:13.397952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:05:13.397952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:13.397952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:05:13.397952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:13.398325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:05:13.398325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:13.398325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:05:13.398325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:13.398325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:05:13.398325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:05:13.398325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:13.398325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:05:13.398325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:13.398325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:05:13.398325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:05:13.398325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:13.398325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:05:13.398694 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:13.398694 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:05:13.399157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:13.399157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:05:13.399157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:05:13.399157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:05:13.399157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:05:13.399157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:05:13.399157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:05:13.399157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:05:13.399157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:05:13.399157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:05:13.399157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:05:13.399157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:05:13.399157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:05:13.399157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:05:13.399157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:05:13.399157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:05:13.399667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:05:13.399667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:05:13.399667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:13.399667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:05:13.399667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:13.399667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:05:13.399667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:13.399667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:05:13.399667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:13.399667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:05:13.399667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:13.399667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:05:13.399667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:05:13.399667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:13.399667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:05:13.399667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:13.399667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:05:13.399667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:05:13.399667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:13.400209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:05:13.400209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:13.400209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:05:17.828125 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:05:17.828802 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:05:17.828943 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:05:17.830298 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:05:17.866989 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:05:17.867341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:17.868145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:05:17.868145 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:17.868145 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:05:17.868145 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:05:17.868145 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:17.868145 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:05:17.868145 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:17.868145 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:05:17.869060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:17.869060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:05:17.869060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:05:17.869060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:05:17.869060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:05:17.869060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:05:17.869060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:05:17.869060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:17.869060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:05:17.869060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:17.869060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:05:17.869060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:17.869728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:05:17.869728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:17.869728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:05:17.869728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:17.869728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:05:17.869728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:05:17.869728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:17.869728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:05:17.869728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:17.869728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:05:17.869728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:05:17.869728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:17.869728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:05:17.870312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:17.870312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:05:17.870312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:17.870312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:05:17.870312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:05:17.870312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:05:17.870312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:05:17.870312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:05:17.870312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:05:17.870312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:05:17.870312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:05:17.870312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:05:17.870312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:05:17.870312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:05:17.870312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:05:17.870767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:05:17.870767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:05:17.870767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:05:17.870767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:05:17.870767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:05:17.870767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:17.870767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:05:17.870767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:17.870767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:05:17.870767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:17.870767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:05:17.870767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:17.870767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:05:17.870767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:17.870767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:05:17.870767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:05:17.870767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:17.870767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:05:17.870767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:17.871853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:05:17.871853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:05:17.871853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:17.871853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:05:17.871853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:17.871853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:05:18.348564 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:05:18.349041 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:05:18.349266 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:05:18.350509 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:05:18.350928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:18.351872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:05:18.351872 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:18.351872 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:05:18.351872 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:05:18.351872 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:18.351872 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:05:18.351872 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:18.351872 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:05:18.353388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:18.353388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:05:18.353388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:05:18.353388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:05:18.353388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:05:18.353388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:05:18.353388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:05:18.353388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:18.353388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:05:18.353388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:18.353388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:05:18.353388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:18.354003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:05:18.354003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:18.354003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:05:18.354003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:18.354003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:05:18.354003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:05:18.354003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:18.354003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:05:18.354003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:18.354003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:05:18.354003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:05:18.354003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:18.354003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:05:18.354571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:18.354571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:05:18.354925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:18.354925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:05:18.354925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:05:18.354925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:05:18.354925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:05:18.354925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:05:18.354925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:05:18.354925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:05:18.354925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:05:18.354925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:05:18.354925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:05:18.354925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:05:18.354925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:05:18.354925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:05:18.354925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:05:18.354925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:05:18.355627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:05:18.355627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:05:18.355627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:18.355627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:05:18.355627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:18.355627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:05:18.355627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:18.355627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:05:18.355627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:18.355627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:05:18.355627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:18.355627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:05:18.355627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:05:18.355627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:18.355627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:05:18.355627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:18.355627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:05:18.355627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:05:18.355627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:18.356458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:05:18.356458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:18.356458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:05:18.361527 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:05:18.361812 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:05:18.361932 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:05:18.362112 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:05:18.362619 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:05:18.362986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:18.363876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:05:18.363876 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:18.363876 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:05:18.363876 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:05:18.363876 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:18.363876 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:05:18.363876 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:18.363876 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:05:18.365141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:18.365141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:05:18.365141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:05:18.365141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:05:18.365141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:18.365141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:05:18.365141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:18.365141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:05:18.365141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:18.365141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:05:18.365141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:18.365141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:05:18.365141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:18.365141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:05:18.365141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:05:18.365581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:18.365581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:05:18.365581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:18.365581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:05:18.365581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:05:18.365581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:18.365581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:05:18.365581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:18.365581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:05:18.366535 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:18.366535 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:05:18.366535 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:05:18.366535 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:05:18.366535 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:05:18.366535 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:05:18.366535 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:05:18.366535 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:05:18.366535 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:05:18.366535 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:05:18.366535 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:18.366535 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:05:18.367101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:18.367101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:05:18.367101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:18.367101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:05:18.367101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:18.367101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:05:18.367101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:18.367101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:05:18.367101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:05:18.367101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:18.367101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:05:18.367101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:18.367101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:05:18.367101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:05:18.367698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:18.367698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:05:18.367698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:18.367698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:05:18.368169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:18.368169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:05:18.368169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:05:18.368169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:05:18.368169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:05:18.368169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:05:18.368169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:05:18.368169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:05:18.368169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:05:18.368169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:05:18.368169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:05:18.368169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:05:18.368169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:05:18.368169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:05:18.368169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:05:18.368169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:05:18.368904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:05:18.368904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:05:18.368904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:05:18.368904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:05:18.368904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:05:18.368904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:18.368904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:05:18.368904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:18.368904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:05:18.368904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:18.368904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:05:18.368904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:18.368904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:05:18.368904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:18.368904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:05:18.368904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:05:18.368904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:18.368904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:05:18.369714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:18.369714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:05:18.369714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:05:18.369714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:18.369714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:05:18.369714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:18.369714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:05:18.387276 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:05:18.387496 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:05:18.387601 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:05:18.387781 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:05:18.388248 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:05:18.388635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:18.389414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:05:18.389414 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:18.389414 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:05:18.389414 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:05:18.389414 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:18.389414 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:05:18.389414 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:18.389414 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:05:18.390161 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:18.390161 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:05:18.390161 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:05:18.390161 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:05:18.390161 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:18.390161 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:05:18.390161 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:18.390161 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:05:18.390161 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:18.390161 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:05:18.390161 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:18.390161 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:05:18.390161 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:18.390161 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:05:18.390161 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:05:18.390849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:18.390849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:05:18.390849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:18.390849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:05:18.390849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:05:18.390849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:18.390849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:05:18.390849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:18.390849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:05:18.391857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:18.391857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:05:18.391857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:05:18.391857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:05:18.391857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:05:18.391857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:05:18.391857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:05:18.391857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:05:18.391857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:05:18.391857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:18.391857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:05:18.392354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:18.392354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:05:18.392354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:18.392354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:05:18.392354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:18.392354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:05:18.392354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:18.392354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:05:18.392354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:05:18.392354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:18.392354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:05:18.392354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:18.392354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:05:18.392354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:05:18.392969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:18.392969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:05:18.392969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:18.392969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:05:18.394541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:18.394541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:05:18.394541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:05:18.394541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:05:18.394541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:05:18.394541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:05:18.394541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:05:18.394541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:05:18.394541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:05:18.394541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:05:18.394541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:05:18.394541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:05:18.394541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:05:18.394541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:05:18.394541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:05:18.394541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:05:18.394541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:05:18.395394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:05:18.395394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:05:18.395394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:05:18.395394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:18.395394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:05:18.395394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:18.395394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:05:18.395394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:18.395394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:05:18.395394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:18.395394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:05:18.395394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:18.395394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:05:18.395394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:05:18.395394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:18.395394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:05:18.395394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:18.395394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:05:18.395394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:05:18.399704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:18.399704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:05:18.399704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:18.399704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:05:18.399704 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:05:18.399704 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:05:18.399704 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:05:18.399704 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:05:18.400148 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:05:18.400148 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:05:18.400148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:18.400148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:05:18.400148 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:18.400148 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:05:18.400148 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:05:18.400148 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:18.400148 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:05:18.400148 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:18.400148 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:05:18.400148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:18.400534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:05:18.400534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:05:18.400534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:05:18.400534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:05:18.400534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:05:18.400534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:05:18.400534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:18.400534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:05:18.400534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:18.400534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:05:18.400534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:18.400534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:05:18.400534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:18.400534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:05:18.400993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:18.400993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:05:18.400993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:05:18.400993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:18.400993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:05:18.400993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:18.400993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:05:18.400993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:05:18.400993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:18.400993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:05:18.400993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:18.400993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:05:18.401340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:18.401340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:05:18.401340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:05:18.401340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:05:18.401340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:05:18.401340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:05:18.401340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:05:18.401340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:05:18.401340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:05:18.401340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:05:18.401340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:05:18.401340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:05:18.401340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:05:18.401340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:05:18.401340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:05:18.401340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:05:18.401859 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:05:18.401859 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:05:18.401859 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:18.401859 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:05:18.401859 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:18.401859 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:05:18.401859 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:18.401859 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:05:18.401859 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:18.401859 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:05:18.401859 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:18.401859 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:05:18.401859 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:05:18.401859 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:18.401859 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:05:18.401859 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:18.401859 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:05:18.401859 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:05:18.401859 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:18.402429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:05:18.402429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:18.402429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:05:22.828824 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:05:22.829468 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:05:22.831667 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:05:22.831667 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:05:22.876397 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:05:22.878778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:22.878778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:05:22.878778 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:22.878778 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:05:22.878778 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:05:22.878778 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:22.878778 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:05:22.878778 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:22.878778 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:05:22.878778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.048 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:22.878778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:05:22.878778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:05:22.878778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:05:22.878778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:05:22.878778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:05:22.879162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:05:22.879162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:22.879162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:05:22.879162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:22.879162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:05:22.879162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:22.879162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:05:22.879162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:22.879162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:05:22.879162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:22.879162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:05:22.879162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:05:22.879162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:22.879162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:05:22.879462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:22.879462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:05:22.879462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:05:22.879462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:22.879462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:05:22.879462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:22.879462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:05:22.882852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:22.882852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:05:22.882852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:05:22.882852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:05:22.882852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:05:22.882852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:05:22.882852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:05:22.882852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:05:22.882852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:05:22.882852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:05:22.882852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:05:22.882852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:05:22.882852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:05:22.882852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:05:22.882852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:05:22.882852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:05:22.883196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:05:22.883196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:05:22.883196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:22.883196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:05:22.883196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:22.883196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:05:22.883196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:22.883196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:05:22.883196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:22.883196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:05:22.883196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:22.883196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:05:22.883196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:05:22.883196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:22.883196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:05:22.883196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:22.883196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:05:22.883196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:05:22.883196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:22.883545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:05:22.883545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:22.883545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:05:23.349542 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:05:23.350367 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:05:23.350623 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:05:23.351771 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:05:23.352178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:23.353040 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:05:23.353040 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:23.353040 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:05:23.353040 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:05:23.353040 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:23.353040 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:05:23.353040 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:23.353040 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:05:23.354487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:23.354487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:05:23.354487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:05:23.354487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:05:23.354487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:05:23.354487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:05:23.354487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:05:23.354487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:23.354487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:05:23.354487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:23.354487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:05:23.354487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:23.355137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:05:23.355137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:23.355137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:05:23.355137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:23.355137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:05:23.355137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:05:23.355137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:23.355137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:05:23.355137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:23.355137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:05:23.355137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:05:23.355137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:23.355137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:05:23.355733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:23.355733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:05:23.355733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:23.355733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:05:23.355733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:05:23.355733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:05:23.355733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:05:23.355733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:05:23.355733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:05:23.355733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:05:23.355733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:05:23.355733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:05:23.355733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:05:23.355733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:05:23.355733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:05:23.356417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:05:23.356417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:05:23.356417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:05:23.356417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:05:23.356417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:05:23.356417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:23.356417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:05:23.356417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:23.356417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:05:23.356417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:23.356417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:05:23.356417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:23.356417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:05:23.356417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:23.356417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:05:23.356417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:05:23.356417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:23.356417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:05:23.356417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:23.357190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:05:23.357190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:05:23.357190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:23.357190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:05:23.357190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:23.357190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:05:23.362116 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:05:23.362465 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:05:23.362592 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:05:23.362735 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:05:23.363220 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:05:23.363525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:23.364368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:05:23.364368 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:23.364368 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:05:23.364368 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:05:23.364368 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:23.364368 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:05:23.364368 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:23.364368 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:05:23.365190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:23.365190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:05:23.365190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:05:23.365190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:05:23.365190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:23.365190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:05:23.365190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:23.365190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:05:23.365190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:23.365190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:05:23.365190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:23.365190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:05:23.365190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:23.365190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:05:23.365190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:05:23.365598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:23.365598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:05:23.365598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:23.365598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:05:23.365598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:05:23.365598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:23.365598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:05:23.365598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:23.365598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:05:23.366059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:23.366059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:05:23.366059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:05:23.366059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:05:23.366059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:05:23.366059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:05:23.366059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:05:23.366059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:05:23.366059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:05:23.366059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:05:23.366059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:23.366059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:05:23.366404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:23.366404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:05:23.366404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:23.366404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:05:23.366404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:23.366404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:05:23.366404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:23.366404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:05:23.366404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:05:23.366404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:23.366404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:05:23.366404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:23.366404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:05:23.366404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:05:23.366993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:23.366993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:05:23.366993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:23.366993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:05:23.367163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:23.367163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:05:23.367163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:05:23.367163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:05:23.367163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:05:23.367163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:05:23.367163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:05:23.367163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:05:23.367163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:05:23.367163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:05:23.367163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:05:23.367163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:05:23.367163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:05:23.367163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:05:23.367163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:05:23.367163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:05:23.367814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:05:23.367814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:05:23.367814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:05:23.367814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:05:23.367814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:05:23.367814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:23.367814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:05:23.367814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:23.367814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:05:23.367814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:23.367814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:05:23.367814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:23.367814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:05:23.367814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:23.367814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:05:23.367814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:05:23.367814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:23.367814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:05:23.368541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:23.368541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:05:23.368541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:05:23.368541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:23.368541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:05:23.368541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:23.368541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:05:23.387979 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:05:23.388442 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:05:23.388738 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:05:23.388998 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:05:23.389579 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:05:23.390032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:23.390753 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:05:23.390753 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:23.390753 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:05:23.390753 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:05:23.390753 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:23.390753 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:05:23.390753 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:23.390753 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:05:23.391550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:23.391550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:05:23.391550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:05:23.391550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:05:23.391550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:23.391550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:05:23.391550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:23.391550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:05:23.391550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:23.391550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:05:23.391550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:23.391550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:05:23.391550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:23.391550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:05:23.391550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:05:23.392246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:23.392246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:05:23.392246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:23.392246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:05:23.392246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:05:23.392246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:23.392246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:05:23.392246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:23.392246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:05:23.392700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:23.392700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:05:23.392700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:05:23.392700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:05:23.392700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:05:23.392700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:05:23.392700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:05:23.392700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:05:23.392700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:05:23.392700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:23.392700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:05:23.393246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:23.393246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:05:23.393246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:23.393246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:05:23.393246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:23.393246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:05:23.393246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:23.393246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:05:23.393246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:05:23.393246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:23.393246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:05:23.393246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:23.393246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:05:23.393246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:05:23.393722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:23.393722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:05:23.393722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:23.393722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:05:23.393722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:23.393722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:05:23.393722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:05:23.393722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:05:23.393722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:05:23.393722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:05:23.393722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:05:23.393722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:05:23.393722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:05:23.393722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:05:23.394137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:05:23.394137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:05:23.394137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:05:23.394137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:05:23.394137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:05:23.394137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:05:23.394137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:05:23.394137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:05:23.394137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:05:23.394137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:05:23.394137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:23.394137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:05:23.394137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:23.394137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:05:23.394137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:23.394137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:05:23.394137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:23.394137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:05:23.394137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:23.394666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:05:23.394666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:05:23.394666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:23.394666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:05:23.394666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:23.394666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:05:23.394666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:05:23.394666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:23.394666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:05:23.394666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:23.394666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:05:23.395460 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:05:23.395842 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:05:23.395959 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:05:23.396108 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:05:23.396546 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:05:23.397126 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:05:23.397317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:23.397928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:05:23.397928 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:23.397928 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:05:23.397928 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:05:23.397928 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:23.397928 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:05:23.397928 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:23.397928 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:05:23.398722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:23.398722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:05:23.398722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:05:23.398722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:05:23.398722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:05:23.398722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:05:23.398722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:05:23.398722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:23.398722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:05:23.398722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:23.398722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:05:23.398722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:23.399181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:05:23.399181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:23.399181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:05:23.399181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:23.399181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:05:23.399181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:05:23.399181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:23.399181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:05:23.399181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:23.399181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:05:23.399181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:05:23.399181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:23.399181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:05:23.399522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:23.399522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:05:23.399676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:23.399676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:05:23.399676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:05:23.399676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:05:23.399676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:05:23.399676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:05:23.399676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:05:23.399676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:05:23.399676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:05:23.399676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:05:23.399676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:05:23.399676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:05:23.399676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:05:23.399676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:05:23.399676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:05:23.399676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:05:23.400118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:05:23.400118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:05:23.400118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:23.400118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:05:23.400118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:23.400118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:05:23.400118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:23.400118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:05:23.400118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:23.400118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:05:23.400118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:23.400118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:05:23.400118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:05:23.400118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:23.400118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:05:23.400118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:23.400118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:05:23.400118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:05:23.400118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:23.400601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:05:23.400601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:23.400601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:05:27.831571 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:05:27.831571 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:05:27.831571 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:05:27.831571 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:05:27.876609 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:05:27.876609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:27.877864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:05:27.877864 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:27.877864 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:05:27.877864 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:05:27.877864 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:27.877864 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:05:27.877864 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:27.877864 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:05:27.878869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.047 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:27.878869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:05:27.878869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:05:27.878869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:05:27.878869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:05:27.878869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:05:27.878869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:05:27.878869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:27.878869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:05:27.878869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:27.878869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:05:27.878869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:27.880977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:05:27.880977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:27.880977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:05:27.880977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:27.880977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:05:27.880977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:05:27.880977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:27.880977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:05:27.880977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:27.880977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:05:27.880977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:05:27.880977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:27.880977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:05:27.882958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:27.882958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:05:27.882958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:27.882958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:05:27.882958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:05:27.882958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:05:27.882958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:05:27.882958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:05:27.882958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:05:27.882958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:05:27.882958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:05:27.882958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:05:27.882958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:05:27.882958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:05:27.882958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:05:27.884636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:05:27.884636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:05:27.884636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:05:27.884636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:05:27.884636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:05:27.884636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:27.884636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:05:27.884636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:27.884636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:05:27.884636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:27.884636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:05:27.884636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:27.884636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:05:27.884636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:27.884636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:05:27.884636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:05:27.884636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:27.884636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:05:27.884636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:27.885971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:05:27.885971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:05:27.885971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:27.885971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:05:27.885971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:27.885971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:05:28.350385 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:05:28.350941 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:05:28.351172 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:05:28.352048 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:05:28.352391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:28.353416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:05:28.353416 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:28.353416 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:05:28.353416 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:05:28.353416 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:28.353416 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:05:28.353416 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:28.353416 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:05:28.355006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:28.355006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:05:28.355006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:05:28.355006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:05:28.355006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:05:28.355006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:05:28.355006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:05:28.355006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:28.355006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:05:28.355006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:28.355006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:05:28.355006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:28.355406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:05:28.355406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:28.355406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:05:28.355406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:28.355406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:05:28.355406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:05:28.355406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:28.355406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:05:28.355406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:28.355406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:05:28.355406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:05:28.355406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:28.355406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:05:28.355766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:28.355766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:05:28.356450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:28.356450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:05:28.356450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:05:28.356450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:05:28.356450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:05:28.356450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:05:28.356450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:05:28.356450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:05:28.356450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:05:28.356450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:05:28.356450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:05:28.356450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:05:28.356450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:05:28.356450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:05:28.356450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:05:28.356450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:05:28.356945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:05:28.356945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:05:28.356945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:28.356945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:05:28.356945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:28.356945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:05:28.356945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:28.356945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:05:28.356945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:28.356945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:05:28.356945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:28.356945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:05:28.356945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:05:28.356945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:28.356945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:05:28.356945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:28.356945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:05:28.356945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:05:28.356945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:28.357446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:05:28.357446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:28.357446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:05:28.363002 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:05:28.363002 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:05:28.363002 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:05:28.363126 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:05:28.363699 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:05:28.364041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:28.364948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:05:28.364948 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:28.364948 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:05:28.364948 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:05:28.364948 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:28.364948 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:05:28.364948 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:28.364948 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:05:28.366190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:28.366190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:05:28.366190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:05:28.366190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:05:28.366190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:28.366190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:05:28.366190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:28.366190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:05:28.366190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:28.366190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:05:28.366190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:28.366190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:05:28.366190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:28.366190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:05:28.366190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:05:28.366578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:28.366578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:05:28.366578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:28.366578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:05:28.366578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:05:28.366578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:28.366578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:05:28.366578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:28.366578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:05:28.367592 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:28.367592 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:05:28.367592 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:05:28.367592 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:05:28.367592 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:05:28.367592 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:05:28.367592 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:05:28.367592 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:05:28.367592 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:05:28.367592 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:05:28.367592 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:28.367592 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:05:28.368241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:28.368241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:05:28.368241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:28.368241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:05:28.368241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:28.368241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:05:28.368241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:28.368241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:05:28.368241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:05:28.368241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:28.368241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:05:28.368241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:28.368241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:05:28.368241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:05:28.369046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:28.369046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:05:28.369046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:28.369046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:05:28.369313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:28.369313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:05:28.369313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:05:28.369313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:05:28.369313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:05:28.369313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:05:28.369313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:05:28.369313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:05:28.369313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:05:28.369313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:05:28.369313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:05:28.369313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:05:28.369313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:05:28.369313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:05:28.369313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:05:28.369313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:05:28.370362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:05:28.370362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:05:28.370362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:05:28.370362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:05:28.370362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:05:28.370362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:28.370362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:05:28.370362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:28.370362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:05:28.370362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:28.370362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:05:28.370362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:28.370362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:05:28.370362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:28.370362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:05:28.370362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:05:28.370362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:28.370362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:05:28.371503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:28.371503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:05:28.371503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:05:28.371503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:28.371503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:05:28.371503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:28.371503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:05:28.388554 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:05:28.389038 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:05:28.389249 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:05:28.389472 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:05:28.390143 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:05:28.390592 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:28.391349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:05:28.391349 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:28.391349 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:05:28.391349 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:05:28.391349 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:28.391349 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:05:28.391349 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:28.391349 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:05:28.392702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:28.392702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:05:28.392702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:05:28.392702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:05:28.392702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:28.392702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:05:28.392702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:28.392702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:05:28.392702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:28.392702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:05:28.392702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:28.392702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:05:28.392702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:28.392702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:05:28.392702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:05:28.393441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:28.393441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:05:28.393441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:28.393441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:05:28.393441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:05:28.393441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:28.393441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:05:28.393441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:28.393441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:05:28.394702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:28.394702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:05:28.394702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:05:28.394702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:05:28.394702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:05:28.394702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:05:28.394702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:05:28.394702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:05:28.394702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:05:28.394702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:28.394702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:05:28.395403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:28.395403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:05:28.395403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:28.395403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:05:28.395403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:28.395403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:05:28.395403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:28.395403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:05:28.395403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:05:28.395403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:28.395403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:05:28.395403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:28.395403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:05:28.395403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:05:28.396175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:28.396175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:05:28.396175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:28.396175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:05:28.396950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:28.396950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:05:28.396950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:05:28.396950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:05:28.396950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:05:28.396950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:05:28.396950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:05:28.396950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:05:28.396950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:05:28.396950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:05:28.396950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:05:28.396950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:05:28.396950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:05:28.396950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:05:28.396950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:05:28.396950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:05:28.396950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:05:28.397874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:05:28.397874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:05:28.397874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:05:28.397874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:28.397874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:05:28.397874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:28.397874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:05:28.397874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:28.397874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:05:28.397874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:28.397874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:05:28.397874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:28.397874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:05:28.397874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:05:28.397874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:28.397874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:05:28.397874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:28.397874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:05:28.397874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:05:28.398951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:28.398951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:05:28.398951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:28.398951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:05:28.398951 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:05:28.398951 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:05:28.398951 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:05:28.398951 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:05:28.399401 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:05:28.399401 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:05:28.399401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:28.399401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:05:28.399401 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:28.399401 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:05:28.399401 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:05:28.399401 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:28.399401 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:05:28.399401 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:28.399401 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:05:28.400053 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:28.400053 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:05:28.400053 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:05:28.400053 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:05:28.400053 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:05:28.400053 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:05:28.400053 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:05:28.400053 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:28.400053 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:05:28.400053 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:28.400053 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:05:28.400053 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:28.400595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:05:28.400595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:28.400595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:05:28.400595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:28.400595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:05:28.400595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:05:28.400595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:28.400595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:05:28.400595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:28.400595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:05:28.400595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:05:28.400595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:28.400595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:05:28.400980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:28.400980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:05:28.400980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:28.400980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:05:28.400980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:05:28.400980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:05:28.400980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:05:28.400980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:05:28.400980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:05:28.400980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:05:28.400980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:05:28.400980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:05:28.400980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:05:28.400980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:05:28.400980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:05:28.401405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:05:28.401405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:05:28.401405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:05:28.401405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:05:28.401405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:05:28.401405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:28.401405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:05:28.401405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:28.401405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:05:28.401405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:28.401405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:05:28.401405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:28.401405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:05:28.401405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:28.401405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:05:28.401405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:05:28.401405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:28.401405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:05:28.401405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:28.401966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:05:28.401966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:05:28.401966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:28.401966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:05:28.401966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:28.401966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:05:32.831251 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:05:32.832169 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:05:32.832298 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:05:32.833146 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:05:32.862879 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:05:32.863290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:32.864527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:05:32.864527 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:32.864527 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:05:32.864527 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:05:32.864527 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:32.864527 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:05:32.864527 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:32.864527 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:05:32.865876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.032 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:32.865876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:05:32.865876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:05:32.865876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:05:32.865876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:05:32.865876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:05:32.865876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:05:32.865876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:32.865876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:05:32.865876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:32.865876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:05:32.865876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:32.866648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:05:32.866648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:32.866648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:05:32.866648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:32.866648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:05:32.866648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:05:32.866648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:32.866648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:05:32.866648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:32.866648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:05:32.866648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:05:32.866648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:32.866648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:05:32.867376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:32.867376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:05:32.867376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:32.867376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:05:32.867376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:05:32.867376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:05:32.867376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:05:32.867376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:05:32.867376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:05:32.867376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:05:32.867376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:05:32.867376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:05:32.867376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:05:32.867376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:05:32.867376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:05:32.868169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:05:32.868169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:05:32.868169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:05:32.868169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:05:32.868169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:05:32.868169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:32.868169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:05:32.868169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:32.868169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:05:32.868169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:32.868169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:05:32.868169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:32.868169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:05:32.868169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:32.868169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:05:32.868169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:05:32.868169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:32.868169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:05:32.868169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:32.869269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:05:32.869269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:05:32.869269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:32.869269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:05:32.869269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:32.869269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:05:33.351187 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:05:33.351872 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:05:33.352132 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:05:33.353140 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:05:33.353539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:33.354377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:05:33.354377 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:33.354377 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:05:33.354377 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:05:33.354377 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:33.354377 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:05:33.354377 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:33.354377 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:05:33.355408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:33.355408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:05:33.355408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:05:33.355408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:05:33.355408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:05:33.355408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:05:33.355408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:05:33.355408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:33.355408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:05:33.355408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:33.355408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:05:33.355408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:33.356118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:05:33.356118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:33.356118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:05:33.356118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:33.356118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:05:33.356118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:05:33.356118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:33.356118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:05:33.356118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:33.356118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:05:33.356118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:05:33.356118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:33.356118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:05:33.356655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:33.356655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:05:33.356912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:33.356912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:05:33.356912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:05:33.356912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:05:33.356912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:05:33.356912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:05:33.356912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:05:33.356912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:05:33.356912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:05:33.356912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:05:33.356912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:05:33.356912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:05:33.356912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:05:33.356912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:05:33.356912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:05:33.356912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:05:33.357539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:05:33.357539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:05:33.357539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:33.357539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:05:33.357539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:33.357539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:05:33.357539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:33.357539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:05:33.357539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:33.357539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:05:33.357539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:33.357539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:05:33.357539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:05:33.357539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:33.357539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:05:33.357539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:33.357539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:05:33.357539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:05:33.357539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:33.358296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:05:33.358296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:33.358296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:05:33.363047 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:05:33.363314 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:05:33.363449 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:05:33.363571 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:05:33.364031 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:05:33.364355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:33.365076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:05:33.365076 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:33.365076 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:05:33.365076 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:05:33.365076 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:33.365076 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:05:33.365076 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:33.365076 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:05:33.365920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:33.365920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:05:33.365920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:05:33.365920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:05:33.365920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:33.365920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:05:33.365920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:33.365920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:05:33.365920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:33.365920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:05:33.365920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:33.365920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:05:33.365920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:33.365920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:05:33.365920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:05:33.366637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:33.366637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:05:33.366637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:33.366637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:05:33.366637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:05:33.366637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:33.366637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:05:33.366637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:33.366637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:05:33.367072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:33.367072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:05:33.367072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:05:33.367072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:05:33.367072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:05:33.367072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:05:33.367072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:05:33.367072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:05:33.367072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:05:33.367072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:05:33.367072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:33.367072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:05:33.367634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:33.367634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:05:33.367634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:33.367634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:05:33.367634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:33.367634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:05:33.367634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:33.367634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:05:33.367634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:05:33.367634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:33.367634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:05:33.367634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:33.367634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:05:33.367634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:05:33.368251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:33.368251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:05:33.368251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:33.368251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:05:33.368251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:33.368251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:05:33.368251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:05:33.368251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:05:33.368251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:05:33.368251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:05:33.368251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:05:33.368251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:05:33.368251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:05:33.368251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:05:33.369152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:05:33.369152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:05:33.369152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:05:33.369152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:05:33.369152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:05:33.369152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:05:33.369152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:05:33.369152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:05:33.369152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:05:33.369152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:05:33.369152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:05:33.369152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:33.369152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:05:33.369152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:33.369152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:05:33.369152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:33.369152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:05:33.369152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:33.369152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:05:33.369993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:33.369993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:05:33.369993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:05:33.369993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:33.369993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:05:33.369993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:33.369993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:05:33.369993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:05:33.369993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:33.369993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:05:33.369993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:33.369993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:05:33.388976 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:05:33.389293 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:05:33.389389 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:05:33.389536 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:05:33.389978 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:05:33.390228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:33.390817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:05:33.390817 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:33.390817 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:05:33.390817 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:05:33.390817 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:33.390817 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:05:33.390817 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:33.390817 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:05:33.391610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:33.391610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:05:33.391610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:05:33.391610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:05:33.391610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:33.391610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:05:33.391610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:33.391610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:05:33.391610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:33.391610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:05:33.391610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:33.391610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:05:33.391610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:33.391610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:05:33.391610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:05:33.392076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:33.392076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:05:33.392076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:33.392076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:05:33.392076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:05:33.392076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:33.392076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:05:33.392076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:33.392076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:05:33.392524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:33.392524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:05:33.392524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:05:33.392524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:05:33.392524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:05:33.392524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:05:33.392524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:05:33.392524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:05:33.392524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:05:33.392524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:33.392524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:05:33.392884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:33.392884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:05:33.392884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:33.392884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:05:33.392884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:33.392884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:05:33.392884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:33.392884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:05:33.392884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:05:33.392884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:33.392884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:05:33.392884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:33.392884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:05:33.392884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:05:33.393279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:33.393279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:05:33.393279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:33.393279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:05:33.393902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:33.393902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:05:33.393902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:05:33.393902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:05:33.393902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:05:33.393902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:05:33.393902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:05:33.393902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:05:33.393902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:05:33.393902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:05:33.393902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:05:33.393902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:05:33.393902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:05:33.393902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:05:33.393902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:05:33.393902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:05:33.393902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:05:33.394372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:05:33.394372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:05:33.394372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:05:33.394372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:33.394372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:05:33.394372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:33.394372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:05:33.394372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:33.394372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:05:33.394372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:33.394372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:05:33.394372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:33.394372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:05:33.394372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:05:33.394372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:33.394372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:05:33.394372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:33.394372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:05:33.394372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:05:33.394945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:33.394945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:05:33.394945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:33.394945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:05:33.397823 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:05:33.398153 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:05:33.398268 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:05:33.398443 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:05:33.399555 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:05:33.399966 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:05:33.400208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:33.401034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:05:33.401034 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:33.401034 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:05:33.401034 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:05:33.401034 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:33.401034 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:05:33.401034 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:33.401034 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:05:33.401979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:33.401979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:05:33.401979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:05:33.401979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:05:33.401979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:05:33.401979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:05:33.401979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:05:33.401979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:33.401979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:05:33.401979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:33.401979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:05:33.401979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:33.402424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:05:33.402424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:33.402424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:05:33.402424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:33.402424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:05:33.402424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:05:33.402424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:33.402424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:05:33.402424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:33.402424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:05:33.402424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:05:33.402424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:33.402424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:05:33.402794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:33.402794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:05:33.403108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:33.403108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:05:33.403108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:05:33.403108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:05:33.403108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:05:33.403108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:05:33.403108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:05:33.403108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:05:33.403108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:05:33.403108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:05:33.403108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:05:33.403108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:05:33.403108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:05:33.403108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:05:33.403108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:05:33.403108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:05:33.403607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:05:33.403607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:05:33.403607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:33.403607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:05:33.403607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:33.403607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:05:33.403607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:33.403607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:05:33.403607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:33.403607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:05:33.403607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:33.403607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:05:33.403607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:05:33.403607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:33.403607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:05:33.403607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:33.403607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:05:33.403607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:05:33.403607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:33.404149 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:05:33.404149 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:33.404149 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:05:37.832699 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:05:37.833559 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:05:37.833559 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:05:37.834792 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:05:37.836631 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG dbcounter [-] [106314] Writing DB stats neutron:SELECT=580,neutron:INSERT=1,neutron:UPDATE=2,neutron:DELETE=1 {{(pid=106314) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 09 09:05:37.871479 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:05:37.871710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:37.872193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:05:37.872193 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:37.872193 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:05:37.872193 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:05:37.872193 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:37.872193 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:05:37.872193 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:37.872193 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:05:37.872861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:37.872861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:05:37.872861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:05:37.872861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:05:37.872861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:05:37.872861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:05:37.872861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:05:37.872861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:37.872861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:05:37.872861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:37.872861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:05:37.872861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:37.873528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:05:37.873528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:37.873528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:05:37.873528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:37.873528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:05:37.873528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:05:37.873528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:37.873528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:05:37.873528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:37.873528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:05:37.873528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:05:37.873528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:37.873528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:05:37.874212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:37.874212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:05:37.874212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:37.874212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:05:37.874212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:05:37.874212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:05:37.874212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:05:37.874212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:05:37.874212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:05:37.874212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:05:37.874212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:05:37.874212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:05:37.874212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:05:37.874212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:05:37.874212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:05:37.875056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:05:37.875056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:05:37.875056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:05:37.875056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:05:37.875056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:05:37.875056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:37.875056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:05:37.875056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:37.875056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:05:37.875056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:37.875056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:05:37.875056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:37.875056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:05:37.875056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:37.875056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:05:37.875056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:05:37.875056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:37.875056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:05:37.875056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:37.875945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:05:37.875945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:05:37.875945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:37.875945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:05:37.875945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:37.875945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:05:38.351712 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:05:38.352182 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:05:38.352322 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:05:38.353584 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:05:38.353949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:38.354873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:05:38.354873 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:38.354873 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:05:38.354873 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:05:38.354873 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:38.354873 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:05:38.354873 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:38.354873 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:05:38.356696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:38.356696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:05:38.356696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:05:38.356696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:05:38.356696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:05:38.356696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:05:38.356696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:05:38.356696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:38.356696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:05:38.356696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:38.356696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:05:38.356696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:38.357351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:05:38.357351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:38.357351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:05:38.357351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:38.357351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:05:38.357351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:05:38.357351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:38.357351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:05:38.357351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:38.357351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:05:38.357351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:05:38.357351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:38.357351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:05:38.357966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:38.357966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:05:38.359072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:38.359072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:05:38.359072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:05:38.359072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:05:38.359072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:05:38.359072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:05:38.359072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:05:38.359072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:05:38.359072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:05:38.359072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:05:38.359072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:05:38.359072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:05:38.359072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:05:38.359072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:05:38.359072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:05:38.359072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:05:38.359907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:05:38.359907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:05:38.359907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:38.359907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:05:38.359907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:38.359907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:05:38.359907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:38.359907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:05:38.359907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:38.359907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:05:38.359907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:38.359907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:05:38.359907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:05:38.359907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:38.359907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:05:38.359907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:38.359907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:05:38.359907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:05:38.359907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:38.360889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:05:38.360889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:38.360889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:05:38.363621 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:05:38.363976 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:05:38.364096 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:05:38.364218 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:05:38.364806 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:05:38.365118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:38.366042 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:05:38.366042 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:38.366042 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:05:38.366042 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:05:38.366042 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:38.366042 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:05:38.366042 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:38.366042 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:05:38.367028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:38.367028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:05:38.367028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:05:38.367028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:05:38.367028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:38.367028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:05:38.367028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:38.367028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:05:38.367028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:38.367028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:05:38.367028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:38.367028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:05:38.367028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:38.367028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:05:38.367028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:05:38.367486 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:38.367486 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:05:38.367486 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:38.367486 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:05:38.367486 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:05:38.367486 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:38.367486 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:05:38.367486 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:38.367486 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:05:38.368354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:38.368354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:05:38.368354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:05:38.368354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:05:38.368354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:05:38.368354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:05:38.368354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:05:38.368354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:05:38.368354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:05:38.368354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:05:38.368354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:38.368354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:05:38.368803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:38.368803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:05:38.368803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:38.368803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:05:38.368803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:38.368803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:05:38.368803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:38.368803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:05:38.368803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:05:38.368803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:38.368803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:05:38.368803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:38.368803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:05:38.368803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:05:38.369538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:38.369538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:05:38.369538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:38.369538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:05:38.369727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:38.369727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:05:38.369727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:05:38.369727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:05:38.369727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:05:38.369727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:05:38.369727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:05:38.369727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:05:38.369727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:05:38.369727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:05:38.369727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:05:38.369727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:05:38.369727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:05:38.369727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:05:38.369727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:05:38.369727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:05:38.370463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:05:38.370463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:05:38.370463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:05:38.370463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:05:38.370463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:05:38.370463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:38.370463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:05:38.370463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:38.370463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:05:38.370463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:38.370463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:05:38.370463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:38.370463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:05:38.370463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:38.370463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:05:38.370463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:05:38.370463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:38.370463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:05:38.371439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:38.371439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:05:38.371439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:05:38.371439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:38.371439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:05:38.371439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:38.371439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:05:38.389501 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:05:38.390047 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:05:38.390241 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:05:38.390448 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:05:38.390975 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:05:38.391355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:38.392352 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:05:38.392352 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:38.392352 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:05:38.392352 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:05:38.392352 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:38.392352 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:05:38.392352 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:38.392352 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:05:38.394182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:38.394182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:05:38.394182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:05:38.394182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:05:38.394182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:38.394182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:05:38.394182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:38.394182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:05:38.394182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:38.394182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:05:38.394182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:38.394182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:05:38.394182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:38.394182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:05:38.394182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:05:38.394842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:38.394842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:05:38.394842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:38.394842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:05:38.394842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:05:38.394842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:38.394842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:05:38.394842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:38.394842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:05:38.396899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:38.396899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:05:38.396899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:05:38.396899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:05:38.396899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:05:38.396899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:05:38.396899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:05:38.396899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:05:38.396899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:05:38.396899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:38.396899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:05:38.397492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:38.397492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:05:38.397492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:38.397492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:05:38.397492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:38.397492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:05:38.397492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:38.397492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:05:38.397492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:05:38.397492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:38.397492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:05:38.397492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:38.397492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:05:38.397492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:05:38.398167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:38.398167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:05:38.398167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:38.398167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:05:38.399363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:38.399363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:05:38.399363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:05:38.399363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:05:38.399363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:05:38.399363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:05:38.399363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:05:38.399363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:05:38.399363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:05:38.399363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:05:38.399363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:05:38.399363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:05:38.399363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:05:38.399363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:05:38.399363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:05:38.399363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:05:38.399363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:05:38.400205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:05:38.400205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:05:38.400205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:05:38.400205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:38.400205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:05:38.400205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:38.400205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:05:38.400205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:38.400205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:05:38.400205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:38.400205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:05:38.400205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:38.400205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:05:38.400205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:05:38.400205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:38.400205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:05:38.400205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:38.400205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:05:38.400205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:05:38.401135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:38.401135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:05:38.401135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:38.401135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:05:38.401135 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:05:38.401135 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:05:38.401135 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:05:38.401135 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:05:38.401406 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:05:38.401503 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:05:38.401685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:38.402518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:05:38.402518 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:38.402518 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:05:38.402518 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:05:38.402518 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:38.402518 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:05:38.402518 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:38.402518 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:05:38.403338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:38.403338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:05:38.403338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:05:38.403338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:05:38.403338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:05:38.403338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:05:38.403338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:05:38.403338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:38.403338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:05:38.403338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:38.403338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:05:38.403338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:38.404025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:05:38.404025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:38.404025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:05:38.404025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:38.404025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:05:38.404025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:05:38.404025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:38.404025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:05:38.404025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:38.404025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:05:38.404025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:05:38.404025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:38.404025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:05:38.404685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:38.404685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:05:38.404685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:38.404685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:05:38.404685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:05:38.404685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:05:38.404685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:05:38.404685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:05:38.404685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:05:38.404685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:05:38.404685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:05:38.404685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:05:38.404685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:05:38.404685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:05:38.404685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:05:38.405435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:05:38.405435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:05:38.405435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:05:38.405435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:05:38.405435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:05:38.405435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:38.405435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:05:38.405435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:38.405435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:05:38.405435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:38.405435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:05:38.405435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:38.405435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:05:38.405435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:38.405435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:05:38.405435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:05:38.405435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:38.405435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:05:38.405435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:38.406358 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:05:38.406358 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:05:38.406358 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:38.406358 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:05:38.406358 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:38.406358 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:05:42.833249 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:05:42.834060 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:05:42.834155 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:05:42.835648 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:05:42.876859 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:05:42.877079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:42.877929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:05:42.877929 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:42.877929 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:05:42.877929 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:05:42.877929 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:42.877929 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:05:42.877929 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:42.877929 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:05:42.878695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.044 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:42.878695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:05:42.878695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:05:42.878695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:05:42.878695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:05:42.878695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:05:42.878695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:05:42.878695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:42.878695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:05:42.878695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:42.878695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:05:42.878695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:42.879352 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:05:42.879352 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:42.879352 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:05:42.879352 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:42.879352 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:05:42.879352 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:05:42.879352 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:42.879352 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:05:42.879352 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:42.879352 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:05:42.879352 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:05:42.879352 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:42.879352 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:05:42.879974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:42.879974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:05:42.879974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:42.879974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:05:42.879974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:05:42.879974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:05:42.879974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:05:42.879974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:05:42.879974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:05:42.879974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:05:42.879974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:05:42.879974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:05:42.879974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:05:42.879974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:05:42.879974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:05:42.880816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:05:42.880816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:05:42.880816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:05:42.880816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:05:42.880816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:05:42.880816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:42.880816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:05:42.880816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:42.880816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:05:42.880816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:42.880816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:05:42.880816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:42.880816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:05:42.880816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:42.880816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:05:42.880816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:05:42.880816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:42.880816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:05:42.880816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:42.881435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:05:42.881435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:05:42.881435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:42.881435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:05:42.881435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:42.881435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:05:43.352400 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:05:43.352945 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:05:43.353041 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:05:43.354162 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:05:43.354488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:43.355357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:05:43.355357 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:43.355357 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:05:43.355357 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:05:43.355357 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:43.355357 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:05:43.355357 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:43.355357 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:05:43.356885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:43.356885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:05:43.356885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:05:43.356885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:05:43.356885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:05:43.356885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:05:43.356885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:05:43.356885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:43.356885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:05:43.356885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:43.356885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:05:43.356885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:43.357456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:05:43.357456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:43.357456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:05:43.357456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:43.357456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:05:43.357456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:05:43.357456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:43.357456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:05:43.357456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:43.357456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:05:43.357456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:05:43.357456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:43.357456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:05:43.358104 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:43.358104 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:05:43.358564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:43.358564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:05:43.358564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:05:43.358564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:05:43.358564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:05:43.358564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:05:43.358564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:05:43.358564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:05:43.358564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:05:43.358564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:05:43.358564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:05:43.358564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:05:43.358564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:05:43.358564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:05:43.358564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:05:43.358564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:05:43.359438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:05:43.359438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:05:43.359438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:43.359438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:05:43.359438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:43.359438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:05:43.359438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:43.359438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:05:43.359438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:43.359438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:05:43.359438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:43.359438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:05:43.359438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:05:43.359438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:43.359438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:05:43.359438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:43.359438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:05:43.359438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:05:43.359438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:43.360381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:05:43.360381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:43.360381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:05:43.364143 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:05:43.364422 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:05:43.364539 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:05:43.364660 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:05:43.365140 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:05:43.365443 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:43.366531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:05:43.366531 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:43.366531 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:05:43.366531 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:05:43.366531 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:43.366531 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:05:43.366531 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:43.366531 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:05:43.367328 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:43.367328 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:05:43.367328 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:05:43.367328 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:05:43.367328 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:43.367328 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:05:43.367328 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:43.367328 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:05:43.367328 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:43.367328 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:05:43.367328 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:43.367328 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:05:43.367328 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:43.367328 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:05:43.367328 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:05:43.367862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:43.367862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:05:43.367862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:43.367862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:05:43.367862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:05:43.367862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:43.367862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:05:43.367862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:43.367862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:05:43.368137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:43.368137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:05:43.368137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:05:43.368137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:05:43.368137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:05:43.368137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:05:43.368137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:05:43.368137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:05:43.368137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:05:43.368137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:05:43.368137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:43.368137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:05:43.368541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:43.368541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:05:43.368541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:43.368541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:05:43.368541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:43.368541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:05:43.368541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:43.368541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:05:43.368541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:05:43.368541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:43.368541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:05:43.368541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:43.368541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:05:43.368541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:05:43.368994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:43.368994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:05:43.368994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:43.368994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:05:43.369185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:43.369185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:05:43.369185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:05:43.369185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:05:43.369185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:05:43.369185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:05:43.369185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:05:43.369185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:05:43.369185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:05:43.369185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:05:43.369185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:05:43.369185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:05:43.369185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:05:43.369185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:05:43.369185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:05:43.369185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:05:43.369820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:05:43.369820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:05:43.369820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:05:43.369820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:05:43.369820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:05:43.369820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:43.369820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:05:43.369820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:43.369820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:05:43.369820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:43.369820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:05:43.369820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:43.369820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:05:43.369820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:43.369820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:05:43.369820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:05:43.369820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:43.369820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:05:43.370364 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:43.370364 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:05:43.370364 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:05:43.370364 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:43.370364 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:05:43.370364 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:43.370364 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:05:43.390127 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:05:43.390440 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:05:43.390614 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:05:43.390864 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:05:43.392558 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:05:43.392845 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:43.393533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:05:43.393533 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:43.393533 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:05:43.393533 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:05:43.393533 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:43.393533 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:05:43.393533 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:43.393533 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:05:43.394765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:43.394765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:05:43.394765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:05:43.394765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:05:43.394765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:43.394765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:05:43.394765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:43.394765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:05:43.394765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:43.394765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:05:43.394765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:43.394765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:05:43.394765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:43.394765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:05:43.394765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:05:43.395454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:43.395454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:05:43.395454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:43.395454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:05:43.395454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:05:43.395454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:43.395454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:05:43.395454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:43.395454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:05:43.396580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:43.396580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:05:43.396580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:05:43.396580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:05:43.396580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:05:43.396580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:05:43.396580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:05:43.396580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:05:43.396580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:05:43.396580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:43.396580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:05:43.397356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:43.397356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:05:43.397356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:43.397356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:05:43.397356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:43.397356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:05:43.397356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:43.397356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:05:43.397356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:05:43.397356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:43.397356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:05:43.397356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:43.397356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:05:43.397356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:05:43.398204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:43.398204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:05:43.398204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:43.398204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:05:43.398516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:43.398516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:05:43.398516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:05:43.398516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:05:43.398516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:05:43.398516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:05:43.398516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:05:43.398516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:05:43.398516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:05:43.398516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:05:43.398516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:05:43.398516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:05:43.398516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:05:43.398516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:05:43.398516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:05:43.398516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:05:43.398516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:05:43.399259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:05:43.399259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:05:43.399259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:05:43.399259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:43.399259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:05:43.399259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:43.399259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:05:43.399259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:43.399259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:05:43.399259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:43.399259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:05:43.399259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:43.399259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:05:43.399259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:05:43.399259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:43.399259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:05:43.399259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:43.399259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:05:43.399259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:05:43.399780 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:43.399780 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:05:43.399780 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:43.399780 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:05:43.400144 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:05:43.400461 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:05:43.400602 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:05:43.400802 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:05:43.401357 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:05:43.401839 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:05:43.402088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:43.402624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:05:43.402624 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:43.402624 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:05:43.402624 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:05:43.402624 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:43.402624 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:05:43.402624 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:43.402624 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:05:43.403654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:43.403654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:05:43.403654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:05:43.403654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:05:43.403654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:05:43.403654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:05:43.403654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:05:43.403654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:43.403654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:05:43.403654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:43.403654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:05:43.403654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:43.404046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:05:43.404046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:43.404046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:05:43.404046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:43.404046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:05:43.404046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:05:43.404046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:43.404046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:05:43.404046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:43.404046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:05:43.404046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:05:43.404046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:43.404046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:05:43.404409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:43.404409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:05:43.404633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:43.404633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:05:43.404633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:05:43.404633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:05:43.404633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:05:43.404633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:05:43.404633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:05:43.404633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:05:43.404633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:05:43.404633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:05:43.404633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:05:43.404633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:05:43.404633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:05:43.404633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:05:43.404633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:05:43.404633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:05:43.405244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:05:43.405244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:05:43.405244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:43.405244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:05:43.405244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:43.405244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:05:43.405244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:43.405244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:05:43.405244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:43.405244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:05:43.405244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:43.405244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:05:43.405244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:05:43.405244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:43.405244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:05:43.405244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:43.405244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:05:43.405244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:05:43.405244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:43.405992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:05:43.405992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:43.405992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:05:47.834148 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:05:47.835063 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:05:47.835172 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:05:47.836398 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:05:47.874739 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:05:47.875121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:47.876042 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:05:47.876042 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:47.876042 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:05:47.876042 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:05:47.876042 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:47.876042 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:05:47.876042 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:47.876042 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:05:47.876849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:47.876849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:05:47.876849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:05:47.876849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:05:47.876849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:05:47.876849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:05:47.876849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:05:47.876849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:47.876849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:05:47.876849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:47.876849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:05:47.876849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:47.877307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:05:47.877307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:47.877307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:05:47.877307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:47.877307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:05:47.877307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:05:47.877307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:47.877307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:05:47.877307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:47.877307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:05:47.877307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:05:47.877307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:47.877307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:05:47.877880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:47.877880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:05:47.877880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:47.877880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:05:47.877880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:05:47.877880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:05:47.877880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:05:47.877880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:05:47.877880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:05:47.877880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:05:47.877880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:05:47.877880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:05:47.877880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:05:47.877880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:05:47.877880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:05:47.878520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:05:47.878520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:05:47.878520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:05:47.878520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:05:47.878520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:05:47.878520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:47.878520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:05:47.878520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:47.878520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:05:47.878520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:47.878520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:05:47.878520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:47.878520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:05:47.878520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:47.878520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:05:47.878520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:05:47.878520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:47.878520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:05:47.878520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:47.879339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:05:47.879339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:05:47.879339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:47.879339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:05:47.879339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:47.879339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:05:48.353292 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:05:48.354088 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:05:48.354285 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:05:48.355148 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:05:48.355500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:48.356631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:05:48.356631 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:48.356631 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:05:48.356631 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:05:48.356631 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:48.356631 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:05:48.356631 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:48.356631 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:05:48.358817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:48.358817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:05:48.358817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:05:48.358817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:05:48.358817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:05:48.358817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:05:48.358817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:05:48.358817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:48.358817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:05:48.358817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:48.358817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:05:48.358817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:48.359456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:05:48.359456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:48.359456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:05:48.359456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:48.359456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:05:48.359456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:05:48.359456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:48.359456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:05:48.359456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:48.359456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:05:48.359456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:05:48.359456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:48.359456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:05:48.360070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:48.360070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:05:48.360070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:48.360070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:05:48.360070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:05:48.360070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:05:48.360070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:05:48.360070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:05:48.360070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:05:48.360070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:05:48.360070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:05:48.360070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:05:48.360070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:05:48.360070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:05:48.360070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:05:48.360759 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:05:48.360759 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:05:48.360759 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:05:48.360759 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:05:48.360759 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:05:48.360759 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:48.360759 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:05:48.360759 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:48.360759 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:05:48.360759 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:48.360759 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:05:48.360759 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:48.360759 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:05:48.360759 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:48.360759 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:05:48.360759 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:05:48.360759 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:48.360759 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:05:48.360759 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:48.361628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:05:48.361628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:05:48.361628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:48.361628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:05:48.361628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:48.361628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:05:48.364760 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:05:48.365050 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:05:48.365159 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:05:48.365346 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:05:48.366912 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:05:48.367141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:48.367704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:05:48.367704 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:48.367704 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:05:48.367704 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:05:48.367704 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:48.367704 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:05:48.367704 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:48.367704 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:05:48.368478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:48.368478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:05:48.368478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:05:48.368478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:05:48.368478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:48.368478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:05:48.368478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:48.368478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:05:48.368478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:48.368478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:05:48.368478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:48.368478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:05:48.368478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:48.368478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:05:48.368478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:05:48.369316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:48.369316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:05:48.369316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:48.369316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:05:48.369316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:05:48.369316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:48.369316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:05:48.369316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:48.369316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:05:48.369316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:48.369316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:05:48.369316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:05:48.369316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:05:48.369316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:05:48.370181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:05:48.370181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:05:48.370181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:05:48.370181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:05:48.370181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:05:48.370181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:48.370181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:05:48.370181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:48.370181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:05:48.370181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:48.370181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:05:48.370181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:48.370181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:05:48.370905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:48.370905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:05:48.370905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:05:48.370905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:48.370905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:05:48.370905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:48.370905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:05:48.370905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:05:48.370905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:48.370905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:05:48.370905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:48.370905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:05:48.371570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:48.371570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:05:48.371570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:05:48.371570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:05:48.371570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:05:48.371570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:05:48.371570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:05:48.371570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:05:48.371570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:05:48.371570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:05:48.371570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:05:48.371570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:05:48.371570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:05:48.371570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:05:48.371570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:05:48.371570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:05:48.372133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:05:48.372133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:05:48.372133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:05:48.372133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:05:48.372133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:05:48.372133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:48.372133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:05:48.372133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:48.372133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:05:48.372133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:48.372133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:05:48.372133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:48.372133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:05:48.372133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:48.372133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:05:48.372133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:05:48.372133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:48.372133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:05:48.372578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:48.372578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:05:48.372578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:05:48.372578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:48.372578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:05:48.372578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:48.372578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:05:48.390681 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:05:48.392040 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:05:48.392284 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:05:48.392611 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:05:48.393267 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:05:48.393859 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:48.394673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:05:48.394673 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:48.394673 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:05:48.394673 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:05:48.394673 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:48.394673 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:05:48.394673 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:48.394673 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:05:48.396441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:48.396441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:05:48.396441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:05:48.396441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:05:48.396441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:48.396441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:05:48.396441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:48.396441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:05:48.396441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:48.396441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:05:48.396441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:48.396441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:05:48.396441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:48.396441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:05:48.396441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:05:48.397409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:48.397409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:05:48.397409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:48.397409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:05:48.397409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:05:48.397409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:48.397409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:05:48.397409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:48.397409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:05:48.398550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:48.398550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:05:48.398550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:05:48.398550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:05:48.398550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:05:48.398550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:05:48.398550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:05:48.398550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:05:48.398550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:05:48.398550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:48.398550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:05:48.399264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:48.399264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:05:48.399264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:48.399264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:05:48.399264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:48.399264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:05:48.399264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:48.399264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:05:48.399264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:05:48.399264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:48.399264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:05:48.399264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:48.399264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:05:48.399264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:05:48.400134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:48.400134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:05:48.400134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:48.400134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:05:48.400134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:48.400134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:05:48.400134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:05:48.400134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:05:48.400134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:05:48.400134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:05:48.400134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:05:48.400134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:05:48.400134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:05:48.400134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:05:48.400811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:05:48.400811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:05:48.400811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:05:48.400811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:05:48.400811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:05:48.400811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:05:48.400811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:05:48.400811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:05:48.400811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:05:48.400811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:05:48.400811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:48.400811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:05:48.400811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:48.400811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:05:48.400811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:48.400811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:05:48.400811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:48.400811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:05:48.400811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:48.401836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:05:48.401836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:05:48.401836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:48.401836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:05:48.401836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:48.401836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:05:48.401836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:05:48.401836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:48.401836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:05:48.401836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:48.401836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:05:48.401836 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:05:48.401836 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:05:48.402370 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:05:48.402370 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:05:48.402370 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:05:48.402370 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:05:48.402370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:48.402537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:05:48.402537 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:48.402537 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:05:48.402537 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:05:48.402537 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:48.402537 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:05:48.402537 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:48.402537 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:05:48.403281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:48.403281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:05:48.403281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:05:48.403281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:05:48.403281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:05:48.403281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:05:48.403281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:05:48.403281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:48.403281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:05:48.403281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:48.403281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:05:48.403281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:48.403636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:05:48.403636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:48.403636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:05:48.403636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:48.403636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:05:48.403636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:05:48.403636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:48.403636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:05:48.403636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:48.403636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:05:48.403636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:05:48.403636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:48.403636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:05:48.403984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:48.403984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:05:48.404548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:48.404548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:05:48.404548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:05:48.404548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:05:48.404548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:05:48.404548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:05:48.404548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:05:48.404548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:05:48.404548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:05:48.404548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:05:48.404548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:05:48.404548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:05:48.404548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:05:48.404548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:05:48.404548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:05:48.404548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:05:48.404919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:05:48.404919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:05:48.404919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:48.404919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:05:48.404919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:48.404919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:05:48.404919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:48.404919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:05:48.404919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:48.404919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:05:48.404919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:48.404919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:05:48.404919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:05:48.404919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:48.404919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:05:48.404919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:48.404919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:05:48.404919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:05:48.404919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:48.405244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:05:48.405244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:48.405244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:05:52.834982 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:05:52.835677 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:05:52.835923 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:05:52.837516 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:05:52.866665 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:05:52.867116 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:52.868071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:05:52.868071 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:52.868071 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:05:52.868071 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:05:52.868071 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:52.868071 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:05:52.868071 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:52.868071 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:05:52.869063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.032 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:52.869063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:05:52.869063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:05:52.869063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:05:52.869063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:05:52.869063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:05:52.869063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:05:52.869063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:52.869063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:05:52.869063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:52.869063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:05:52.869063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:52.869563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:05:52.869563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:52.869563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:05:52.869563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:52.869563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:05:52.869563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:05:52.869563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:52.869563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:05:52.869563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:52.869563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:05:52.869563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:05:52.869563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:52.869563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:05:52.869837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:52.869837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:05:52.870292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:52.870292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:05:52.870292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:05:52.870292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:05:52.870292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:05:52.870292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:05:52.870292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:05:52.870292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:05:52.870292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:05:52.870292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:05:52.870292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:05:52.870292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:05:52.870292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:05:52.870292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:05:52.870292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:05:52.870292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:05:52.870630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:05:52.870630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:05:52.870630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:52.870630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:05:52.870630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:52.870630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:05:52.870630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:52.870630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:05:52.870630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:52.870630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:05:52.870630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:52.870630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:05:52.870630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:05:52.870630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:52.870630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:05:52.870630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:52.870630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:05:52.870630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:05:52.870630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:52.870995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:05:52.870995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:52.870995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:05:53.354240 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:05:53.354839 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:05:53.354839 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:05:53.355906 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:05:53.356240 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:53.357224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:05:53.357224 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:53.357224 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:05:53.357224 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:05:53.357224 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:53.357224 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:05:53.357224 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:53.357224 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:05:53.359047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:53.359047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:05:53.359047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:05:53.359047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:05:53.359047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:05:53.359047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:05:53.359047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:05:53.359047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:53.359047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:05:53.359047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:53.359047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:05:53.359047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:53.359570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:05:53.359570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:53.359570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:05:53.359570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:53.359570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:05:53.359570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:05:53.359570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:53.359570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:05:53.359570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:53.359570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:05:53.359570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:05:53.359570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:53.359570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:05:53.359934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:53.359934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:05:53.360861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:53.360861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:05:53.360861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:05:53.360861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:05:53.360861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:05:53.360861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:05:53.360861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:05:53.360861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:05:53.360861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:05:53.360861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:05:53.360861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:05:53.360861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:05:53.360861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:05:53.360861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:05:53.360861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:05:53.360861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:05:53.362006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:05:53.362006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:05:53.362006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:53.362006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:05:53.362006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:53.362006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:05:53.362006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:53.362006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:05:53.362006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:53.362006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:05:53.362006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:53.362006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:05:53.362006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:05:53.362006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:53.362006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:05:53.362006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:53.362006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:05:53.362006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:05:53.362006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:53.362679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:05:53.362679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:53.362679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:05:53.366429 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:05:53.366726 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:05:53.366877 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:05:53.367017 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:05:53.367492 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:05:53.367989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:53.368781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:05:53.368781 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:53.368781 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:05:53.368781 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:05:53.368781 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:53.368781 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:05:53.368781 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:53.368781 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:05:53.369527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:53.369527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:05:53.369527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:05:53.369527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:05:53.369527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:53.369527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:05:53.369527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:53.369527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:05:53.369527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:53.369527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:05:53.369527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:53.369527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:05:53.369527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:53.369527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:05:53.369527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:05:53.369944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:53.369944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:05:53.369944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:53.369944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:05:53.369944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:05:53.369944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:53.369944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:05:53.369944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:53.369944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:05:53.370370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:53.370370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:05:53.370370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:05:53.370370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:05:53.370370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:05:53.370370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:05:53.370370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:05:53.370370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:05:53.370370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:05:53.370370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:05:53.370370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:53.370370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:05:53.370679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:53.370679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:05:53.370679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:53.370679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:05:53.370679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:53.370679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:05:53.370679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:53.370679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:05:53.370679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:05:53.370679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:53.370679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:05:53.370679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:53.370679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:05:53.370679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:05:53.371065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:53.371065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:05:53.371065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:53.371065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:05:53.371660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:53.371660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:05:53.371660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:05:53.371660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:05:53.371660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:05:53.371660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:05:53.371660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:05:53.371660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:05:53.371660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:05:53.371660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:05:53.371660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:05:53.371660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:05:53.371660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:05:53.371660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:05:53.371660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:05:53.371660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:05:53.372397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:05:53.372397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:05:53.372397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:05:53.372397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:05:53.372397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:05:53.372397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:53.372397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:05:53.372397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:53.372397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:05:53.372397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:53.372397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:05:53.372397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:53.372397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:05:53.372397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:53.372397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:05:53.372397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:05:53.372397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:53.372397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:05:53.373453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:53.373453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:05:53.373453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:05:53.373453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:53.373453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:05:53.373453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:53.373453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:05:53.391989 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:05:53.392358 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:05:53.392560 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:05:53.392681 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:05:53.393170 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:05:53.393417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:53.394066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:05:53.394066 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:53.394066 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:05:53.394066 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:05:53.394066 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:53.394066 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:05:53.394066 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:53.394066 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:05:53.395335 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:53.395335 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:05:53.395335 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:05:53.395335 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:05:53.395335 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:53.395335 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:05:53.395335 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:53.395335 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:05:53.395335 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:53.395335 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:05:53.395335 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:53.395335 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:05:53.395335 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:53.395335 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:05:53.395335 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:05:53.396019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:53.396019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:05:53.396019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:53.396019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:05:53.396019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:05:53.396019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:53.396019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:05:53.396019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:53.396019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:05:53.397001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:53.397001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:05:53.397001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:05:53.397001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:05:53.397001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:05:53.397001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:05:53.397001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:05:53.397001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:05:53.397001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:05:53.397001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:53.397001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:05:53.397721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:53.397721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:05:53.397721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:53.397721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:05:53.397721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:53.397721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:05:53.397721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:53.397721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:05:53.397721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:05:53.397721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:53.397721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:05:53.397721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:53.397721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:05:53.397721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:05:53.398338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:53.398338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:05:53.398338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:53.398338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:05:53.399065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:53.399065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:05:53.399065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:05:53.399065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:05:53.399065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:05:53.399065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:05:53.399065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:05:53.399065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:05:53.399065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:05:53.399065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:05:53.399065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:05:53.399065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:05:53.399065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:05:53.399065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:05:53.399065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:05:53.399065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:05:53.399065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:05:53.399868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:05:53.399868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:05:53.399868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:05:53.399868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:53.399868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:05:53.399868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:53.399868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:05:53.399868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:53.399868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:05:53.399868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:53.399868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:05:53.399868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:53.399868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:05:53.399868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:05:53.399868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:53.399868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:05:53.399868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:53.399868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:05:53.399868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:05:53.400713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:53.400713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:05:53.400713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:53.400713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:05:53.401016 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:05:53.401427 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:05:53.401570 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:05:53.401738 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:05:53.402253 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:05:53.402605 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:05:53.402818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:53.403521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:05:53.403521 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:53.403521 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:05:53.403521 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:05:53.403521 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:53.403521 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:05:53.403521 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:53.403521 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:05:53.404357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:53.404357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:05:53.404357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:05:53.404357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:05:53.404357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:05:53.404357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:05:53.404357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:05:53.404357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:53.404357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:05:53.404357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:53.404357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:05:53.404357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:53.404762 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:05:53.404762 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:53.404762 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:05:53.404762 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:53.404762 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:05:53.404762 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:05:53.404762 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:53.404762 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:05:53.404762 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:53.404762 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:05:53.404762 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:05:53.404762 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:53.404762 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:05:53.405099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:53.405099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:05:53.405334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:53.405334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:05:53.405334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:05:53.405334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:05:53.405334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:05:53.405334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:05:53.405334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:05:53.405334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:05:53.405334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:05:53.405334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:05:53.405334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:05:53.405334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:05:53.405334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:05:53.405334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:05:53.405334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:05:53.405334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:05:53.405773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:05:53.405773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:05:53.405773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:53.405773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:05:53.405773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:53.405773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:05:53.405773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:53.405773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:05:53.405773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:53.405773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:05:53.405773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:53.405773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:05:53.405773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:05:53.405773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:53.405773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:05:53.405773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:53.405773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:05:53.405773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:05:53.405773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:53.406282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:05:53.406282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:53.406282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:05:57.836420 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:05:57.837232 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:05:57.837232 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:05:57.838436 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:05:57.890938 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:05:57.891407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:57.892436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:05:57.892436 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:57.892436 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:05:57.892436 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:05:57.892436 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:57.892436 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:05:57.892436 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:57.892436 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:05:57.893449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.056 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:57.893449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:05:57.893449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:05:57.893449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:05:57.893449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:05:57.893449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:05:57.893449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:05:57.893449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:57.893449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:05:57.893449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:57.893449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:05:57.893449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:57.894133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:05:57.894133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:57.894133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:05:57.894133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:57.894133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:05:57.894133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:05:57.894133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:57.894133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:05:57.894133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:57.894133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:05:57.894133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:05:57.894133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:57.894133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:05:57.895252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:57.895252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:05:57.895252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:57.895252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:05:57.895252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:05:57.895252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:05:57.895252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:05:57.895252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:05:57.895252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:05:57.895252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:05:57.895252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:05:57.895252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:05:57.895252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:05:57.895252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:05:57.895252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:05:57.895944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:05:57.895944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:05:57.895944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:05:57.895944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:05:57.895944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:05:57.895944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:57.895944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:05:57.895944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:57.895944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:05:57.895944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:57.895944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:05:57.895944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:57.895944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:05:57.895944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:57.895944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:05:57.895944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:05:57.895944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:57.895944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:05:57.895944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:57.897475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:05:57.897475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:05:57.897475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:57.897475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:05:57.897475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:57.897475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:05:58.355051 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:05:58.355940 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:05:58.356204 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:05:58.357429 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:05:58.357875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:58.358894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:05:58.358894 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:58.358894 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:05:58.358894 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:05:58.358894 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:58.358894 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:05:58.358894 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:58.358894 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:05:58.360671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:58.360671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:05:58.360671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:05:58.360671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:05:58.360671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:05:58.360671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:05:58.360671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:05:58.360671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:58.360671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:05:58.360671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:58.360671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:05:58.360671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:58.361159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:05:58.361159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:58.361159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:05:58.361159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:58.361159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:05:58.361159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:05:58.361159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:58.361159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:05:58.361159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:58.361159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:05:58.361159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:05:58.361159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:58.361159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:05:58.361505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:58.361505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:05:58.362412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:58.362412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:05:58.362412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:05:58.362412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:05:58.362412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:05:58.362412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:05:58.362412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:05:58.362412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:05:58.362412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:05:58.362412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:05:58.362412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:05:58.362412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:05:58.362412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:05:58.362412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:05:58.362412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:05:58.362412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:05:58.362890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:05:58.362890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:05:58.362890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:58.362890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:05:58.362890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:58.362890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:05:58.362890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:58.362890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:05:58.362890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:58.362890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:05:58.362890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:58.362890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:05:58.362890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:05:58.362890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:58.362890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:05:58.362890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:58.362890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:05:58.362890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:05:58.362890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:58.363437 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:05:58.363437 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:58.363437 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:05:58.367156 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:05:58.367550 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:05:58.367672 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:05:58.367822 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:05:58.368388 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:05:58.368705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:58.369561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:05:58.369561 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:58.369561 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:05:58.369561 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:05:58.369561 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:58.369561 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:05:58.369561 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:58.369561 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:05:58.370384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:58.370384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:05:58.370384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:05:58.370384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:05:58.370384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:58.370384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:05:58.370384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:58.370384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:05:58.370384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:58.370384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:05:58.370384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:58.370384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:05:58.370384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:58.370384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:05:58.370384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:05:58.370895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:58.370895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:05:58.370895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:58.370895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:05:58.370895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:05:58.370895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:58.370895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:05:58.370895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:58.370895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:05:58.371374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:58.371374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:05:58.371374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:05:58.371374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:05:58.371374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:05:58.371374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:05:58.371374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:05:58.371374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:05:58.371374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:05:58.371374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:05:58.371374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:58.371374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:05:58.372291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:58.372291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:05:58.372291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:58.372291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:05:58.372291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:58.372291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:05:58.372291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:58.372291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:05:58.372291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:05:58.372291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:58.372291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:05:58.372291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:58.372291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:05:58.372291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:05:58.373088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:58.373088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:05:58.373088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:58.373088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:05:58.373088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:58.373088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:05:58.373088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:05:58.373088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:05:58.373088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:05:58.373088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:05:58.373088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:05:58.373088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:05:58.373088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:05:58.373088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:05:58.373709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:05:58.373709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:05:58.373709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:05:58.373709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:05:58.373709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:05:58.373709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:05:58.373709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:05:58.373709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:05:58.373709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:05:58.373709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:05:58.373709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:05:58.373709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:58.373709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:05:58.373709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:58.373709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:05:58.373709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:58.373709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:05:58.373709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:58.373709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:05:58.374355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:58.374355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:05:58.374355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:05:58.374355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:58.374355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:05:58.374355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:58.374355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:05:58.374355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:05:58.374355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:58.374355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:05:58.374355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:58.374355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:05:58.392685 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:05:58.393258 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:05:58.393627 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:05:58.393901 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:05:58.394577 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:05:58.395102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:58.396026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:05:58.396026 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:58.396026 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:05:58.396026 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:05:58.396026 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:58.396026 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:05:58.396026 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:58.396026 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:05:58.397505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:58.397505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:05:58.397505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:05:58.397505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:05:58.397505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:58.397505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:05:58.397505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:58.397505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:05:58.397505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:58.397505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:05:58.397505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:58.397505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:05:58.397505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:58.397505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:05:58.397505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:05:58.398935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:58.398935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:05:58.398935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:58.398935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:05:58.398935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:05:58.398935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:58.398935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:05:58.398935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:58.398935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:05:58.398935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:58.398935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:05:58.398935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:05:58.398935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:05:58.398935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:05:58.399629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:05:58.399629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:05:58.399629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:05:58.399629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:05:58.399629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:58.399629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:05:58.399629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:58.399629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:05:58.399629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:58.399629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:05:58.399629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:58.399629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:05:58.399629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:58.400198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:05:58.400198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:05:58.400198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:58.400198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:05:58.400198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:58.400198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:05:58.400198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:05:58.400198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:58.400198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:05:58.400198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:58.400198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:05:58.400687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:58.400687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:05:58.400687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:05:58.400687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:05:58.400687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:05:58.400687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:05:58.400687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:05:58.400687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:05:58.400687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:05:58.400687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:05:58.400687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:05:58.400687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:05:58.400687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:05:58.400687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:05:58.400687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:05:58.400687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:05:58.400687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:05:58.401472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:05:58.401472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:05:58.401472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:05:58.401472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:58.401472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:05:58.401472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:58.401472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:05:58.401472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:58.401472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:05:58.401472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:58.401472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:05:58.401472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:58.401472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:05:58.401472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:05:58.401472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:58.401472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:05:58.401472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:58.401472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:05:58.401472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:05:58.402271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:58.402271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:05:58.402271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:58.402271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:05:58.402271 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:05:58.402271 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:05:58.402583 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:05:58.402583 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:05:58.403213 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:05:58.403660 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:05:58.403916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:58.404676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:05:58.404676 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:58.404676 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:05:58.404676 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:05:58.404676 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:58.404676 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:05:58.404676 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:58.404676 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:05:58.405546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:58.405546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:05:58.405546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:05:58.405546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:05:58.405546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:05:58.405546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:05:58.405546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:05:58.405546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:58.405546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:05:58.405546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:58.405546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:05:58.405546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:58.405943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:05:58.405943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:58.405943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:05:58.405943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:58.405943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:05:58.405943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:05:58.405943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:58.405943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:05:58.405943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:58.405943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:05:58.405943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:05:58.405943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:58.405943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:05:58.406341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:58.406341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:05:58.406523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:58.406523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:05:58.406523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:05:58.406523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:05:58.406523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:05:58.406523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:05:58.406523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:05:58.406523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:05:58.406523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:05:58.406523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:05:58.406523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:05:58.406523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:05:58.406523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:05:58.406523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:05:58.406523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:05:58.406523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:05:58.406986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:05:58.406986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:05:58.406986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:58.406986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:05:58.406986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:05:58.406986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:05:58.406986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:05:58.406986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:05:58.406986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:05:58.406986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:05:58.406986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:05:58.406986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:05:58.406986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:05:58.406986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:05:58.406986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:05:58.406986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:05:58.406986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:05:58.406986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:05:58.406986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:05:58.407646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:05:58.407646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:05:58.407646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:06:02.836694 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:06:02.837447 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:06:02.837634 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:06:02.839091 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:06:02.876375 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:06:02.876579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:02.877190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:06:02.877190 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:02.877190 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:06:02.877190 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:06:02.877190 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:02.877190 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:06:02.877190 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:02.877190 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:06:02.878359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:02.878359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:06:02.878359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:06:02.878359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:06:02.878359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:06:02.878359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:06:02.878359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:06:02.878359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:02.878359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:06:02.878359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:02.878359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:06:02.878359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:02.878999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:06:02.878999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:02.878999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:06:02.878999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:02.878999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:06:02.878999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:06:02.878999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:02.878999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:06:02.878999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:02.878999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:06:02.878999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:06:02.878999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:02.878999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:06:02.879637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:02.879637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:06:02.879637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:02.879637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:06:02.879637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:06:02.879637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:06:02.879637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:06:02.879637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:06:02.879637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:06:02.879637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:06:02.879637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:06:02.879637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:06:02.879637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:06:02.879637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:06:02.879637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:06:02.880436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:06:02.880436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:06:02.880436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:06:02.880436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:06:02.880436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:06:02.880436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:02.880436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:06:02.880436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:02.880436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:06:02.880436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:02.880436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:06:02.880436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:02.880436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:06:02.880436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:02.880436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:06:02.880436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:06:02.880436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:02.880436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:06:02.880436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:02.881322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:06:02.881322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:06:02.881322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:02.881322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:06:02.881322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:02.881322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:06:03.355697 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:06:03.356283 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:06:03.356452 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:06:03.357506 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:06:03.357934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:03.358758 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:06:03.358758 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:03.358758 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:06:03.358758 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:06:03.358758 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:03.358758 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:06:03.358758 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:03.358758 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:06:03.360175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:03.360175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:06:03.360175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:06:03.360175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:06:03.360175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:06:03.360175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:06:03.360175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:06:03.360175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:03.360175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:06:03.360175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:03.360175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:06:03.360175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:03.360480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:06:03.360480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:03.360480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:06:03.360480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:03.360480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:06:03.360480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:06:03.360480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:03.360480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:06:03.360480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:03.360480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:06:03.360480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:06:03.360480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:03.360480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:06:03.360734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:03.360734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:06:03.361340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:03.361340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:06:03.361340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:06:03.361340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:06:03.361340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:06:03.361340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:06:03.361340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:06:03.361340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:06:03.361340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:06:03.361340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:06:03.361340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:06:03.361340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:06:03.361340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:06:03.361340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:06:03.361340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:06:03.361340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:06:03.361760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:06:03.361760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:06:03.361760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:03.361760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:06:03.361760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:03.361760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:06:03.361760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:03.361760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:06:03.361760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:03.361760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:06:03.361760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:03.361760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:06:03.361760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:06:03.361760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:03.361760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:06:03.361760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:03.361760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:06:03.361760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:06:03.361760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:03.362184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:06:03.362184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:03.362184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:06:03.368391 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:06:03.369133 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:06:03.369271 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:06:03.369652 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:06:03.369980 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:06:03.370183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:03.370836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:06:03.370836 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:03.370836 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:06:03.370836 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:06:03.370836 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:03.370836 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:06:03.370836 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:03.370836 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:06:03.371901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:03.371901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:06:03.371901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:06:03.371901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:06:03.371901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:03.371901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:06:03.371901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:03.371901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:06:03.371901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:03.371901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:06:03.371901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:03.371901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:06:03.371901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:03.371901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:06:03.371901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:06:03.372802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:03.372802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:06:03.372802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:03.372802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:06:03.372802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:06:03.372802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:03.372802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:06:03.372802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:03.372802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:06:03.372802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:03.372802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:06:03.372802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:06:03.372802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:06:03.372802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:06:03.373564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:06:03.373564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:06:03.373564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:06:03.373564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:06:03.373564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:06:03.373564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:03.373564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:06:03.373564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:03.373564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:06:03.373564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:03.373564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:06:03.373564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:03.373564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:06:03.374654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:03.374654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:06:03.374654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:06:03.374654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:03.374654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:06:03.374654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:03.374654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:06:03.374654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:06:03.374654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:03.374654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:06:03.374654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:03.374654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:06:03.375085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:03.375085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:06:03.375085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:06:03.375085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:06:03.375085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:06:03.375085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:06:03.375085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:06:03.375085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:06:03.375085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:06:03.375085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:06:03.375085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:06:03.375085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:06:03.375085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:06:03.375085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:06:03.375085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:06:03.375085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:06:03.375552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:06:03.375552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:06:03.375552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:06:03.375552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:06:03.375552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:06:03.375552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:03.375552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:06:03.375552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:03.375552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:06:03.375552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:03.375552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:06:03.375552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:03.375552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:06:03.375552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:03.375552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:06:03.375552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:06:03.375552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:03.375552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:06:03.376228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:03.376228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:06:03.376228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:06:03.376228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:03.376228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:06:03.376228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:03.376228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:06:03.393310 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:06:03.393839 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:06:03.393962 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:06:03.394124 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:06:03.395111 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:06:03.395354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:03.396963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:06:03.396963 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:03.396963 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:06:03.396963 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:06:03.396963 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:03.396963 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:06:03.396963 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:03.396963 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:06:03.397912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:03.397912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:06:03.397912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:06:03.397912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:06:03.397912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:03.397912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:06:03.397912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:03.397912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:06:03.397912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:03.397912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:06:03.397912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:03.397912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:06:03.397912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:03.397912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:06:03.397912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:06:03.399037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:03.399037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:06:03.399037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:03.399037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:06:03.399037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:06:03.399037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:03.399037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:06:03.399037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:03.399037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:06:03.399463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:03.399463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:06:03.399463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:06:03.399463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:06:03.399463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:06:03.399463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:06:03.399463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:06:03.399463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:06:03.399463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:06:03.399463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:03.399463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:06:03.400117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:03.400117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:06:03.400117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:03.400117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:06:03.400117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:03.400117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:06:03.400117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:03.400117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:06:03.400117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:06:03.400117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:03.400117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:06:03.400117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:03.400117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:06:03.400117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:06:03.400979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:03.400979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:06:03.400979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:03.400979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:06:03.400979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:03.400979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:06:03.400979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:06:03.400979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:06:03.400979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:06:03.400979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:06:03.400979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:06:03.400979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:06:03.400979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:06:03.400979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:06:03.402029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:06:03.402029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:06:03.402029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:06:03.402029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:06:03.402029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:06:03.402029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:06:03.402029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:06:03.402029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:06:03.402029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:06:03.402029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:06:03.402029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:03.402029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:06:03.402029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:03.402029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:06:03.402029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:03.402029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:06:03.402029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:03.402029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:06:03.402029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:03.403036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:06:03.403036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:06:03.403036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:03.403036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:06:03.403036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:03.403036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:06:03.403036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:06:03.403036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:03.403036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:06:03.403036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:03.403036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:06:03.403036 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:06:03.403036 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:06:03.403631 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:06:03.403631 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:06:03.403631 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:06:03.403810 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:06:03.404008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:03.404791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:06:03.404791 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:03.404791 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:06:03.404791 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:06:03.404791 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:03.404791 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:06:03.404791 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:03.404791 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:06:03.405779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:03.405779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:06:03.405779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:06:03.405779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:06:03.405779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:06:03.405779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:06:03.405779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:06:03.405779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:03.405779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:06:03.405779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:03.405779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:06:03.405779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:03.406568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:06:03.406568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:03.406568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:06:03.406568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:03.406568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:06:03.406568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:06:03.406568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:03.406568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:06:03.406568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:03.406568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:06:03.406568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:06:03.406568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:03.406568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:06:03.407139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:03.407139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:06:03.407139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:03.407139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:06:03.407139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:06:03.407139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:06:03.407139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:06:03.407139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:06:03.407139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:06:03.407139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:06:03.407139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:06:03.407139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:06:03.407139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:06:03.407139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:06:03.407139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:06:03.407782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:06:03.407782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:06:03.407782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:06:03.407782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:06:03.407782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:06:03.407782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:03.407782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:06:03.407782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:03.407782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:06:03.407782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:03.407782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:06:03.407782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:03.407782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:06:03.407782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:03.407782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:06:03.407782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:06:03.407782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:03.407782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:06:03.407782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:03.408549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:06:03.408549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:06:03.408549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:03.408549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:06:03.408549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:03.408549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:06:07.837550 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:06:07.838666 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:06:07.838899 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:06:07.840443 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:06:07.877235 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:06:07.877695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:07.879207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:06:07.879207 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:07.879207 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:06:07.879207 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:06:07.879207 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:07.879207 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:06:07.879207 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:07.879207 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:06:07.881227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:07.881227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:06:07.881227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:06:07.881227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:06:07.881227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:06:07.881227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:06:07.881227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:06:07.881227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:07.881227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:06:07.881227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:07.881227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:06:07.881227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:07.882302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:06:07.882302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:07.882302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:06:07.882302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:07.882302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:06:07.882302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:06:07.882302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:07.882302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:06:07.882302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:07.882302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:06:07.882302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:06:07.882302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:07.882302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:06:07.883157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:07.883157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:06:07.883840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:07.883840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:06:07.883840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:06:07.883840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:06:07.883840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:06:07.883840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:06:07.883840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:06:07.883840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:06:07.883840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:06:07.883840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:06:07.883840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:06:07.883840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:06:07.883840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:06:07.883840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:06:07.883840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:06:07.883840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:06:07.884687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:06:07.884687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:06:07.884687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:07.884687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:06:07.884687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:07.884687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:06:07.884687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:07.884687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:06:07.884687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:07.884687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:06:07.884687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:07.884687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:06:07.884687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:06:07.884687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:07.884687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:06:07.884687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:07.884687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:06:07.884687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:06:07.884687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:07.885455 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:06:07.885455 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:07.885455 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:06:08.356499 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:06:08.357166 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:06:08.357385 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:06:08.358469 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:06:08.358941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:08.359984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:06:08.359984 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:08.359984 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:06:08.359984 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:06:08.359984 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:08.359984 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:06:08.359984 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:08.359984 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:06:08.361635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:08.361635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:06:08.361635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:06:08.361635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:06:08.361635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:06:08.361635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:06:08.361635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:06:08.361635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:08.361635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:06:08.361635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:08.361635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:06:08.361635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:08.362520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:06:08.362520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:08.362520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:06:08.362520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:08.362520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:06:08.362520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:06:08.362520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:08.362520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:06:08.362520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:08.362520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:06:08.362520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:06:08.362520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:08.362520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:06:08.363097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:08.363097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:06:08.363561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:08.363561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:06:08.363561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:06:08.363561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:06:08.363561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:06:08.363561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:06:08.363561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:06:08.363561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:06:08.363561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:06:08.363561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:06:08.363561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:06:08.363561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:06:08.363561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:06:08.363561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:06:08.363561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:06:08.363561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:06:08.365346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:06:08.365346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:06:08.365346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:08.365346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:06:08.365346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:08.365346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:06:08.365346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:08.365346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:06:08.365346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:08.365346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:06:08.365346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:08.365346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:06:08.365346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:06:08.365346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:08.365346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:06:08.365346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:08.365346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:06:08.365346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:06:08.365346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:08.366392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:06:08.366392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:08.366392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:06:08.369195 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:06:08.369626 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:06:08.369864 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:06:08.369999 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:06:08.370375 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:06:08.370618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:08.371180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:06:08.371180 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:08.371180 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:06:08.371180 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:06:08.371180 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:08.371180 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:06:08.371180 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:08.371180 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:06:08.371941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:08.371941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:06:08.371941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:06:08.371941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:06:08.371941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:08.371941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:06:08.371941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:08.371941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:06:08.371941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:08.371941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:06:08.371941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:08.371941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:06:08.371941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:08.371941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:06:08.371941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:06:08.372370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:08.372370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:06:08.372370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:08.372370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:06:08.372370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:06:08.372370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:08.372370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:06:08.372370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:08.372370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:06:08.372801 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:08.372801 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:06:08.372801 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:06:08.372801 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:06:08.372801 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:06:08.372801 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:06:08.372801 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:06:08.372801 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:06:08.372801 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:06:08.372801 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:06:08.372801 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:08.372801 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:06:08.373564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:08.373564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:06:08.373564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:08.373564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:06:08.373564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:08.373564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:06:08.373564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:08.373564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:06:08.373564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:06:08.373564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:08.373564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:06:08.373564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:08.373564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:06:08.373564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:06:08.374399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:08.374399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:06:08.374399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:08.374399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:06:08.374399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:08.374399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:06:08.374399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:06:08.374399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:06:08.374399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:06:08.374399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:06:08.374399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:06:08.374399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:06:08.374399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:06:08.374399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:06:08.374951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:06:08.374951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:06:08.374951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:06:08.374951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:06:08.374951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:06:08.374951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:06:08.374951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:06:08.374951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:06:08.374951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:06:08.374951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:06:08.374951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:06:08.374951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:08.374951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:06:08.374951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:08.374951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:06:08.374951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:08.374951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:06:08.374951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:08.374951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:06:08.375406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:08.375406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:06:08.375406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:06:08.375406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:08.375406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:06:08.375406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:08.375406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:06:08.375406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:06:08.375406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:08.375406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:06:08.375406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:08.375406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:06:08.394196 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:06:08.394739 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:06:08.395027 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:06:08.395217 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:06:08.395792 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:06:08.396348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:08.397288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:06:08.397288 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:08.397288 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:06:08.397288 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:06:08.397288 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:08.397288 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:06:08.397288 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:08.397288 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:06:08.398540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:08.398540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:06:08.398540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:06:08.398540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:06:08.398540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:08.398540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:06:08.398540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:08.398540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:06:08.398540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:08.398540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:06:08.398540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:08.398540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:06:08.398540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:08.398540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:06:08.398540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:06:08.399287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:08.399287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:06:08.399287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:08.399287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:06:08.399287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:06:08.399287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:08.399287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:06:08.399287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:08.399287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:06:08.399985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:08.399985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:06:08.399985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:06:08.399985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:06:08.399985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:06:08.399985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:06:08.399985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:06:08.399985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:06:08.399985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:06:08.399985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:08.399985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:06:08.400536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:08.400536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:06:08.400536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:08.400536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:06:08.400536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:08.400536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:06:08.400536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:08.400536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:06:08.400536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:06:08.400536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:08.400536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:06:08.400536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:08.400536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:06:08.400536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:06:08.401231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:08.401231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:06:08.401231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:08.401231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:06:08.401895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:08.401895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:06:08.401895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:06:08.401895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:06:08.401895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:06:08.401895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:06:08.401895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:06:08.401895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:06:08.401895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:06:08.401895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:06:08.401895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:06:08.401895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:06:08.401895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:06:08.401895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:06:08.401895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:06:08.401895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:06:08.401895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:06:08.403186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:06:08.403186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:06:08.403186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:06:08.403186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:08.403186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:06:08.403186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:08.403186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:06:08.403186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:08.403186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:06:08.403186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:08.403186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:06:08.403186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:08.403186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:06:08.403186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:06:08.403186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:08.403186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:06:08.403186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:08.403186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:06:08.403186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:06:08.404431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:08.404431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:06:08.404431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:08.404431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:06:08.404431 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:06:08.404431 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:06:08.404431 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:06:08.404431 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:06:08.405000 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:06:08.405000 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:06:08.405000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:08.405593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:06:08.405593 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:08.405593 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:06:08.405593 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:06:08.405593 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:08.405593 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:06:08.405593 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:08.405593 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:06:08.406561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:08.406561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:06:08.406561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:06:08.406561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:06:08.406561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:06:08.406561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:06:08.406561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:06:08.406561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:08.406561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:06:08.406561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:08.406561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:06:08.406561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:08.406950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:06:08.406950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:08.406950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:06:08.406950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:08.406950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:06:08.406950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:06:08.406950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:08.406950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:06:08.406950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:08.406950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:06:08.406950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:06:08.406950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:08.406950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:06:08.407282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:08.407282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:06:08.407562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:08.407562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:06:08.407562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:06:08.407562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:06:08.407562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:06:08.407562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:06:08.407562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:06:08.407562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:06:08.407562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:06:08.407562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:06:08.407562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:06:08.407562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:06:08.407562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:06:08.407562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:06:08.407562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:06:08.407562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:06:08.408035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:06:08.408035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:06:08.408035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:08.408035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:06:08.408035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:08.408035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:06:08.408035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:08.408035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:06:08.408035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:08.408035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:06:08.408035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:08.408035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:06:08.408035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:06:08.408035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:08.408035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:06:08.408035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:08.408035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:06:08.408035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:06:08.408035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:08.408550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:06:08.408550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:08.408550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:06:12.838520 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:06:12.839131 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:06:12.839274 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:06:12.840146 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:06:12.873848 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:06:12.874390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:12.875417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:06:12.875417 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:12.875417 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:06:12.875417 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:06:12.875417 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:12.875417 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:06:12.875417 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:12.875417 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:06:12.876479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:12.876479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:06:12.876479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:06:12.876479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:06:12.876479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:06:12.876479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:06:12.876479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:06:12.876479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:12.876479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:06:12.876479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:12.876479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:06:12.876479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:12.877120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:06:12.877120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:12.877120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:06:12.877120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:12.877120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:06:12.877120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:06:12.877120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:12.877120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:06:12.877120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:12.877120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:06:12.877120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:06:12.877120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:12.877120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:06:12.878241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:12.878241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:06:12.878241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:12.878241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:06:12.878241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:06:12.878241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:06:12.878241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:06:12.878241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:06:12.878241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:06:12.878241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:06:12.878241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:06:12.878241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:06:12.878241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:06:12.878241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:06:12.878241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:06:12.879315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:06:12.879315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:06:12.879315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:06:12.879315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:06:12.879315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:06:12.879315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:12.879315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:06:12.879315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:12.879315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:06:12.879315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:12.879315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:06:12.879315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:12.879315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:06:12.879315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:12.879315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:06:12.879315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:06:12.879315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:12.879315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:06:12.879315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:12.879934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:06:12.879934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:06:12.879934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:12.879934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:06:12.879934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:12.879934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:06:13.357199 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:06:13.357526 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:06:13.357623 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:06:13.358784 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:06:13.359262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:13.361280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:06:13.361280 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:13.361280 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:06:13.361280 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:06:13.361280 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:13.361280 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:06:13.361280 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:13.361280 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:06:13.363243 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:13.363243 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:06:13.363243 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:06:13.363243 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:06:13.363243 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:06:13.363243 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:06:13.363243 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:06:13.363243 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:13.363243 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:06:13.363243 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:13.363243 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:06:13.363243 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:13.364130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:06:13.364130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:13.364130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:06:13.364130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:13.364130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:06:13.364130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:06:13.364130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:13.364130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:06:13.364130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:13.364130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:06:13.364130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:06:13.364130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:13.364130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:06:13.364640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:13.364640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:06:13.365019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:13.365019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:06:13.365019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:06:13.365019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:06:13.365019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:06:13.365019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:06:13.365019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:06:13.365019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:06:13.365019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:06:13.365019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:06:13.365019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:06:13.365019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:06:13.365019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:06:13.365019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:06:13.365019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:06:13.365019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:06:13.365490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:06:13.365490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:06:13.365490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:13.365490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:06:13.365490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:13.365490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:06:13.365490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:13.365490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:06:13.365490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:13.365490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:06:13.365490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:13.365490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:06:13.365490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:06:13.365490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:13.365490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:06:13.365490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:13.365490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:06:13.365490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:06:13.365490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:13.366076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:06:13.366076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:13.366076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:06:13.369733 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:06:13.370043 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:06:13.370181 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:06:13.370371 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:06:13.371924 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:06:13.372195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:13.373083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:06:13.373083 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:13.373083 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:06:13.373083 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:06:13.373083 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:13.373083 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:06:13.373083 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:13.373083 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:06:13.374050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:13.374050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:06:13.374050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:06:13.374050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:06:13.374050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:13.374050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:06:13.374050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:13.374050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:06:13.374050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:13.374050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:06:13.374050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:13.374050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:06:13.374050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:13.374050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:06:13.374050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:06:13.374879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:13.374879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:06:13.374879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:13.374879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:06:13.374879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:06:13.374879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:13.374879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:06:13.374879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:13.374879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:06:13.375300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:13.375300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:06:13.375300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:06:13.375300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:06:13.375300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:06:13.375300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:06:13.375300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:06:13.375300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:06:13.375300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:06:13.375300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:06:13.375300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:13.375300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:06:13.376039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:13.376039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:06:13.376039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:13.376039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:06:13.376039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:13.376039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:06:13.376039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:13.376039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:06:13.376039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:06:13.376039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:13.376039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:06:13.376039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:13.376039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:06:13.376039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:06:13.376897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:13.376897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:06:13.376897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:13.376897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:06:13.376897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:13.376897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:06:13.376897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:06:13.376897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:06:13.376897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:06:13.376897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:06:13.376897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:06:13.376897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:06:13.376897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:06:13.376897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:06:13.377570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:06:13.377570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:06:13.377570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:06:13.377570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:06:13.377570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:06:13.377570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:06:13.377570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:06:13.377570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:06:13.377570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:06:13.377570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:06:13.377570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:06:13.377570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:13.377570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:06:13.377570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:13.377570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:06:13.377570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:13.377570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:06:13.377570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:13.377570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:06:13.378774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:13.378774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:06:13.378774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:06:13.378774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:13.378774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:06:13.378774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:13.378774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:06:13.378774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:06:13.378774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:13.378774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:06:13.378774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:13.378774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:06:13.394715 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:06:13.395152 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:06:13.395352 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:06:13.395553 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:06:13.396365 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:06:13.396867 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:13.398031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:06:13.398031 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:13.398031 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:06:13.398031 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:06:13.398031 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:13.398031 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:06:13.398031 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:13.398031 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:06:13.399340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:13.399340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:06:13.399340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:06:13.399340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:06:13.399340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:13.399340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:06:13.399340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:13.399340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:06:13.399340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:13.399340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:06:13.399340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:13.399340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:06:13.399340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:13.399340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:06:13.399340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:06:13.400062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:13.400062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:06:13.400062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:13.400062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:06:13.400062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:06:13.400062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:13.400062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:06:13.400062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:13.400062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:06:13.400880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:13.400880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:06:13.400880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:06:13.400880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:06:13.400880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:06:13.400880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:06:13.400880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:06:13.400880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:06:13.400880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:06:13.400880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:13.400880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:06:13.401216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:13.401216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:06:13.401216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:13.401216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:06:13.401216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:13.401216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:06:13.401216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:13.401216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:06:13.401216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:06:13.401216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:13.401216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:06:13.401216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:13.401216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:06:13.401216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:06:13.401567 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:13.401567 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:06:13.401567 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:13.401567 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:06:13.403077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:13.403077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:06:13.403077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:06:13.403077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:06:13.403077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:06:13.403077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:06:13.403077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:06:13.403077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:06:13.403077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:06:13.403077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:06:13.403077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:06:13.403077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:06:13.403077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:06:13.403077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:06:13.403077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:06:13.403077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:06:13.403077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:06:13.403584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:06:13.403584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:06:13.403584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:06:13.403584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:13.403584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:06:13.403584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:13.403584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:06:13.403584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:13.403584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:06:13.403584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:13.403584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:06:13.403584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:13.403584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:06:13.403584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:06:13.403584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:13.403584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:06:13.403584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:13.403584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:06:13.403584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:06:13.404081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:13.404081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:06:13.404081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:13.404081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:06:13.404081 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:06:13.404081 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:06:13.404081 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:06:13.404303 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:06:13.404810 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:06:13.405292 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:06:13.405547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:13.406363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:06:13.406363 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:13.406363 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:06:13.406363 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:06:13.406363 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:13.406363 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:06:13.406363 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:13.406363 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:06:13.407311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:13.407311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:06:13.407311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:06:13.407311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:06:13.407311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:06:13.407311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:06:13.407311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:06:13.407311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:13.407311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:06:13.407311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:13.407311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:06:13.407311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:13.407672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:06:13.407672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:13.407672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:06:13.407672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:13.407672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:06:13.407672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:06:13.407672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:13.407672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:06:13.407672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:13.407672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:06:13.407672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:06:13.407672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:13.407672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:06:13.408037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:13.408037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:06:13.408343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:13.408343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:06:13.408343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:06:13.408343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:06:13.408343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:06:13.408343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:06:13.408343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:06:13.408343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:06:13.408343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:06:13.408343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:06:13.408343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:06:13.408343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:06:13.408343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:06:13.408343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:06:13.408343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:06:13.408343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:06:13.408803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:06:13.408803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:06:13.408803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:13.408803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:06:13.408803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:13.408803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:06:13.408803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:13.408803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:06:13.408803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:13.408803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:06:13.408803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:13.408803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:06:13.408803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:06:13.408803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:13.408803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:06:13.408803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:13.408803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:06:13.408803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:06:13.408803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:13.409312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:06:13.409312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:13.409312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:06:17.839265 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:06:17.840145 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:06:17.840145 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:06:17.841209 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:06:17.874833 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:06:17.875287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:17.876367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:06:17.876367 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:17.876367 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:06:17.876367 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:06:17.876367 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:17.876367 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:06:17.876367 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:17.876367 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:06:17.877258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:17.877258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:06:17.877258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:06:17.877258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:06:17.877258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:06:17.877258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:06:17.877258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:06:17.877258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:17.877258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:06:17.877258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:17.877258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:06:17.877258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:17.877863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:06:17.877863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:17.877863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:06:17.877863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:17.877863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:06:17.877863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:06:17.877863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:17.877863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:06:17.877863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:17.877863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:06:17.877863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:06:17.877863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:17.877863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:06:17.878404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:17.878404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:06:17.878404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:17.878404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:06:17.878404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:06:17.878404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:06:17.878404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:06:17.878404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:06:17.878404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:06:17.878404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:06:17.878404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:06:17.878404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:06:17.878404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:06:17.878404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:06:17.878404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:06:17.879095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:06:17.879095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:06:17.879095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:06:17.879095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:06:17.879095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:06:17.879095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:17.879095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:06:17.879095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:17.879095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:06:17.879095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:17.879095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:06:17.879095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:17.879095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:06:17.879095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:17.879095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:06:17.879095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:06:17.879095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:17.879095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:06:17.879095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:17.879927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:06:17.879927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:06:17.879927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:17.879927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:06:17.879927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:17.879927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:06:18.358158 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:06:18.358568 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:06:18.358683 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:06:18.360008 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:06:18.360424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:18.361406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:06:18.361406 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:18.361406 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:06:18.361406 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:06:18.361406 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:18.361406 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:06:18.361406 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:18.361406 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:06:18.362701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:18.362701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:06:18.362701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:06:18.362701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:06:18.362701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:06:18.362701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:06:18.362701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:06:18.362701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:18.362701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:06:18.362701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:18.362701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:06:18.362701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:18.363510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:06:18.363510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:18.363510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:06:18.363510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:18.363510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:06:18.363510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:06:18.363510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:18.363510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:06:18.363510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:18.363510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:06:18.363510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:06:18.363510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:18.363510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:06:18.364369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:18.364369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:06:18.364369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:18.364369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:06:18.364369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:06:18.364369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:06:18.364369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:06:18.364369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:06:18.364369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:06:18.364369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:06:18.364369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:06:18.364369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:06:18.364369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:06:18.364369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:06:18.364369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:06:18.365306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:06:18.365306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:06:18.365306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:06:18.365306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:06:18.365306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:06:18.365306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:18.365306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:06:18.365306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:18.365306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:06:18.365306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:18.365306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:06:18.365306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:18.365306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:06:18.365306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:18.365306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:06:18.365306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:06:18.365306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:18.365306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:06:18.365306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:18.366397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:06:18.366397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:06:18.366397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:18.366397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:06:18.366397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:18.366397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:06:18.370320 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:06:18.370906 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:06:18.371116 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:06:18.371251 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:06:18.371994 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:06:18.372338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:18.373219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:06:18.373219 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:18.373219 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:06:18.373219 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:06:18.373219 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:18.373219 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:06:18.373219 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:18.373219 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:06:18.374442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:18.374442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:06:18.374442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:06:18.374442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:06:18.374442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:18.374442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:06:18.374442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:18.374442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:06:18.374442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:18.374442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:06:18.374442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:18.374442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:06:18.374442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:18.374442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:06:18.374442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:06:18.374830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:18.374830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:06:18.374830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:18.374830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:06:18.374830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:06:18.374830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:18.374830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:06:18.374830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:18.374830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:06:18.375948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:18.375948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:06:18.375948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:06:18.375948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:06:18.375948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:06:18.375948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:06:18.375948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:06:18.375948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:06:18.375948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:06:18.375948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:06:18.375948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:18.375948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:06:18.376290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:18.376290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:06:18.376290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:18.376290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:06:18.376290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:18.376290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:06:18.376290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:18.376290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:06:18.376290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:06:18.376290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:18.376290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:06:18.376290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:18.376290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:06:18.376290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:06:18.376664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:18.376664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:06:18.376664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:18.376664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:06:18.377628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:18.377628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:06:18.377628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:06:18.377628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:06:18.377628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:06:18.377628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:06:18.377628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:06:18.377628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:06:18.377628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:06:18.377628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:06:18.377628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:06:18.377628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:06:18.377628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:06:18.377628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:06:18.377628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:06:18.377628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:06:18.378151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:06:18.378151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:06:18.378151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:06:18.378151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:06:18.378151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:06:18.378151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:18.378151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:06:18.378151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:18.378151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:06:18.378151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:18.378151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:06:18.378151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:18.378151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:06:18.378151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:18.378151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:06:18.378151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:06:18.378151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:18.378151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:06:18.378647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:18.378647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:06:18.378647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:06:18.378647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:18.378647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:06:18.378647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:18.378647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:06:18.395283 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:06:18.395902 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:06:18.396021 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:06:18.396180 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:06:18.396914 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:06:18.397375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:18.398438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:06:18.398438 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:18.398438 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:06:18.398438 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:06:18.398438 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:18.398438 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:06:18.398438 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:18.398438 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:06:18.399720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:18.399720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:06:18.399720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:06:18.399720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:06:18.399720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:18.399720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:06:18.399720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:18.399720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:06:18.399720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:18.399720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:06:18.399720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:18.399720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:06:18.399720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:18.399720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:06:18.399720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:06:18.400530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:18.400530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:06:18.400530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:18.400530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:06:18.400530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:06:18.400530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:18.400530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:06:18.400530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:18.400530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:06:18.401420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:18.401420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:06:18.401420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:06:18.401420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:06:18.401420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:06:18.401420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:06:18.401420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:06:18.401420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:06:18.401420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:06:18.401420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:18.401420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:06:18.402143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:18.402143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:06:18.402143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:18.402143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:06:18.402143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:18.402143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:06:18.402143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:18.402143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:06:18.402143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:06:18.402143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:18.402143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:06:18.402143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:18.402143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:06:18.402143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:06:18.403017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:18.403017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:06:18.403017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:18.403017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:06:18.403250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:18.403250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:06:18.403250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:06:18.403250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:06:18.403250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:06:18.403250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:06:18.403250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:06:18.403250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:06:18.403250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:06:18.403250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:06:18.403250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:06:18.403250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:06:18.403250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:06:18.403250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:06:18.403250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:06:18.403250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:06:18.403250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:06:18.404283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:06:18.404283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:06:18.404283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:06:18.404283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:18.404283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:06:18.404283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:18.404283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:06:18.404283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:18.404283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:06:18.404283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:18.404283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:06:18.404283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:18.404283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:06:18.404283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:06:18.404283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:18.404283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:06:18.404283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:18.404283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:06:18.404283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:06:18.405360 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:18.405360 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:06:18.405360 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:18.405360 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:06:18.405360 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:06:18.405360 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:06:18.405360 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:06:18.405360 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:06:18.406057 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:06:18.406057 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:06:18.406057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:18.406057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:06:18.406057 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:18.406057 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:06:18.406057 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:06:18.406057 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:18.406057 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:06:18.406057 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:18.406057 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:06:18.407157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:18.407157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:06:18.407157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:06:18.407157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:06:18.407157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:06:18.407157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:06:18.407157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:06:18.407157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:18.407157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:06:18.407157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:18.407157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:06:18.407157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:18.407655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:06:18.407655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:18.407655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:06:18.407655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:18.407655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:06:18.407655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:06:18.407655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:18.407655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:06:18.407655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:18.407655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:06:18.407655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:06:18.407655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:18.407655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:06:18.408183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:18.408183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:06:18.408550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:18.408550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:06:18.408550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:06:18.408550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:06:18.408550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:06:18.408550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:06:18.408550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:06:18.408550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:06:18.408550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:06:18.408550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:06:18.408550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:06:18.408550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:06:18.408550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:06:18.408550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:06:18.408550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:06:18.408550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:06:18.409538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:06:18.409538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:06:18.409538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:18.409538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:06:18.409538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:18.409538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:06:18.409538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:18.409538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:06:18.409538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:18.409538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:06:18.409538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:18.409538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:06:18.409538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:06:18.409538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:18.409538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:06:18.409538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:18.409538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:06:18.409538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:06:18.409538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:18.410282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:06:18.410282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:18.410282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:06:22.840040 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:06:22.841162 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:06:22.841162 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:06:22.842216 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:06:22.912640 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:06:22.913127 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:22.914073 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:06:22.914073 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:22.914073 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:06:22.914073 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:06:22.914073 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:22.914073 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:06:22.914073 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:22.914073 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:06:22.915174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.073 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:22.915174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:06:22.915174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:06:22.915174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:06:22.915174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:06:22.915174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:06:22.915174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:06:22.915174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:22.915174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:06:22.915174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:22.915174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:06:22.915174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:22.915944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:06:22.915944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:22.915944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:06:22.915944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:22.915944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:06:22.915944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:06:22.915944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:22.915944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:06:22.915944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:22.915944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:06:22.915944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:06:22.915944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:22.915944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:06:22.916564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:22.916564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:06:22.916564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:22.916564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:06:22.916564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:06:22.916564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:06:22.916564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:06:22.916564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:06:22.916564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:06:22.916564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:06:22.916564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:06:22.916564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:06:22.916564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:06:22.916564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:06:22.916564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:06:22.917298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:06:22.917298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:06:22.917298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:06:22.917298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:06:22.917298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:06:22.917298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:22.917298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:06:22.917298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:22.917298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:06:22.917298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:22.917298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:06:22.917298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:22.917298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:06:22.917298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:22.917298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:06:22.917298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:06:22.917298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:22.917298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:06:22.917298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:22.918240 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:06:22.918240 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:06:22.918240 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:22.918240 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:06:22.918240 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:22.918240 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:06:23.358858 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:06:23.359489 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:06:23.359684 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:06:23.360643 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:06:23.361092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:23.361990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:06:23.361990 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:23.361990 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:06:23.361990 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:06:23.361990 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:23.361990 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:06:23.361990 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:23.361990 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:06:23.367975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:23.367975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:06:23.367975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:06:23.367975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:06:23.367975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:06:23.367975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:06:23.367975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:06:23.367975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:23.367975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:06:23.367975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:23.367975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:06:23.367975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:23.368610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:06:23.368610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:23.368610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:06:23.368610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:23.368610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:06:23.368610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:06:23.368610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:23.368610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:06:23.368610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:23.368610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:06:23.368610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:06:23.368610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:23.368610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:06:23.369339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:23.369339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:06:23.369339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:23.369339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:06:23.369339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:06:23.369339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:06:23.369339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:06:23.369339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:06:23.369339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:06:23.369339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:06:23.369339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:06:23.369339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:06:23.369339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:06:23.369339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:06:23.369339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:06:23.370037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:06:23.370037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:06:23.370037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:06:23.370037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:06:23.370037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:06:23.370037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:23.370037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:06:23.370037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:23.370037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:06:23.370037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:23.370037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:06:23.370037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:23.370037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:06:23.370037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:23.370037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:06:23.370037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:06:23.370037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:23.370037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:06:23.370037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:23.370992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:06:23.370992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:06:23.370992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:23.370992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:06:23.370992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:23.370992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:06:23.370992 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:06:23.371329 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:06:23.371555 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:06:23.371841 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:06:23.372357 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:06:23.372780 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:23.373513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:06:23.373513 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:23.373513 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:06:23.373513 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:06:23.373513 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:23.373513 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:06:23.373513 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:23.373513 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:06:23.375003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:23.375003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:06:23.375003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:06:23.375003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:06:23.375003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:23.375003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:06:23.375003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:23.375003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:06:23.375003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:23.375003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:06:23.375003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:23.375003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:06:23.375003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:23.375003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:06:23.375003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:06:23.375427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:23.375427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:06:23.375427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:23.375427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:06:23.375427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:06:23.375427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:23.375427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:06:23.375427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:23.375427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:06:23.379784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:23.379784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:06:23.379784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:06:23.379784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:06:23.379784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:06:23.379784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:06:23.379784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:06:23.379784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:06:23.379784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:06:23.379784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:06:23.379784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:23.379784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:06:23.380139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:23.380139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:06:23.380139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:23.380139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:06:23.380139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:23.380139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:06:23.380139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:23.380139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:06:23.380139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:06:23.380139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:23.380139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:06:23.380139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:23.380139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:06:23.380139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:06:23.380528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:23.380528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:06:23.380528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:23.380528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:06:23.380528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:23.380528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:06:23.380528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:06:23.380528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:06:23.380528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:06:23.380528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:06:23.380528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:06:23.380528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:06:23.380528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:06:23.380528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:06:23.381083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:06:23.381083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:06:23.381083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:06:23.381083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:06:23.381083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:06:23.381083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:06:23.381083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:06:23.381083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:06:23.381083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:06:23.381083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:06:23.381083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:06:23.381083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:23.381083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:06:23.381083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:23.381083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:06:23.381083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:23.381083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:06:23.381083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:23.381083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:06:23.381709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:23.381709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:06:23.381709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:06:23.381709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:23.381709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:06:23.381709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:23.381709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:06:23.381709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:06:23.381709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:23.381709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:06:23.381709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:23.381709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:06:23.395847 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:06:23.396302 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:06:23.396492 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:06:23.396883 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:06:23.397423 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:06:23.397876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:23.398776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:06:23.398776 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:23.398776 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:06:23.398776 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:06:23.398776 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:23.398776 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:06:23.398776 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:23.398776 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:06:23.400107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:23.400107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:06:23.400107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:06:23.400107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:06:23.400107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:23.400107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:06:23.400107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:23.400107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:06:23.400107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:23.400107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:06:23.400107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:23.400107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:06:23.400107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:23.400107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:06:23.400107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:06:23.400815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:23.400815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:06:23.400815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:23.400815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:06:23.400815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:06:23.400815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:23.400815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:06:23.400815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:23.400815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:06:23.401673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:23.401673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:06:23.401673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:06:23.401673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:06:23.401673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:06:23.401673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:06:23.401673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:06:23.401673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:06:23.401673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:06:23.401673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:23.401673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:06:23.402255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:23.402255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:06:23.402255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:23.402255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:06:23.402255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:23.402255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:06:23.402255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:23.402255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:06:23.402255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:06:23.402255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:23.402255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:06:23.402255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:23.402255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:06:23.402255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:06:23.402635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:23.402635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:06:23.402635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:23.402635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:06:23.402773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:23.402773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:06:23.402773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:06:23.402773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:06:23.402773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:06:23.402773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:06:23.402773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:06:23.402773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:06:23.402773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:06:23.402773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:06:23.402773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:06:23.402773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:06:23.402773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:06:23.402773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:06:23.402773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:06:23.402773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:06:23.402773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:06:23.403270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:06:23.403270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:06:23.403270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:06:23.403270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:23.403270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:06:23.403270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:23.403270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:06:23.403270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:23.403270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:06:23.403270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:23.403270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:06:23.403270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:23.403270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:06:23.403270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:06:23.403270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:23.403270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:06:23.403270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:23.403270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:06:23.403270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:06:23.403764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:23.403764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:06:23.403764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:23.403764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:06:23.404565 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:06:23.404924 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:06:23.405040 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:06:23.405194 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:06:23.405650 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:06:23.406077 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:06:23.406275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:23.407030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:06:23.407030 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:23.407030 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:06:23.407030 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:06:23.407030 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:23.407030 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:06:23.407030 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:23.407030 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:06:23.407922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:23.407922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:06:23.407922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:06:23.407922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:06:23.407922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:06:23.407922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:06:23.407922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:06:23.407922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:23.407922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:06:23.407922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:23.407922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:06:23.407922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:23.411762 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:06:23.411762 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:23.411762 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:06:23.411762 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:23.411762 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:06:23.411762 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:06:23.411762 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:23.411762 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:06:23.411762 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:23.411762 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:06:23.411762 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:06:23.411762 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:23.411762 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:06:23.412178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:23.412178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:06:23.412178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:23.412178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:06:23.412178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:06:23.412178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:06:23.412178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:06:23.412178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:06:23.412178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:06:23.412178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:06:23.412178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:06:23.412178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:06:23.412178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:06:23.412178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:06:23.412178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:06:23.412569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:06:23.412569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:06:23.412569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:06:23.412569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:06:23.412569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:06:23.412569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:23.412569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:06:23.412569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:23.412569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:06:23.412569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:23.412569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:06:23.412569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:23.412569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:06:23.412569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:23.412569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:06:23.412569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:06:23.412569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:23.412569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:06:23.412569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:23.413109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:06:23.413109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:06:23.413109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:23.413109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:06:23.413109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:23.413109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:06:27.840735 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:06:27.841274 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:06:27.841274 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:06:27.842564 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:06:27.877182 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:06:27.877616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:27.878588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:06:27.878588 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:27.878588 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:06:27.878588 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:06:27.878588 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:27.878588 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:06:27.878588 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:27.878588 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:06:27.879362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:27.879362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:06:27.879362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:06:27.879362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:06:27.879362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:06:27.879362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:06:27.879362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:06:27.879362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:27.879362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:06:27.879362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:27.879362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:06:27.879362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:27.880081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:06:27.880081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:27.880081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:06:27.880081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:27.880081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:06:27.880081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:06:27.880081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:27.880081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:06:27.880081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:27.880081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:06:27.880081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:06:27.880081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:27.880081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:06:27.880693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:27.880693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:06:27.880693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:27.880693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:06:27.880693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:06:27.880693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:06:27.880693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:06:27.880693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:06:27.880693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:06:27.880693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:06:27.880693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:06:27.880693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:06:27.880693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:06:27.880693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:06:27.880693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:06:27.881465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:06:27.881465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:06:27.881465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:06:27.881465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:06:27.881465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:06:27.881465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:27.881465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:06:27.881465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:27.881465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:06:27.881465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:27.881465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:06:27.881465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:27.881465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:06:27.881465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:27.881465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:06:27.881465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:06:27.881465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:27.881465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:06:27.881465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:27.882326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:06:27.882326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:06:27.882326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:27.882326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:06:27.882326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:27.882326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:06:28.359550 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:06:28.360170 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:06:28.360437 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:06:28.361299 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:06:28.361588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:28.362931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:06:28.362931 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:28.362931 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:06:28.362931 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:06:28.362931 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:28.362931 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:06:28.362931 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:28.362931 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:06:28.364608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:28.364608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:06:28.364608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:06:28.364608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:06:28.364608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:06:28.364608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:06:28.364608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:06:28.364608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:28.364608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:06:28.364608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:28.364608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:06:28.364608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:28.365239 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:06:28.365239 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:28.365239 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:06:28.365239 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:28.365239 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:06:28.365239 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:06:28.365239 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:28.365239 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:06:28.365239 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:28.365239 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:06:28.365239 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:06:28.365239 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:28.365239 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:06:28.365848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:28.365848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:06:28.368022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:28.368022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:06:28.368022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:06:28.368022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:06:28.368022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:06:28.368022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:06:28.368022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:06:28.368022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:06:28.368022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:06:28.368022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:06:28.368022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:06:28.368022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:06:28.368022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:06:28.368022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:06:28.368022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:06:28.368022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:06:28.369022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:06:28.369022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:06:28.369022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:28.369022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:06:28.369022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:28.369022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:06:28.369022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:28.369022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:06:28.369022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:28.369022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:06:28.369022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:28.369022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:06:28.369022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:06:28.369022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:28.369022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:06:28.369022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:28.369022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:06:28.369022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:06:28.369022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:28.369932 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:06:28.369932 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:28.369932 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:06:28.371268 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:06:28.371732 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:06:28.371910 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:06:28.372041 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:06:28.372571 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:06:28.372773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:28.373434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:06:28.373434 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:28.373434 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:06:28.373434 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:06:28.373434 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:28.373434 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:06:28.373434 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:28.373434 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:06:28.374276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:28.374276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:06:28.374276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:06:28.374276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:06:28.374276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:28.374276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:06:28.374276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:28.374276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:06:28.374276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:28.374276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:06:28.374276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:28.374276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:06:28.374276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:28.374276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:06:28.374276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:06:28.375193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:28.375193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:06:28.375193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:28.375193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:06:28.375193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:06:28.375193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:28.375193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:06:28.375193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:28.375193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:06:28.375699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:28.375699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:06:28.375699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:06:28.375699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:06:28.375699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:06:28.375699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:06:28.375699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:06:28.375699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:06:28.375699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:06:28.375699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:06:28.375699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:28.375699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:06:28.376442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:28.376442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:06:28.376442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:28.376442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:06:28.376442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:28.376442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:06:28.376442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:28.376442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:06:28.376442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:06:28.376442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:28.376442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:06:28.376442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:28.376442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:06:28.376442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:06:28.377242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:28.377242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:06:28.377242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:28.377242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:06:28.377242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:28.377242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:06:28.377242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:06:28.377242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:06:28.377242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:06:28.377242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:06:28.377242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:06:28.377242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:06:28.377242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:06:28.377242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:06:28.378145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:06:28.378145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:06:28.378145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:06:28.378145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:06:28.378145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:06:28.378145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:06:28.378145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:06:28.378145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:06:28.378145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:06:28.378145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:06:28.378145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:06:28.378145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:28.378145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:06:28.378145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:28.378145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:06:28.378145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:28.378145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:06:28.378145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:28.378145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:06:28.379369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:28.379369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:06:28.379369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:06:28.379369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:28.379369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:06:28.379369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:28.379369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:06:28.379369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:06:28.379369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:28.379369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:06:28.379369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:28.379369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:06:28.396421 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:06:28.397043 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:06:28.397289 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:06:28.397519 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:06:28.398096 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:06:28.398370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:28.399226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:06:28.399226 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:28.399226 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:06:28.399226 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:06:28.399226 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:28.399226 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:06:28.399226 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:28.399226 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:06:28.400554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:28.400554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:06:28.400554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:06:28.400554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:06:28.400554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:28.400554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:06:28.400554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:28.400554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:06:28.400554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:28.400554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:06:28.400554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:28.400554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:06:28.400554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:28.400554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:06:28.400554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:06:28.401032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:28.401032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:06:28.401032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:28.401032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:06:28.401032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:06:28.401032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:28.401032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:06:28.401032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:28.401032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:06:28.402419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:28.402419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:06:28.402419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:06:28.402419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:06:28.402419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:06:28.402419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:06:28.402419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:06:28.402419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:06:28.402419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:06:28.402419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:28.402419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:06:28.402865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:28.402865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:06:28.402865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:28.402865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:06:28.402865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:28.402865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:06:28.402865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:28.402865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:06:28.402865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:06:28.402865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:28.402865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:06:28.402865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:28.402865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:06:28.402865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:06:28.403249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:28.403249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:06:28.403249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:28.403249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:06:28.404231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:28.404231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:06:28.404231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:06:28.404231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:06:28.404231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:06:28.404231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:06:28.404231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:06:28.404231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:06:28.404231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:06:28.404231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:06:28.404231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:06:28.404231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:06:28.404231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:06:28.404231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:06:28.404231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:06:28.404231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:06:28.404231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:06:28.404731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:06:28.404731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:06:28.404731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:06:28.404731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:28.404731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:06:28.404731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:28.404731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:06:28.404731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:28.404731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:06:28.404731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:28.404731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:06:28.404731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:28.404731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:06:28.404731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:06:28.404731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:28.404731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:06:28.404731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:28.404731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:06:28.404731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:06:28.405373 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:28.405373 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:06:28.405373 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:28.405373 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:06:28.405373 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:06:28.405522 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:06:28.405570 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:06:28.405772 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:06:28.406282 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:06:28.406454 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:06:28.406617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:28.407331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:06:28.407331 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:28.407331 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:06:28.407331 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:06:28.407331 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:28.407331 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:06:28.407331 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:28.407331 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:06:28.408359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:28.408359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:06:28.408359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:06:28.408359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:06:28.408359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:06:28.408359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:06:28.408359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:06:28.408359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:28.408359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:06:28.408359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:28.408359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:06:28.408359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:28.408708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:06:28.408708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:28.408708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:06:28.408708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:28.408708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:06:28.408708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:06:28.408708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:28.408708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:06:28.408708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:28.408708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:06:28.408708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:06:28.408708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:28.408708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:06:28.409088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:28.409088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:06:28.409476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:28.409476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:06:28.409476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:06:28.409476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:06:28.409476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:06:28.409476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:06:28.409476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:06:28.409476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:06:28.409476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:06:28.409476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:06:28.409476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:06:28.409476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:06:28.409476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:06:28.409476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:06:28.409476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:06:28.409476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:06:28.409988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:06:28.409988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:06:28.409988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:28.409988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:06:28.409988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:28.409988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:06:28.409988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:28.409988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:06:28.409988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:28.409988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:06:28.409988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:28.409988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:06:28.409988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:06:28.409988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:28.409988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:06:28.409988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:28.409988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:06:28.409988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:06:28.409988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:28.410464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:06:28.410464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:28.410464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:06:32.841366 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:06:32.842563 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:06:32.842563 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:06:32.843463 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:06:32.875153 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:06:32.875376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:32.875926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:06:32.875926 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:32.875926 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:06:32.875926 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:06:32.875926 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:32.875926 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:06:32.875926 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:32.875926 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:06:32.876711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:32.876711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:06:32.876711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:06:32.876711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:06:32.876711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:06:32.876711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:06:32.876711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:06:32.876711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:32.876711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:06:32.876711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:32.876711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:06:32.876711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:32.877391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:06:32.877391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:32.877391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:06:32.877391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:32.877391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:06:32.877391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:06:32.877391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:32.877391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:06:32.877391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:32.877391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:06:32.877391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:06:32.877391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:32.877391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:06:32.878016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:32.878016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:06:32.878016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:32.878016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:06:32.878016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:06:32.878016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:06:32.878016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:06:32.878016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:06:32.878016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:06:32.878016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:06:32.878016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:06:32.878016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:06:32.878016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:06:32.878016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:06:32.878016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:06:32.878719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:06:32.878719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:06:32.878719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:06:32.878719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:06:32.878719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:06:32.878719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:32.878719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:06:32.878719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:32.878719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:06:32.878719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:32.878719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:06:32.878719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:32.878719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:06:32.878719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:32.878719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:06:32.878719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:06:32.878719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:32.878719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:06:32.878719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:32.880191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:06:32.880191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:06:32.880191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:32.880191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:06:32.880191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:32.880191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:06:33.360238 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:06:33.360970 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:06:33.361169 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:06:33.362167 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:06:33.362526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:33.365607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:06:33.365607 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:33.365607 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:06:33.365607 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:06:33.365607 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:33.365607 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:06:33.365607 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:33.365607 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:06:33.365607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:33.365607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:06:33.365607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:06:33.365607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:06:33.365607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:06:33.365607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:06:33.365607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:06:33.366527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:33.366527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:06:33.366527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:33.366527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:06:33.366527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:33.366527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:06:33.366527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:33.366527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:06:33.366527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:33.366527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:06:33.366527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:06:33.366527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:33.366527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:06:33.366527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:33.367170 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:06:33.367170 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:06:33.367170 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:33.367170 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:06:33.367170 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:33.367170 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:06:33.367487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:33.367487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:06:33.367487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:06:33.367487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:06:33.367487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:06:33.367487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:06:33.367487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:06:33.367487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:06:33.367487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:06:33.367487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:06:33.367487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:06:33.367487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:06:33.367487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:06:33.367487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:06:33.367487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:06:33.367487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:06:33.368241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:06:33.368241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:06:33.368241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:33.368241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:06:33.368241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:33.368241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:06:33.368241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:33.368241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:06:33.368241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:33.368241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:06:33.368241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:33.368241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:06:33.368241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:06:33.368241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:33.368241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:06:33.368241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:33.368241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:06:33.368241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:06:33.368241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:33.369220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:06:33.369220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:33.369220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:06:33.371689 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:06:33.372068 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:06:33.372232 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:06:33.372417 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:06:33.372795 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:06:33.372999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:33.373465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:06:33.373465 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:33.373465 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:06:33.373465 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:06:33.373465 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:33.373465 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:06:33.373465 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:33.373465 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:06:33.374214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:33.374214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:06:33.374214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:06:33.374214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:06:33.374214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:33.374214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:06:33.374214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:33.374214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:06:33.374214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:33.374214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:06:33.374214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:33.374214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:06:33.374214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:33.374214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:06:33.374214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:06:33.374688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:33.374688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:06:33.374688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:33.374688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:06:33.374688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:06:33.374688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:33.374688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:06:33.374688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:33.374688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:06:33.375106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:33.375106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:06:33.375106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:06:33.375106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:06:33.375106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:06:33.375106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:06:33.375106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:06:33.375106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:06:33.375106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:06:33.375106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:06:33.375106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:33.375106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:06:33.375513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:33.375513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:06:33.375513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:33.375513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:06:33.375513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:33.375513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:06:33.375513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:33.375513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:06:33.375513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:06:33.375513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:33.375513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:06:33.375513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:33.375513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:06:33.375513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:06:33.375948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:33.375948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:06:33.375948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:33.375948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:06:33.376130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:33.376130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:06:33.376130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:06:33.376130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:06:33.376130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:06:33.376130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:06:33.376130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:06:33.376130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:06:33.376130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:06:33.376130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:06:33.376130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:06:33.376130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:06:33.376130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:06:33.376130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:06:33.376130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:06:33.376130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:06:33.376688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:06:33.376688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:06:33.376688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:06:33.376688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:06:33.376688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:06:33.376688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:33.376688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:06:33.376688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:33.376688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:06:33.376688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:33.376688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:06:33.376688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:33.376688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:06:33.376688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:33.376688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:06:33.376688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:06:33.376688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:33.376688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:06:33.377536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:33.377536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:06:33.377536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:06:33.377536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:33.377536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:06:33.377536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:33.377536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:06:33.396879 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:06:33.397324 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:06:33.397512 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:06:33.397734 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:06:33.398351 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:06:33.398781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:33.399578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:06:33.399578 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:33.399578 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:06:33.399578 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:06:33.399578 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:33.399578 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:06:33.399578 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:33.399578 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:06:33.401047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:33.401047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:06:33.401047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:06:33.401047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:06:33.401047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:33.401047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:06:33.401047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:33.401047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:06:33.401047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:33.401047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:06:33.401047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:33.401047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:06:33.401047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:33.401047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:06:33.401047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:06:33.401699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:33.401699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:06:33.401699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:33.401699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:06:33.401699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:06:33.401699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:33.401699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:06:33.401699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:33.401699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:06:33.402646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:33.402646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:06:33.402646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:06:33.402646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:06:33.402646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:06:33.402646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:06:33.402646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:06:33.402646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:06:33.402646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:06:33.402646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:33.402646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:06:33.403315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:33.403315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:06:33.403315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:33.403315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:06:33.403315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:33.403315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:06:33.403315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:33.403315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:06:33.403315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:06:33.403315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:33.403315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:06:33.403315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:33.403315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:06:33.403315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:06:33.404072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:33.404072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:06:33.404072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:33.404072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:06:33.404989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:33.404989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:06:33.404989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:06:33.404989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:06:33.404989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:06:33.404989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:06:33.404989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:06:33.404989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:06:33.404989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:06:33.404989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:06:33.404989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:06:33.404989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:06:33.404989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:06:33.404989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:06:33.404989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:06:33.404989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:06:33.404989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:06:33.405483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:06:33.405483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:06:33.405483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:06:33.405483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:33.405483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:06:33.405483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:33.405483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:06:33.405483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:33.405483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:06:33.405483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:33.405483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:06:33.405483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:33.405483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:06:33.405483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:06:33.405483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:33.405483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:06:33.405483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:33.405483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:06:33.405483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:06:33.406042 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:33.406042 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:06:33.406042 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:33.406042 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:06:33.406042 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:06:33.406042 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:06:33.406042 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:06:33.406042 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:06:33.406483 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:06:33.407001 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:06:33.407237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:33.408101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:06:33.408101 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:33.408101 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:06:33.408101 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:06:33.408101 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:33.408101 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:06:33.408101 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:33.408101 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:06:33.409194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:33.409194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:06:33.409194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:06:33.409194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:06:33.409194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:06:33.409194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:06:33.409194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:06:33.409194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:33.409194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:06:33.409194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:33.409194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:06:33.409194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:33.409550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:06:33.409550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:33.409550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:06:33.409550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:33.409550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:06:33.409550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:06:33.409550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:33.409550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:06:33.409550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:33.409550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:06:33.409550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:06:33.409550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:33.409550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:06:33.409905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:33.409905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:06:33.410432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:33.410432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:06:33.410432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:06:33.410432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:06:33.410432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:06:33.410432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:06:33.410432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:06:33.410432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:06:33.410432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:06:33.410432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:06:33.410432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:06:33.410432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:06:33.410432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:06:33.410432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:06:33.410432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:06:33.410432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:06:33.410890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:06:33.410890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:06:33.410890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:33.410890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:06:33.410890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:33.410890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:06:33.410890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:33.410890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:06:33.410890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:33.410890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:06:33.410890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:33.410890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:06:33.410890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:06:33.410890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:33.410890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:06:33.410890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:33.410890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:06:33.410890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:06:33.410890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:33.411425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:06:33.411425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:33.411425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:06:37.842400 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:06:37.843174 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:06:37.843174 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:06:37.843539 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:06:37.870801 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG dbcounter [-] [106314] Writing DB stats neutron:SELECT=247 {{(pid=106314) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 09 09:06:37.885194 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:06:37.885379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:37.885908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:06:37.885908 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:37.885908 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:06:37.885908 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:06:37.885908 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:37.885908 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:06:37.885908 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:37.885908 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:06:37.886642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.043 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:37.886642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:06:37.886642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:06:37.886642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:06:37.886642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:06:37.886642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:06:37.886642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:06:37.886642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:37.886642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:06:37.886642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:37.886642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:06:37.886642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:37.887006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:06:37.887006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:37.887006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:06:37.887006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:37.887006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:06:37.887006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:06:37.887006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:37.887006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:06:37.887006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:37.887006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:06:37.887006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:06:37.887006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:37.887006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:06:37.887390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:37.887390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:06:37.888453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:37.888453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:06:37.888453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:06:37.888453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:06:37.888453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:06:37.888453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:06:37.888453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:06:37.888453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:06:37.888453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:06:37.888453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:06:37.888453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:06:37.888453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:06:37.888453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:06:37.888453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:06:37.888453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:06:37.888453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:06:37.889147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:06:37.889147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:06:37.889147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:37.889147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:06:37.889147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:37.889147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:06:37.889147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:37.889147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:06:37.889147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:37.889147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:06:37.889147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:37.889147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:06:37.889147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:06:37.889147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:37.889147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:06:37.889147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:37.889147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:06:37.889147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:06:37.889147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:37.889890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:06:37.889890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:37.889890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:06:38.361005 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:06:38.361592 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:06:38.361808 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:06:38.363193 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:06:38.363464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:38.364104 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:06:38.364104 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:38.364104 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:06:38.364104 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:06:38.364104 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:38.364104 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:06:38.364104 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:38.364104 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:06:38.365194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:38.365194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:06:38.365194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:06:38.365194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:06:38.365194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:06:38.365194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:06:38.365194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:06:38.365194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:38.365194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:06:38.365194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:38.365194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:06:38.365194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:38.365708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:06:38.365708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:38.365708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:06:38.365708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:38.365708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:06:38.365708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:06:38.365708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:38.365708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:06:38.365708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:38.365708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:06:38.365708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:06:38.365708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:38.365708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:06:38.366342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:38.366342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:06:38.366342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:38.366342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:06:38.366342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:06:38.366342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:06:38.366342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:06:38.366342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:06:38.366342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:06:38.366342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:06:38.366342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:06:38.366342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:06:38.366342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:06:38.366342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:06:38.366342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:06:38.366992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:06:38.366992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:06:38.366992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:06:38.366992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:06:38.366992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:06:38.366992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:38.366992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:06:38.366992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:38.366992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:06:38.366992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:38.366992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:06:38.366992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:38.366992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:06:38.366992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:38.366992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:06:38.366992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:06:38.366992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:38.366992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:06:38.366992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:38.367785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:06:38.367785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:06:38.367785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:38.367785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:06:38.367785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:38.367785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:06:38.372219 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:06:38.372553 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:06:38.372698 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:06:38.372839 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:06:38.373300 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:06:38.373627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:38.374797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:06:38.374797 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:38.374797 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:06:38.374797 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:06:38.374797 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:38.374797 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:06:38.374797 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:38.374797 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:06:38.375638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:38.375638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:06:38.375638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:06:38.375638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:06:38.375638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:38.375638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:06:38.375638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:38.375638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:06:38.375638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:38.375638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:06:38.375638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:38.375638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:06:38.375638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:38.375638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:06:38.375638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:06:38.376087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:38.376087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:06:38.376087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:38.376087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:06:38.376087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:06:38.376087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:38.376087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:06:38.376087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:38.376087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:06:38.376483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:38.376483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:06:38.376483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:06:38.376483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:06:38.376483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:06:38.376483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:06:38.376483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:06:38.376483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:06:38.376483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:06:38.376483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:06:38.376483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:38.376483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:06:38.376917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:38.376917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:06:38.376917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:38.376917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:06:38.376917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:38.376917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:06:38.376917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:38.376917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:06:38.376917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:06:38.376917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:38.376917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:06:38.376917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:38.376917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:06:38.376917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:06:38.377498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:38.377498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:06:38.377498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:38.377498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:06:38.377674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:38.377674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:06:38.377674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:06:38.377674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:06:38.377674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:06:38.377674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:06:38.377674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:06:38.377674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:06:38.377674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:06:38.377674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:06:38.377674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:06:38.377674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:06:38.377674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:06:38.377674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:06:38.377674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:06:38.377674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:06:38.378299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:06:38.378299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:06:38.378299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:06:38.378299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:06:38.378299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:06:38.378299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:38.378299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:06:38.378299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:38.378299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:06:38.378299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:38.378299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:06:38.378299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:38.378299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:06:38.378299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:38.378299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:06:38.378299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:06:38.378299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:38.378299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:06:38.379038 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:38.379038 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:06:38.379038 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:06:38.379038 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:38.379038 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:06:38.379038 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:38.379038 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:06:38.397666 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:06:38.398164 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:06:38.398354 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:06:38.398543 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:06:38.399170 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:06:38.399684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:38.400537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:06:38.400537 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:38.400537 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:06:38.400537 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:06:38.400537 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:38.400537 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:06:38.400537 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:38.400537 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:06:38.401709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:38.401709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:06:38.401709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:06:38.401709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:06:38.401709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:38.401709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:06:38.401709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:38.401709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:06:38.401709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:38.401709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:06:38.401709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:38.401709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:06:38.401709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:38.401709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:06:38.401709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:06:38.402361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:38.402361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:06:38.402361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:38.402361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:06:38.402361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:06:38.402361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:38.402361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:06:38.402361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:38.402361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:06:38.403124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:38.403124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:06:38.403124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:06:38.403124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:06:38.403124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:06:38.403124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:06:38.403124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:06:38.403124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:06:38.403124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:06:38.403124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:38.403124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:06:38.403588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:38.403588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:06:38.403588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:38.403588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:06:38.403588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:38.403588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:06:38.403588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:38.403588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:06:38.403588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:06:38.403588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:38.403588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:06:38.403588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:38.403588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:06:38.403588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:06:38.404042 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:38.404042 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:06:38.404042 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:38.404042 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:06:38.405194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:38.405194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:06:38.405194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:06:38.405194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:06:38.405194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:06:38.405194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:06:38.405194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:06:38.405194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:06:38.405194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:06:38.405194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:06:38.405194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:06:38.405194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:06:38.405194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:06:38.405194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:06:38.405194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:06:38.405194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:06:38.405194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:06:38.405642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:06:38.405642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:06:38.405642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:06:38.405642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:38.405642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:06:38.405642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:38.405642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:06:38.405642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:38.405642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:06:38.405642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:38.405642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:06:38.405642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:38.405642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:06:38.405642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:06:38.405642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:38.405642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:06:38.405642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:38.405642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:06:38.405642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:06:38.406138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:38.406138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:06:38.406138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:38.406138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:06:38.406138 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:06:38.406320 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:06:38.406365 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:06:38.406587 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:06:38.407020 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:06:38.407490 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:06:38.407490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:38.407930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:06:38.407930 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:38.407930 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:06:38.407930 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:06:38.407930 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:38.407930 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:06:38.407930 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:38.407930 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:06:38.408890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:38.408890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:06:38.408890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:06:38.408890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:06:38.408890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:06:38.408890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:06:38.408890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:06:38.408890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:38.408890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:06:38.408890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:38.408890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:06:38.408890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:38.409224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:06:38.409224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:38.409224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:06:38.409224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:38.409224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:06:38.409224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:06:38.409224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:38.409224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:06:38.409224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:38.409224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:06:38.409224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:06:38.409224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:38.409224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:06:38.409551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:38.409551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:06:38.410331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:38.410331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:06:38.410331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:06:38.410331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:06:38.410331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:06:38.410331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:06:38.410331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:06:38.410331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:06:38.410331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:06:38.410331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:06:38.410331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:06:38.410331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:06:38.410331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:06:38.410331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:06:38.410331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:06:38.410331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:06:38.410800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:06:38.410800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:06:38.410800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:38.410800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:06:38.410800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:38.410800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:06:38.410800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:38.410800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:06:38.410800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:38.410800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:06:38.410800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:38.410800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:06:38.410800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:06:38.410800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:38.410800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:06:38.410800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:38.410800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:06:38.410800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:06:38.410800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:38.411304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:06:38.411304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:38.411304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:06:42.843054 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:06:42.843811 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:06:42.843930 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:06:42.845173 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:06:42.875104 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:06:42.875249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:42.875687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:06:42.875687 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:42.875687 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:06:42.875687 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:06:42.875687 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:42.875687 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:06:42.875687 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:42.875687 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:06:42.876469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.032 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:42.876469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:06:42.876469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:06:42.876469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:06:42.876469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:06:42.876469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:06:42.876469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:06:42.876469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:42.876469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:06:42.876469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:42.876469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:06:42.876469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:42.877130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:06:42.877130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:42.877130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:06:42.877130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:42.877130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:06:42.877130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:06:42.877130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:42.877130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:06:42.877130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:42.877130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:06:42.877130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:06:42.877130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:42.877130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:06:42.877702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:42.877702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:06:42.877702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:42.877702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:06:42.877702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:06:42.877702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:06:42.877702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:06:42.877702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:06:42.877702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:06:42.877702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:06:42.877702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:06:42.877702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:06:42.877702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:06:42.877702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:06:42.877702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:06:42.878383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:06:42.878383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:06:42.878383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:06:42.878383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:06:42.878383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:06:42.878383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:42.878383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:06:42.878383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:42.878383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:06:42.878383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:42.878383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:06:42.878383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:42.878383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:06:42.878383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:42.878383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:06:42.878383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:06:42.878383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:42.878383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:06:42.878383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:42.879210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:06:42.879210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:06:42.879210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:42.879210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:06:42.879210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:42.879210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:06:43.361888 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:06:43.362407 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:06:43.362661 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:06:43.363937 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:06:43.364316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:43.365197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:06:43.365197 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:43.365197 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:06:43.365197 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:06:43.365197 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:43.365197 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:06:43.365197 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:43.365197 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:06:43.367382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:43.367382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:06:43.367382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:06:43.367382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:06:43.367382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:06:43.367382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:06:43.367382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:06:43.367382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:43.367382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:06:43.367382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:43.367382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:06:43.367382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:43.367995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:06:43.367995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:43.367995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:06:43.367995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:43.367995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:06:43.367995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:06:43.367995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:43.367995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:06:43.367995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:43.367995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:06:43.367995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:06:43.367995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:43.367995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:06:43.368538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:43.368538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:06:43.369879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:43.369879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:06:43.369879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:06:43.369879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:06:43.369879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:06:43.369879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:06:43.369879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:06:43.369879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:06:43.369879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:06:43.369879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:06:43.369879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:06:43.369879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:06:43.369879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:06:43.369879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:06:43.369879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:06:43.369879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:06:43.370560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:06:43.370560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:06:43.370560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:43.370560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:06:43.370560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:43.370560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:06:43.370560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:43.370560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:06:43.370560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:43.370560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:06:43.370560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:43.370560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:06:43.370560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:06:43.370560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:43.370560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:06:43.370560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:43.370560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:06:43.370560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:06:43.370560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:43.371839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:06:43.371839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:43.371839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:06:43.372684 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:06:43.373010 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:06:43.373177 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:06:43.373355 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:06:43.373740 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:06:43.373959 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:43.374466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:06:43.374466 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:43.374466 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:06:43.374466 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:06:43.374466 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:43.374466 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:06:43.374466 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:43.374466 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:06:43.375230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:43.375230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:06:43.375230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:06:43.375230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:06:43.375230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:43.375230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:06:43.375230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:43.375230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:06:43.375230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:43.375230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:06:43.375230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:43.375230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:06:43.375230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:43.375230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:06:43.375230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:06:43.375676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:43.375676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:06:43.375676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:43.375676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:06:43.375676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:06:43.375676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:43.375676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:06:43.375676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:43.375676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:06:43.376151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:43.376151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:06:43.376151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:06:43.376151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:06:43.376151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:06:43.376151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:06:43.376151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:06:43.376151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:06:43.376151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:06:43.376151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:06:43.376151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:43.376151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:06:43.376489 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:43.376489 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:06:43.376489 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:43.376489 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:06:43.376489 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:43.376489 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:06:43.376489 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:43.376489 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:06:43.376489 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:06:43.376489 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:43.376489 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:06:43.376489 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:43.376489 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:06:43.376489 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:06:43.376897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:43.376897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:06:43.376897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:43.376897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:06:43.377228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:43.377228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:06:43.377228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:06:43.377228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:06:43.377228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:06:43.377228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:06:43.377228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:06:43.377228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:06:43.377228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:06:43.377228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:06:43.377228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:06:43.377228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:06:43.377228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:06:43.377228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:06:43.377228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:06:43.377228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:06:43.377688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:06:43.377688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:06:43.377688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:06:43.377688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:06:43.377688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:06:43.377688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:43.377688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:06:43.377688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:43.377688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:06:43.377688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:43.377688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:06:43.377688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:43.377688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:06:43.377688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:43.377688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:06:43.377688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:06:43.377688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:43.377688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:06:43.378191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:43.378191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:06:43.378191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:06:43.378191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:43.378191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:06:43.378191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:43.378191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:06:43.398170 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:06:43.398668 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:06:43.398985 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:06:43.399284 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:06:43.400202 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:06:43.400788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:43.401649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:06:43.401649 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:43.401649 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:06:43.401649 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:06:43.401649 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:43.401649 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:06:43.401649 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:43.401649 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:06:43.403297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:43.403297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:06:43.403297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:06:43.403297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:06:43.403297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:43.403297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:06:43.403297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:43.403297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:06:43.403297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:43.403297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:06:43.403297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:43.403297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:06:43.403297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:43.403297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:06:43.403297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:06:43.404354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:43.404354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:06:43.404354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:43.404354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:06:43.404354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:06:43.404354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:43.404354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:06:43.404354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:43.404354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:06:43.405283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:43.405283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:06:43.405283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:06:43.405283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:06:43.405283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:06:43.405283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:06:43.405283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:06:43.405283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:06:43.405283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:06:43.405283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:43.405283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:06:43.405775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:43.405775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:06:43.405775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:43.405775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:06:43.405775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:43.405775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:06:43.405775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:43.405775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:06:43.405775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:06:43.405775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:43.405775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:06:43.405775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:43.405775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:06:43.405775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:06:43.406335 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:43.406335 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:06:43.406335 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:43.406335 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:06:43.406853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:43.406853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:06:43.406853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:06:43.406853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:06:43.406853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:06:43.406853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:06:43.406853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:06:43.406853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:06:43.406853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:06:43.406853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:06:43.406853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:06:43.406853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:06:43.406853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:06:43.406853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:06:43.406853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:06:43.406853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:06:43.406853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:06:43.407491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:06:43.407491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:06:43.407491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:06:43.407491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:43.407491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:06:43.407491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:43.407491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:06:43.407491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:43.407491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:06:43.407491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:43.407491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:06:43.407491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:43.407491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:06:43.407491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:06:43.407491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:43.407491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:06:43.407491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:43.407491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:06:43.407491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:06:43.408260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:43.408260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:06:43.408260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:43.408260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:06:43.408260 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:06:43.408260 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:06:43.408260 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:06:43.408260 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:06:43.408519 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:06:43.408519 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:06:43.408596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:43.409139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:06:43.409139 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:43.409139 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:06:43.409139 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:06:43.409139 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:43.409139 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:06:43.409139 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:43.409139 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:06:43.410117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:43.410117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:06:43.410117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:06:43.410117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:06:43.410117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:06:43.410117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:06:43.410117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:06:43.410117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:43.410117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:06:43.410117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:43.410117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:06:43.410117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:43.410629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:06:43.410629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:43.410629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:06:43.410629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:43.410629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:06:43.410629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:06:43.410629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:43.410629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:06:43.410629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:43.410629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:06:43.410629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:06:43.410629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:43.410629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:06:43.411174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:43.411174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:06:43.411458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:43.411458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:06:43.411458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:06:43.411458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:06:43.411458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:06:43.411458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:06:43.411458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:06:43.411458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:06:43.411458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:06:43.411458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:06:43.411458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:06:43.411458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:06:43.411458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:06:43.411458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:06:43.411458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:06:43.411458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:06:43.412001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:06:43.412001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:06:43.412001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:43.412001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:06:43.412001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:43.412001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:06:43.412001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:43.412001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:06:43.412001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:43.412001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:06:43.412001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:43.412001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:06:43.412001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:06:43.412001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:43.412001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:06:43.412001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:43.412001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:06:43.412001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:06:43.412001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:43.412507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:06:43.412507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:43.412507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:06:47.843692 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:06:47.844818 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:06:47.844818 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:06:47.846627 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:06:47.882554 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:06:47.882981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:47.883767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:06:47.883767 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:47.883767 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:06:47.883767 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:06:47.883767 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:47.883767 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:06:47.883767 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:47.883767 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:06:47.884844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:47.884844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:06:47.884844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:06:47.884844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:06:47.884844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:06:47.884844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:06:47.884844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:06:47.884844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:47.884844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:06:47.884844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:47.884844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:06:47.884844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:47.885371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:06:47.885371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:47.885371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:06:47.885371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:47.885371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:06:47.885371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:06:47.885371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:47.885371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:06:47.885371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:47.885371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:06:47.885371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:06:47.885371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:47.885371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:06:47.885941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:47.885941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:06:47.885941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:47.885941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:06:47.885941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:06:47.885941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:06:47.885941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:06:47.885941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:06:47.885941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:06:47.885941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:06:47.885941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:06:47.885941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:06:47.885941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:06:47.885941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:06:47.885941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:06:47.886576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:06:47.886576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:06:47.886576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:06:47.886576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:06:47.886576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:06:47.886576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:47.886576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:06:47.886576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:47.886576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:06:47.886576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:47.886576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:06:47.886576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:47.886576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:06:47.886576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:47.886576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:06:47.886576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:06:47.886576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:47.886576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:06:47.886576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:47.887478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:06:47.887478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:06:47.887478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:47.887478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:06:47.887478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:47.887478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:06:48.362542 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:06:48.363036 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:06:48.363149 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:06:48.364151 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:06:48.364516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:48.365470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:06:48.365470 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:48.365470 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:06:48.365470 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:06:48.365470 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:48.365470 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:06:48.365470 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:48.365470 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:06:48.366501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:48.366501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:06:48.366501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:06:48.366501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:06:48.366501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:06:48.366501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:06:48.366501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:06:48.366501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:48.366501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:06:48.366501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:48.366501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:06:48.366501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:48.367056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:06:48.367056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:48.367056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:06:48.367056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:48.367056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:06:48.367056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:06:48.367056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:48.367056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:06:48.367056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:48.367056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:06:48.367056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:06:48.367056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:48.367056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:06:48.367646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:48.367646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:06:48.367646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:48.367646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:06:48.367646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:06:48.367646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:06:48.367646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:06:48.367646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:06:48.367646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:06:48.367646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:06:48.367646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:06:48.367646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:06:48.367646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:06:48.367646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:06:48.367646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:06:48.368597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:06:48.368597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:06:48.368597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:06:48.368597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:06:48.368597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:06:48.368597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:48.368597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:06:48.368597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:48.368597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:06:48.368597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:48.368597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:06:48.368597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:48.368597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:06:48.368597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:48.368597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:06:48.368597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:06:48.368597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:48.368597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:06:48.368597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:48.369447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:06:48.369447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:06:48.369447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:48.369447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:06:48.369447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:48.369447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:06:48.373099 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:06:48.373355 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:06:48.373470 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:06:48.373596 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:06:48.374092 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:06:48.374390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:48.375110 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:06:48.375110 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:48.375110 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:06:48.375110 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:06:48.375110 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:48.375110 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:06:48.375110 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:48.375110 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:06:48.375899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:48.375899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:06:48.375899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:06:48.375899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:06:48.375899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:48.375899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:06:48.375899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:48.375899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:06:48.375899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:48.375899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:06:48.375899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:48.375899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:06:48.375899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:48.375899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:06:48.375899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:06:48.376358 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:48.376358 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:06:48.376358 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:48.376358 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:06:48.376358 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:06:48.376358 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:48.376358 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:06:48.376358 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:48.376358 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:06:48.377250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:48.377250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:06:48.377250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:06:48.377250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:06:48.377250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:06:48.377250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:06:48.377250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:06:48.377250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:06:48.377250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:06:48.377250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:06:48.377250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:48.377250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:06:48.377596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:48.377596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:06:48.377596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:48.377596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:06:48.377596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:48.377596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:06:48.377596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:48.377596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:06:48.377596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:06:48.377596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:48.377596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:06:48.377596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:48.377596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:06:48.377596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:06:48.378058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:48.378058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:06:48.378058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:48.378058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:06:48.379066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:48.379066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:06:48.379066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:06:48.379066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:06:48.379066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:06:48.379066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:06:48.379066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:06:48.379066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:06:48.379066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:06:48.379066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:06:48.379066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:06:48.379066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:06:48.379066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:06:48.379066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:06:48.379066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:06:48.379066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:06:48.379851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:06:48.379851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:06:48.379851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:06:48.379851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:06:48.379851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:06:48.379851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:48.379851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:06:48.379851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:48.379851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:06:48.379851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:48.379851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:06:48.379851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:48.379851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:06:48.379851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:48.379851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:06:48.379851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:06:48.379851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:48.379851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:06:48.380647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:48.380647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:06:48.380647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:06:48.380647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:48.380647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:06:48.380647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:48.380647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:06:48.398672 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:06:48.399097 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:06:48.399279 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:06:48.399474 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:06:48.400106 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:06:48.400496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:48.401571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:06:48.401571 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:48.401571 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:06:48.401571 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:06:48.401571 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:48.401571 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:06:48.401571 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:48.401571 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:06:48.402822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:48.402822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:06:48.402822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:06:48.402822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:06:48.402822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:48.402822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:06:48.402822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:48.402822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:06:48.402822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:48.402822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:06:48.402822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:48.402822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:06:48.402822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:48.402822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:06:48.402822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:06:48.403446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:48.403446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:06:48.403446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:48.403446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:06:48.403446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:06:48.403446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:48.403446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:06:48.403446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:48.403446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:06:48.404204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:48.404204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:06:48.404204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:06:48.404204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:06:48.404204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:06:48.404204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:06:48.404204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:06:48.404204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:06:48.404204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:06:48.404204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:48.404204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:06:48.404714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:48.404714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:06:48.404714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:48.404714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:06:48.404714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:48.404714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:06:48.404714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:48.404714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:06:48.404714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:06:48.404714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:48.404714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:06:48.404714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:48.404714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:06:48.404714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:06:48.405335 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:48.405335 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:06:48.405335 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:48.405335 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:06:48.406108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:48.406108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:06:48.406108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:06:48.406108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:06:48.406108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:06:48.406108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:06:48.406108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:06:48.406108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:06:48.406108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:06:48.406108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:06:48.406108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:06:48.406108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:06:48.406108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:06:48.406108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:06:48.406108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:06:48.406108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:06:48.406108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:06:48.406898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:06:48.406898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:06:48.406898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:06:48.406898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:48.406898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:06:48.406898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:48.406898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:06:48.406898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:48.406898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:06:48.406898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:48.406898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:06:48.406898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:48.406898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:06:48.406898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:06:48.406898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:48.406898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:06:48.406898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:48.406898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:06:48.406898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:06:48.407726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:48.407726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:06:48.407726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:48.407726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:06:48.407726 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:06:48.407992 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:06:48.408211 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:06:48.408439 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:06:48.409063 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:06:48.409453 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:06:48.409723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:48.410687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:06:48.410687 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:48.410687 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:06:48.410687 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:06:48.410687 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:48.410687 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:06:48.410687 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:48.410687 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:06:48.411719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:48.411719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:06:48.411719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:06:48.411719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:06:48.411719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:06:48.411719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:06:48.411719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:06:48.411719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:48.411719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:06:48.411719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:48.411719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:06:48.411719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:48.412306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:06:48.412306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:48.412306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:06:48.412306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:48.412306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:06:48.412306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:06:48.412306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:48.412306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:06:48.412306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:48.412306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:06:48.412306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:06:48.412306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:48.412306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:06:48.412966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:48.412966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:06:48.412966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:48.412966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:06:48.412966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:06:48.412966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:06:48.412966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:06:48.412966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:06:48.412966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:06:48.412966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:06:48.412966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:06:48.412966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:06:48.412966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:06:48.412966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:06:48.412966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:06:48.413528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:06:48.413528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:06:48.413528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:06:48.413528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:06:48.413528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:06:48.413528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:48.413528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:06:48.413528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:48.413528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:06:48.413528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:48.413528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:06:48.413528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:48.413528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:06:48.413528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:48.413528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:06:48.413528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:06:48.413528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:48.413528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:06:48.413528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:48.414085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:06:48.414085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:06:48.414085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:48.414085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:06:48.414085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:48.414085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:06:52.844838 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:06:52.848056 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:06:52.848056 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:06:52.848056 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:06:52.878586 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:06:52.879018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:52.879900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:06:52.879900 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:52.879900 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:06:52.879900 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:06:52.879900 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:52.879900 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:06:52.879900 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:52.879900 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:06:52.880692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:52.880692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:06:52.880692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:06:52.880692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:06:52.880692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:06:52.880692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:06:52.880692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:06:52.880692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:52.880692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:06:52.880692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:52.880692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:06:52.880692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:52.881291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:06:52.881291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:52.881291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:06:52.881291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:52.881291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:06:52.881291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:06:52.881291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:52.881291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:06:52.881291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:52.881291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:06:52.881291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:06:52.881291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:52.881291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:06:52.882032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:52.882032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:06:52.882032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:52.882032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:06:52.882032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:06:52.882032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:06:52.882032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:06:52.882032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:06:52.882032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:06:52.882032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:06:52.882032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:06:52.882032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:06:52.882032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:06:52.882032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:06:52.882032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:06:52.882739 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:06:52.882739 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:06:52.882739 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:06:52.882739 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:06:52.882739 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:06:52.882739 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:52.882739 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:06:52.882739 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:52.882739 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:06:52.882739 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:52.882739 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:06:52.882739 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:52.882739 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:06:52.882739 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:52.882739 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:06:52.882739 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:06:52.882739 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:52.882739 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:06:52.882739 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:52.883600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:06:52.883600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:06:52.883600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:52.883600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:06:52.883600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:52.883600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:06:53.363354 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:06:53.364142 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:06:53.364337 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:06:53.365281 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:06:53.365506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:53.366211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:06:53.366211 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:53.366211 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:06:53.366211 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:06:53.366211 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:53.366211 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:06:53.366211 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:53.366211 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:06:53.367962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:53.367962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:06:53.367962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:06:53.367962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:06:53.367962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:06:53.367962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:06:53.367962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:06:53.367962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:53.367962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:06:53.367962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:53.367962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:06:53.367962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:53.368622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:06:53.368622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:53.368622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:06:53.368622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:53.368622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:06:53.368622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:06:53.368622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:53.368622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:06:53.368622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:53.368622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:06:53.368622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:06:53.368622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:53.368622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:06:53.369287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:53.369287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:06:53.369761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:53.369761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:06:53.369761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:06:53.369761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:06:53.369761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:06:53.369761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:06:53.369761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:06:53.369761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:06:53.369761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:06:53.369761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:06:53.369761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:06:53.369761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:06:53.369761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:06:53.369761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:06:53.369761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:06:53.369761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:06:53.370544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:06:53.370544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:06:53.370544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:53.370544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:06:53.370544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:53.370544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:06:53.370544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:53.370544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:06:53.370544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:53.370544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:06:53.370544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:53.370544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:06:53.370544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:06:53.370544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:53.370544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:06:53.370544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:53.370544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:06:53.370544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:06:53.370544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:53.371445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:06:53.371445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:53.371445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:06:53.373560 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:06:53.373910 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:06:53.374010 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:06:53.374143 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:06:53.374659 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:06:53.374996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:53.375698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:06:53.375698 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:53.375698 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:06:53.375698 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:06:53.375698 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:53.375698 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:06:53.375698 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:53.375698 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:06:53.376501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:53.376501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:06:53.376501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:06:53.376501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:06:53.376501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:53.376501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:06:53.376501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:53.376501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:06:53.376501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:53.376501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:06:53.376501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:53.376501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:06:53.376501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:53.376501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:06:53.376501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:06:53.376952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:53.376952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:06:53.376952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:53.376952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:06:53.376952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:06:53.376952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:53.376952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:06:53.376952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:53.376952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:06:53.377396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:53.377396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:06:53.377396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:06:53.377396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:06:53.377396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:06:53.377396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:06:53.377396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:06:53.377396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:06:53.377396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:06:53.377396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:06:53.377396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:53.377396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:06:53.377741 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:53.377741 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:06:53.377741 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:53.377741 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:06:53.377741 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:53.377741 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:06:53.377741 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:53.377741 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:06:53.377741 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:06:53.377741 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:53.377741 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:06:53.377741 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:53.377741 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:06:53.377741 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:06:53.378133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:53.378133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:06:53.378133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:53.378133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:06:53.378476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:53.378476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:06:53.378476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:06:53.378476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:06:53.378476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:06:53.378476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:06:53.378476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:06:53.378476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:06:53.378476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:06:53.378476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:06:53.378476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:06:53.378476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:06:53.378476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:06:53.378476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:06:53.378476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:06:53.378476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:06:53.378952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:06:53.378952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:06:53.378952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:06:53.378952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:06:53.378952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:06:53.378952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:53.378952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:06:53.378952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:53.378952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:06:53.378952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:53.378952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:06:53.378952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:53.378952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:06:53.378952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:53.378952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:06:53.378952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:06:53.378952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:53.378952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:06:53.379419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:53.379419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:06:53.379419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:06:53.379419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:53.379419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:06:53.379419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:53.379419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:06:53.399137 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:06:53.399575 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:06:53.399776 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:06:53.400027 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:06:53.400622 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:06:53.401075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:53.401869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:06:53.401869 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:53.401869 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:06:53.401869 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:06:53.401869 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:53.401869 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:06:53.401869 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:53.401869 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:06:53.403194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:53.403194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:06:53.403194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:06:53.403194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:06:53.403194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:53.403194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:06:53.403194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:53.403194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:06:53.403194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:53.403194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:06:53.403194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:53.403194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:06:53.403194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:53.403194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:06:53.403194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:06:53.403882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:53.403882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:06:53.403882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:53.403882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:06:53.403882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:06:53.403882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:53.403882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:06:53.403882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:53.403882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:06:53.404625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:53.404625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:06:53.404625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:06:53.404625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:06:53.404625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:06:53.404625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:06:53.404625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:06:53.404625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:06:53.404625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:06:53.404625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:53.404625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:06:53.405188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:53.405188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:06:53.405188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:53.405188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:06:53.405188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:53.405188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:06:53.405188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:53.405188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:06:53.405188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:06:53.405188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:53.405188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:06:53.405188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:53.405188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:06:53.405188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:06:53.405839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:53.405839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:06:53.405839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:53.405839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:06:53.406352 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:53.406352 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:06:53.406352 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:06:53.406352 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:06:53.406352 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:06:53.406352 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:06:53.406352 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:06:53.406352 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:06:53.406352 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:06:53.406352 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:06:53.406352 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:06:53.406352 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:06:53.406352 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:06:53.406352 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:06:53.406352 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:06:53.406352 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:06:53.406352 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:06:53.407235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:06:53.407235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:06:53.407235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:06:53.407235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:53.407235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:06:53.407235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:53.407235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:06:53.407235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:53.407235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:06:53.407235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:53.407235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:06:53.407235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:53.407235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:06:53.407235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:06:53.407235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:53.407235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:06:53.407235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:53.407235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:06:53.407235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:06:53.407864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:53.407864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:06:53.407864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:53.407864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:06:53.407989 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:06:53.408360 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:06:53.408475 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:06:53.408632 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:06:53.409124 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:06:53.409473 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:06:53.409680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:53.410392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:06:53.410392 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:53.410392 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:06:53.410392 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:06:53.410392 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:53.410392 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:06:53.410392 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:53.410392 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:06:53.411415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:53.411415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:06:53.411415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:06:53.411415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:06:53.411415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:06:53.411415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:06:53.411415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:06:53.411415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:53.411415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:06:53.411415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:53.411415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:06:53.411415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:53.411791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:06:53.411791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:53.411791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:06:53.411791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:53.411791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:06:53.411791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:06:53.411791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:53.411791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:06:53.411791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:53.411791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:06:53.411791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:06:53.411791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:53.411791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:06:53.412164 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:53.412164 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:06:53.412392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:53.412392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:06:53.412392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:06:53.412392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:06:53.412392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:06:53.412392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:06:53.412392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:06:53.412392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:06:53.412392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:06:53.412392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:06:53.412392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:06:53.412392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:06:53.412392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:06:53.412392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:06:53.412392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:06:53.412392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:06:53.412858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:06:53.412858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:06:53.412858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:53.412858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:06:53.412858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:53.412858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:06:53.412858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:53.412858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:06:53.412858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:53.412858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:06:53.412858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:53.412858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:06:53.412858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:06:53.412858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:53.412858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:06:53.412858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:53.412858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:06:53.412858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:06:53.412858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:53.413392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:06:53.413392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:53.413392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:06:57.845528 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:06:57.846316 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:06:57.846498 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:06:57.847917 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:06:57.884639 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:06:57.885228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:57.886234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:06:57.886234 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:57.886234 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:06:57.886234 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:06:57.886234 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:57.886234 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:06:57.886234 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:57.886234 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:06:57.887130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:57.887130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:06:57.887130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:06:57.887130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:06:57.887130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:06:57.887130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:06:57.887130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:06:57.887130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:57.887130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:06:57.887130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:57.887130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:06:57.887130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:57.887527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:06:57.887527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:57.887527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:06:57.887527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:57.887527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:06:57.887527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:06:57.887527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:57.887527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:06:57.887527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:57.887527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:06:57.887527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:06:57.887527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:57.887527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:06:57.887875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:57.887875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:06:57.888244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:57.888244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:06:57.888244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:06:57.888244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:06:57.888244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:06:57.888244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:06:57.888244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:06:57.888244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:06:57.888244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:06:57.888244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:06:57.888244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:06:57.888244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:06:57.888244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:06:57.888244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:06:57.888244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:06:57.888244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:06:57.888585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:06:57.888585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:06:57.888585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:57.888585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:06:57.888585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:57.888585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:06:57.888585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:57.888585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:06:57.888585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:57.888585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:06:57.888585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:57.888585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:06:57.888585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:06:57.888585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:57.888585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:06:57.888585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:57.888585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:06:57.888585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:06:57.888585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:57.888978 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:06:57.888978 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:57.888978 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:06:58.364170 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:06:58.364928 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:06:58.365072 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:06:58.366209 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:06:58.366558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:58.367571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:06:58.367571 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:58.367571 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:06:58.367571 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:06:58.367571 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:58.367571 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:06:58.367571 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:58.367571 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:06:58.369355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:58.369355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:06:58.369355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:06:58.369355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:06:58.369355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:06:58.369355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:06:58.369355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:06:58.369355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:58.369355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:06:58.369355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:58.369355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:06:58.369355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:58.369997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:06:58.369997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:58.369997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:06:58.369997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:58.369997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:06:58.369997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:06:58.369997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:58.369997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:06:58.369997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:58.369997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:06:58.369997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:06:58.369997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:58.369997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:06:58.370522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:58.370522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:06:58.370522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:58.370522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:06:58.370522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:06:58.370522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:06:58.370522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:06:58.370522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:06:58.370522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:06:58.370522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:06:58.370522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:06:58.370522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:06:58.370522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:06:58.370522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:06:58.370522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:06:58.371279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:06:58.371279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:06:58.371279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:06:58.371279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:06:58.371279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:06:58.371279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:58.371279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:06:58.371279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:58.371279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:06:58.371279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:58.371279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:06:58.371279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:58.371279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:06:58.371279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:58.371279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:06:58.371279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:06:58.371279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:58.371279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:06:58.371279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:58.372090 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:06:58.372090 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:06:58.372090 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:58.372090 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:06:58.372090 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:58.372090 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:06:58.373941 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:06:58.374222 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:06:58.374327 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:06:58.374458 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:06:58.374844 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:06:58.375069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:58.375577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:06:58.375577 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:58.375577 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:06:58.375577 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:06:58.375577 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:58.375577 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:06:58.375577 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:58.375577 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:06:58.376340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:58.376340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:06:58.376340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:06:58.376340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:06:58.376340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:58.376340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:06:58.376340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:58.376340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:06:58.376340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:58.376340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:06:58.376340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:58.376340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:06:58.376340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:58.376340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:06:58.376340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:06:58.376727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:58.376727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:06:58.376727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:58.376727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:06:58.376727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:06:58.376727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:58.376727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:06:58.376727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:58.376727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:06:58.377216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:58.377216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:06:58.377216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:06:58.377216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:06:58.377216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:06:58.377216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:06:58.377216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:06:58.377216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:06:58.377216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:06:58.377216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:06:58.377216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:58.377216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:06:58.377544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:58.377544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:06:58.377544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:58.377544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:06:58.377544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:58.377544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:06:58.377544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:58.377544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:06:58.377544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:06:58.377544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:58.377544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:06:58.377544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:58.377544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:06:58.377544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:06:58.378091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:58.378091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:06:58.378091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:58.378091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:06:58.378311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:58.378311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:06:58.378311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:06:58.378311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:06:58.378311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:06:58.378311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:06:58.378311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:06:58.378311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:06:58.378311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:06:58.378311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:06:58.378311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:06:58.378311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:06:58.378311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:06:58.378311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:06:58.378311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:06:58.378311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:06:58.378842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:06:58.378842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:06:58.378842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:06:58.378842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:06:58.378842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:06:58.378842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:58.378842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:06:58.378842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:58.378842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:06:58.378842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:58.378842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:06:58.378842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:58.378842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:06:58.378842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:58.378842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:06:58.378842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:06:58.378842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:58.378842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:06:58.379318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:58.379318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:06:58.379318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:06:58.379318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:58.379318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:06:58.379318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:58.379318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:06:58.399635 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:06:58.399971 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:06:58.400073 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:06:58.400208 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:06:58.400771 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:06:58.401300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:58.402133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:06:58.402133 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:58.402133 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:06:58.402133 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:06:58.402133 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:58.402133 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:06:58.402133 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:58.402133 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:06:58.402955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:58.402955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:06:58.402955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:06:58.402955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:06:58.402955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:58.402955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:06:58.402955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:58.402955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:06:58.402955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:58.402955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:06:58.402955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:58.402955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:06:58.402955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:58.402955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:06:58.402955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:06:58.403326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:58.403326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:06:58.403326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:58.403326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:06:58.403326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:06:58.403326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:58.403326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:06:58.403326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:58.403326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:06:58.403827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:58.403827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:06:58.403827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:06:58.403827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:06:58.403827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:06:58.403827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:06:58.403827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:06:58.403827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:06:58.403827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:06:58.403827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:58.403827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:06:58.404132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:58.404132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:06:58.404132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:58.404132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:06:58.404132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:58.404132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:06:58.404132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:58.404132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:06:58.404132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:06:58.404132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:58.404132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:06:58.404132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:58.404132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:06:58.404132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:06:58.404480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:58.404480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:06:58.404480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:58.404480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:06:58.404858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:58.404858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:06:58.404858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:06:58.404858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:06:58.404858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:06:58.404858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:06:58.404858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:06:58.404858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:06:58.404858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:06:58.404858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:06:58.404858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:06:58.404858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:06:58.404858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:06:58.404858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:06:58.404858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:06:58.404858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:06:58.404858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:06:58.405329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:06:58.405329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:06:58.405329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:06:58.405329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:58.405329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:06:58.405329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:58.405329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:06:58.405329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:58.405329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:06:58.405329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:58.405329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:06:58.405329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:58.405329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:06:58.405329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:06:58.405329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:58.405329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:06:58.405329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:58.405329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:06:58.405329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:06:58.405877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:58.405877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:06:58.405877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:58.405877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:06:58.408494 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:06:58.408757 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:06:58.408913 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:06:58.409097 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:06:58.409448 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:06:58.409700 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:06:58.409895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:58.410424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:06:58.410424 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:58.410424 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:06:58.410424 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:06:58.410424 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:58.410424 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:06:58.410424 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:58.410424 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:06:58.411257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:58.411257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:06:58.411257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:06:58.411257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:06:58.411257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:06:58.411257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:06:58.411257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:06:58.411257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:58.411257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:06:58.411257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:58.411257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:06:58.411257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:58.411579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:06:58.411579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:58.411579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:06:58.411579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:58.411579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:06:58.411579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:06:58.411579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:58.411579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:06:58.411579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:58.411579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:06:58.411579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:06:58.411579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:58.411579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:06:58.411938 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:58.411938 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:06:58.412200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:58.412200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:06:58.412200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:06:58.412200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:06:58.412200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:06:58.412200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:06:58.412200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:06:58.412200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:06:58.412200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:06:58.412200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:06:58.412200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:06:58.412200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:06:58.412200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:06:58.412200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:06:58.412200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:06:58.412200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:06:58.412615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:06:58.412615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:06:58.412615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:58.412615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:06:58.412615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:06:58.412615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:06:58.412615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:06:58.412615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:06:58.412615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:06:58.412615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:06:58.412615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:06:58.412615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:06:58.412615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:06:58.412615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:06:58.412615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:06:58.412615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:06:58.412615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:06:58.412615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:06:58.412615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:06:58.413169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:06:58.413169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:06:58.413169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:07:02.846422 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:07:02.847453 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:07:02.847453 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:07:02.848217 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:07:02.872496 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:07:02.872918 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:02.873739 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:07:02.873739 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:02.873739 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:07:02.873739 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:07:02.873739 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:02.873739 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:07:02.873739 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:02.873739 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:07:02.874476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.026 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:02.874476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:07:02.874476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:07:02.874476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:07:02.874476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:07:02.874476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:07:02.874476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:07:02.874476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:02.874476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:07:02.874476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:02.874476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:07:02.874476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:02.875043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:07:02.875043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:02.875043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:07:02.875043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:02.875043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:07:02.875043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:07:02.875043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:02.875043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:07:02.875043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:02.875043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:07:02.875043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:07:02.875043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:02.875043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:07:02.876115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:02.876115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:07:02.876115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:02.876115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:07:02.876115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:07:02.876115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:07:02.876115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:07:02.876115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:07:02.876115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:07:02.876115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:07:02.876115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:07:02.876115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:07:02.876115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:07:02.876115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:07:02.876115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:07:02.876670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:07:02.876670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:07:02.876670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:07:02.876670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:07:02.876670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:07:02.876670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:02.876670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:07:02.876670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:02.876670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:07:02.876670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:02.876670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:07:02.876670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:02.876670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:07:02.876670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:02.876670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:07:02.876670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:07:02.876670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:02.876670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:07:02.876670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:02.877264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:07:02.877264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:07:02.877264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:02.877264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:07:02.877264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:02.877264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:07:03.364996 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:07:03.365498 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:07:03.365819 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:07:03.366900 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:07:03.367175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:03.368109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:07:03.368109 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:03.368109 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:07:03.368109 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:07:03.368109 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:03.368109 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:07:03.368109 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:03.368109 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:07:03.369699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:03.369699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:07:03.369699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:07:03.369699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:07:03.369699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:07:03.369699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:07:03.369699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:07:03.369699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:03.369699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:07:03.369699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:03.369699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:07:03.369699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:03.370348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:07:03.370348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:03.370348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:07:03.370348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:03.370348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:07:03.370348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:07:03.370348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:03.370348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:07:03.370348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:03.370348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:07:03.370348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:07:03.370348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:03.370348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:07:03.370979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:03.370979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:07:03.371714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:03.371714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:07:03.371714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:07:03.371714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:07:03.371714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:07:03.371714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:07:03.371714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:07:03.371714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:07:03.371714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:07:03.371714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:07:03.371714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:07:03.371714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:07:03.371714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:07:03.371714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:07:03.371714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:07:03.371714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:07:03.372644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:07:03.372644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:07:03.372644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:03.372644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:07:03.372644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:03.372644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:07:03.372644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:03.372644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:07:03.372644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:03.372644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:07:03.372644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:03.372644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:07:03.372644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:07:03.372644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:03.372644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:07:03.372644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:03.372644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:07:03.372644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:07:03.372644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:03.373533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:07:03.373533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:03.373533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:07:03.374447 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:07:03.374761 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:07:03.374999 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:07:03.375188 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:07:03.375656 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:07:03.376066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:03.376765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:07:03.376765 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:03.376765 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:07:03.376765 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:07:03.376765 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:03.376765 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:07:03.376765 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:03.376765 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:07:03.377541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:03.377541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:07:03.377541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:07:03.377541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:07:03.377541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:03.377541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:07:03.377541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:03.377541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:07:03.377541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:03.377541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:07:03.377541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:03.377541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:07:03.377541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:03.377541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:07:03.377541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:07:03.379211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:03.379211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:07:03.379211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:03.379211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:07:03.379211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:07:03.379211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:03.379211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:07:03.379211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:03.379211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:07:03.379211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:03.379211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:07:03.379211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:07:03.379211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:07:03.379211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:07:03.380110 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:07:03.380110 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:07:03.380110 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:07:03.380110 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:07:03.380110 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:07:03.380110 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:03.380110 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:07:03.380110 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:03.380110 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:07:03.380110 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:03.380110 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:07:03.380110 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:03.380110 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:07:03.380618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:03.380618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:07:03.380618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:07:03.380618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:03.380618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:07:03.380618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:03.380618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:07:03.380618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:07:03.380618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:03.380618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:07:03.380618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:03.380618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:07:03.380966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:03.380966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:07:03.380966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:07:03.380966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:07:03.380966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:07:03.380966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:07:03.380966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:07:03.380966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:07:03.380966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:07:03.380966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:07:03.380966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:07:03.380966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:07:03.380966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:07:03.380966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:07:03.380966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:07:03.380966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:07:03.381363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:07:03.381363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:07:03.381363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:07:03.381363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:07:03.381363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:07:03.381363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:03.381363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:07:03.381363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:03.381363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:07:03.381363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:03.381363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:07:03.381363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:03.381363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:07:03.381363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:03.381363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:07:03.381363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:07:03.381363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:03.381363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:07:03.381918 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:03.381918 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:07:03.381918 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:07:03.381918 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:03.381918 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:07:03.381918 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:03.381918 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:07:03.400581 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:07:03.400978 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:07:03.401168 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:07:03.401382 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:07:03.401957 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:07:03.402367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:03.403204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:07:03.403204 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:03.403204 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:07:03.403204 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:07:03.403204 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:03.403204 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:07:03.403204 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:03.403204 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:07:03.404480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:03.404480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:07:03.404480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:07:03.404480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:07:03.404480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:03.404480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:07:03.404480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:03.404480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:07:03.404480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:03.404480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:07:03.404480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:03.404480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:07:03.404480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:03.404480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:07:03.404480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:07:03.405239 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:03.405239 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:07:03.405239 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:03.405239 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:07:03.405239 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:07:03.405239 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:03.405239 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:07:03.405239 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:03.405239 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:07:03.405877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:03.405877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:07:03.405877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:07:03.405877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:07:03.405877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:07:03.405877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:07:03.405877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:07:03.405877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:07:03.405877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:07:03.405877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:03.405877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:07:03.406417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:03.406417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:07:03.406417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:03.406417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:07:03.406417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:03.406417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:07:03.406417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:03.406417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:07:03.406417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:07:03.406417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:03.406417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:07:03.406417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:03.406417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:07:03.406417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:07:03.406997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:03.406997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:07:03.406997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:03.406997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:07:03.407542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:03.407542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:07:03.407542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:07:03.407542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:07:03.407542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:07:03.407542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:07:03.407542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:07:03.407542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:07:03.407542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:07:03.407542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:07:03.407542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:07:03.407542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:07:03.407542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:07:03.407542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:07:03.407542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:07:03.407542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:07:03.407542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:07:03.409110 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:07:03.409110 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:07:03.409110 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:07:03.409110 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:03.409110 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:07:03.409110 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:03.409110 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:07:03.409110 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:03.409110 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:07:03.409110 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:03.409110 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:07:03.409110 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:03.409110 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:07:03.409110 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:07:03.409110 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:03.409110 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:07:03.409110 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:03.409110 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:07:03.409110 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:07:03.409670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:03.409670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:07:03.409670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:03.409670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:07:03.409670 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:07:03.409670 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:07:03.409670 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:07:03.409670 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:07:03.410117 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:07:03.410483 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:07:03.410661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:03.411353 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:07:03.411353 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:03.411353 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:07:03.411353 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:07:03.411353 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:03.411353 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:07:03.411353 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:03.411353 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:07:03.412696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:03.412696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:07:03.412696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:07:03.412696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:07:03.412696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:07:03.412696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:07:03.412696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:07:03.412696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:03.412696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:07:03.412696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:03.412696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:07:03.412696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:03.413399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:07:03.413399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:03.413399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:07:03.413399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:03.413399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:07:03.413399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:07:03.413399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:03.413399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:07:03.413399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:03.413399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:07:03.413399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:07:03.413399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:03.413399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:07:03.413809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:03.413809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:07:03.413809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:03.413809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:07:03.413809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:07:03.413809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:07:03.413809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:07:03.413809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:07:03.413809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:07:03.413809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:07:03.413809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:07:03.413809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:07:03.413809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:07:03.413809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:07:03.413809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:07:03.414291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:07:03.414291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:07:03.414291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:07:03.414291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:07:03.414291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:07:03.414291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:03.414291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:07:03.414291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:03.414291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:07:03.414291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:03.414291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:07:03.414291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:03.414291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:07:03.414291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:03.414291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:07:03.414291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:07:03.414291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:03.414291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:07:03.414291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:03.414961 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:07:03.414961 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:07:03.414961 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:03.414961 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:07:03.414961 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:03.414961 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:07:07.847323 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:07:07.848144 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:07:07.848144 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:07:07.849434 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:07:07.884786 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:07:07.885340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:07.886328 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:07:07.886328 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:07.886328 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:07:07.886328 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:07:07.886328 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:07.886328 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:07:07.886328 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:07.886328 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:07:07.887322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:07.887322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:07:07.887322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:07:07.887322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:07:07.887322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:07:07.887322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:07:07.887322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:07:07.887322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:07.887322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:07:07.887322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:07.887322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:07:07.887322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:07.888004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:07:07.888004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:07.888004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:07:07.888004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:07.888004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:07:07.888004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:07:07.888004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:07.888004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:07:07.888004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:07.888004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:07:07.888004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:07:07.888004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:07.888004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:07:07.888656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:07.888656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:07:07.888656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:07.888656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:07:07.888656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:07:07.888656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:07:07.888656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:07:07.888656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:07:07.888656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:07:07.888656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:07:07.888656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:07:07.888656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:07:07.888656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:07:07.888656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:07:07.888656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:07:07.889336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:07:07.889336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:07:07.889336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:07:07.889336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:07:07.889336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:07:07.889336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:07.889336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:07:07.889336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:07.889336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:07:07.889336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:07.889336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:07:07.889336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:07.889336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:07:07.889336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:07.889336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:07:07.889336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:07:07.889336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:07.889336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:07:07.889336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:07.890557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:07:07.890557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:07:07.890557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:07.890557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:07:07.890557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:07.890557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:07:08.365563 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:07:08.366185 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:07:08.366383 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:07:08.367469 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:07:08.367834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:08.368735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:07:08.368735 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:08.368735 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:07:08.368735 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:07:08.368735 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:08.368735 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:07:08.368735 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:08.368735 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:07:08.370288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:08.370288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:07:08.370288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:07:08.370288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:07:08.370288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:07:08.370288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:07:08.370288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:07:08.370288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:08.370288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:07:08.370288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:08.370288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:07:08.370288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:08.371017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:07:08.371017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:08.371017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:07:08.371017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:08.371017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:07:08.371017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:07:08.371017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:08.371017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:07:08.371017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:08.371017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:07:08.371017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:07:08.371017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:08.371017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:07:08.371625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:08.371625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:07:08.372016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:08.372016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:07:08.372016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:07:08.372016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:07:08.372016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:07:08.372016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:07:08.372016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:07:08.372016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:07:08.372016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:07:08.372016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:07:08.372016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:07:08.372016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:07:08.372016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:07:08.372016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:07:08.372016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:07:08.372016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:07:08.372823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:07:08.372823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:07:08.372823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:08.372823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:07:08.372823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:08.372823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:07:08.372823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:08.372823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:07:08.372823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:08.372823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:07:08.372823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:08.372823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:07:08.372823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:07:08.372823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:08.372823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:07:08.372823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:08.372823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:07:08.372823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:07:08.372823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:08.373724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:07:08.373724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:08.373724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:07:08.374856 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:07:08.375147 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:07:08.375253 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:07:08.375385 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:07:08.376194 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:07:08.376504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:08.377232 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:07:08.377232 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:08.377232 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:07:08.377232 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:07:08.377232 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:08.377232 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:07:08.377232 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:08.377232 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:07:08.377992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:08.377992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:07:08.377992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:07:08.377992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:07:08.377992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:08.377992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:07:08.377992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:08.377992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:07:08.377992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:08.377992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:07:08.377992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:08.377992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:07:08.377992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:08.377992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:07:08.377992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:07:08.378737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:08.378737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:07:08.378737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:08.378737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:07:08.378737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:07:08.378737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:08.378737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:07:08.378737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:08.378737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:07:08.379029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:08.379029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:07:08.379029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:07:08.379029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:07:08.379029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:07:08.379029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:07:08.379029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:07:08.379029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:07:08.379029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:07:08.379029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:07:08.379029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:08.379029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:07:08.379380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:08.379380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:07:08.379380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:08.379380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:07:08.379380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:08.379380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:07:08.379380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:08.379380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:07:08.379380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:07:08.379380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:08.379380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:07:08.379380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:08.379380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:07:08.379380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:07:08.379782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:08.379782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:07:08.379782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:08.379782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:07:08.379896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:08.379896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:07:08.379896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:07:08.379896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:07:08.379896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:07:08.379896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:07:08.379896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:07:08.379896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:07:08.379896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:07:08.379896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:07:08.379896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:07:08.379896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:07:08.379896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:07:08.379896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:07:08.379896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:07:08.379896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:07:08.380428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:07:08.380428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:07:08.380428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:07:08.380428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:07:08.380428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:07:08.380428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:08.380428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:07:08.380428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:08.380428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:07:08.380428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:08.380428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:07:08.380428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:08.380428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:07:08.380428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:08.380428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:07:08.380428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:07:08.380428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:08.380428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:07:08.380935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:08.380935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:07:08.380935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:07:08.380935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:08.380935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:07:08.380935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:08.380935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:07:08.401004 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:07:08.401314 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:07:08.401428 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:07:08.401554 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:07:08.402184 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:07:08.402648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:08.403603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:07:08.403603 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:08.403603 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:07:08.403603 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:07:08.403603 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:08.403603 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:07:08.403603 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:08.403603 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:07:08.404633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:08.404633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:07:08.404633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:07:08.404633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:07:08.404633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:08.404633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:07:08.404633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:08.404633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:07:08.404633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:08.404633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:07:08.404633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:08.404633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:07:08.404633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:08.404633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:07:08.404633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:07:08.405080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:08.405080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:07:08.405080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:08.405080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:07:08.405080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:07:08.405080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:08.405080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:07:08.405080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:08.405080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:07:08.406030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:08.406030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:07:08.406030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:07:08.406030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:07:08.406030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:07:08.406030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:07:08.406030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:07:08.406030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:07:08.406030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:07:08.406030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:08.406030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:07:08.406373 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:08.406373 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:07:08.406373 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:08.406373 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:07:08.406373 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:08.406373 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:07:08.406373 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:08.406373 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:07:08.406373 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:07:08.406373 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:08.406373 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:07:08.406373 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:08.406373 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:07:08.406373 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:07:08.406848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:08.406848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:07:08.406848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:08.406848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:07:08.407705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:08.407705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:07:08.407705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:07:08.407705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:07:08.407705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:07:08.407705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:07:08.407705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:07:08.407705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:07:08.407705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:07:08.407705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:07:08.407705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:07:08.407705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:07:08.407705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:07:08.407705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:07:08.407705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:07:08.407705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:07:08.407705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:07:08.408195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:07:08.408195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:07:08.408195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:07:08.408195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:08.408195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:07:08.408195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:08.408195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:07:08.408195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:08.408195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:07:08.408195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:08.408195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:07:08.408195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:08.408195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:07:08.408195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:07:08.408195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:08.408195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:07:08.408195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:08.408195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:07:08.408195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:07:08.408704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:08.408704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:07:08.408704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:08.408704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:07:08.409325 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:07:08.409661 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:07:08.409864 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:07:08.410102 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:07:08.410489 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:07:08.410767 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:07:08.410921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:08.411469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:07:08.411469 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:08.411469 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:07:08.411469 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:07:08.411469 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:08.411469 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:07:08.411469 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:08.411469 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:07:08.412758 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:08.412758 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:07:08.412758 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:07:08.412758 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:07:08.412758 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:07:08.412758 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:07:08.412758 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:07:08.412758 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:08.412758 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:07:08.412758 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:08.412758 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:07:08.412758 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:08.413156 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:07:08.413156 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:08.413156 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:07:08.413156 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:08.413156 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:07:08.413156 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:07:08.413156 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:08.413156 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:07:08.413156 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:08.413156 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:07:08.413156 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:07:08.413156 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:08.413156 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:07:08.413737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:08.413737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:07:08.414339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:08.414339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:07:08.414339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:07:08.414339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:07:08.414339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:07:08.414339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:07:08.414339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:07:08.414339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:07:08.414339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:07:08.414339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:07:08.414339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:07:08.414339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:07:08.414339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:07:08.414339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:07:08.414339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:07:08.414339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:07:08.415330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:07:08.415330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:07:08.415330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:08.415330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:07:08.415330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:08.415330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:07:08.415330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:08.415330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:07:08.415330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:08.415330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:07:08.415330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:08.415330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:07:08.415330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:07:08.415330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:08.415330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:07:08.415330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:08.415330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:07:08.415330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:07:08.415330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:08.416284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:07:08.416284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:08.416284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:07:12.848118 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:07:12.849113 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:07:12.849113 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:07:12.850943 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:07:12.888590 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:07:12.888590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:12.888982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:07:12.888982 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:12.888982 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:07:12.888982 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:07:12.888982 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:12.888982 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:07:12.888982 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:12.888982 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:07:12.889838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:12.889838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:07:12.889838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:07:12.889838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:07:12.889838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:07:12.889838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:07:12.889838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:07:12.889838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:12.889838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:07:12.889838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:12.889838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:07:12.889838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:12.890470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:07:12.890470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:12.890470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:07:12.890470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:12.890470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:07:12.890470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:07:12.890470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:12.890470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:07:12.890470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:12.890470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:07:12.890470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:07:12.890470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:12.890470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:07:12.891101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:12.891101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:07:12.891101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:12.891101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:07:12.891101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:07:12.891101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:07:12.891101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:07:12.891101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:07:12.891101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:07:12.891101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:07:12.891101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:07:12.891101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:07:12.891101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:07:12.891101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:07:12.891101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:07:12.892249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:07:12.892249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:07:12.892249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:07:12.892249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:07:12.892249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:07:12.892249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:12.892249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:07:12.892249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:12.892249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:07:12.892249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:12.892249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:07:12.892249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:12.892249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:07:12.892249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:12.892249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:07:12.892249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:07:12.892249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:12.892249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:07:12.892249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:12.894209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:07:12.894209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:07:12.894209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:12.894209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:07:12.894209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:12.894209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:07:13.366152 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:07:13.366579 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:07:13.366923 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:07:13.368916 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:07:13.368916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:13.369206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:07:13.369206 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:13.369206 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:07:13.369206 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:07:13.369206 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:13.369206 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:07:13.369206 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:13.369206 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:07:13.371026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:13.371026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:07:13.371026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:07:13.371026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:07:13.371026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:07:13.371026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:07:13.371026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:07:13.371026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:13.371026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:07:13.371026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:13.371026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:07:13.371026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:13.371670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:07:13.371670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:13.371670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:07:13.371670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:13.371670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:07:13.371670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:07:13.371670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:13.371670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:07:13.371670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:13.371670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:07:13.371670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:07:13.371670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:13.371670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:07:13.372436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:13.372436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:07:13.372916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:13.372916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:07:13.372916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:07:13.372916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:07:13.372916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:07:13.372916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:07:13.372916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:07:13.372916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:07:13.372916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:07:13.372916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:07:13.372916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:07:13.372916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:07:13.372916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:07:13.372916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:07:13.372916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:07:13.372916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:07:13.373709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:07:13.373709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:07:13.373709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:13.373709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:07:13.373709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:13.373709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:07:13.373709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:13.373709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:07:13.373709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:13.373709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:07:13.373709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:13.373709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:07:13.373709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:07:13.373709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:13.373709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:07:13.373709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:13.373709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:07:13.373709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:07:13.373709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:13.374612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:07:13.374612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:13.374612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:07:13.375166 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:07:13.375563 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:07:13.375677 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:07:13.375881 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:07:13.376292 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:07:13.376509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:13.377089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:07:13.377089 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:13.377089 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:07:13.377089 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:07:13.377089 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:13.377089 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:07:13.377089 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:13.377089 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:07:13.378064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:13.378064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:07:13.378064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:07:13.378064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:07:13.378064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:13.378064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:07:13.378064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:13.378064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:07:13.378064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:13.378064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:07:13.378064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:13.378064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:07:13.378064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:13.378064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:07:13.378064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:07:13.378484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:13.378484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:07:13.378484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:13.378484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:07:13.378484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:07:13.378484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:13.378484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:07:13.378484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:13.378484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:07:13.379213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:13.379213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:07:13.379213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:07:13.379213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:07:13.379213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:07:13.379213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:07:13.379213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:07:13.379213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:07:13.379213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:07:13.379213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:07:13.379213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:13.379213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:07:13.379695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:13.379695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:07:13.379695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:13.379695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:07:13.379695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:13.379695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:07:13.379695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:13.379695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:07:13.379695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:07:13.379695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:13.379695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:07:13.379695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:13.379695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:07:13.379695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:07:13.380113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:13.380113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:07:13.380113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:13.380113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:07:13.380654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:13.380654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:07:13.380654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:07:13.380654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:07:13.380654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:07:13.380654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:07:13.380654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:07:13.380654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:07:13.380654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:07:13.380654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:07:13.380654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:07:13.380654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:07:13.380654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:07:13.380654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:07:13.380654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:07:13.380654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:07:13.381207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:07:13.381207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:07:13.381207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:07:13.381207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:07:13.381207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:07:13.381207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:13.381207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:07:13.381207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:13.381207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:07:13.381207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:13.381207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:07:13.381207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:13.381207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:07:13.381207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:13.381207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:07:13.381207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:07:13.381207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:13.381207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:07:13.381663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:13.381663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:07:13.381663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:07:13.381663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:13.381663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:07:13.381663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:13.381663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:07:13.401574 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:07:13.402053 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:07:13.402244 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:07:13.402506 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:07:13.403225 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:07:13.403535 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:13.404402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:07:13.404402 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:13.404402 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:07:13.404402 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:07:13.404402 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:13.404402 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:07:13.404402 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:13.404402 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:07:13.405777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:13.405777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:07:13.405777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:07:13.405777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:07:13.405777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:13.405777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:07:13.405777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:13.405777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:07:13.405777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:13.405777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:07:13.405777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:13.405777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:07:13.405777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:13.405777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:07:13.405777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:07:13.406624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:13.406624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:07:13.406624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:13.406624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:07:13.406624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:07:13.406624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:13.406624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:07:13.406624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:13.406624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:07:13.407402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:13.407402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:07:13.407402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:07:13.407402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:07:13.407402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:07:13.407402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:07:13.407402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:07:13.407402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:07:13.407402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:07:13.407402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:13.407402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:07:13.407949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:13.407949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:07:13.407949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:13.407949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:07:13.407949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:13.407949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:07:13.407949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:13.407949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:07:13.407949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:07:13.407949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:13.407949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:07:13.407949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:13.407949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:07:13.407949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:07:13.408700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:13.408700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:07:13.408700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:13.408700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:07:13.409220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:13.409220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:07:13.409220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:07:13.409220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:07:13.409220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:07:13.409220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:07:13.409220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:07:13.409220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:07:13.409220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:07:13.409220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:07:13.409220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:07:13.409220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:07:13.409220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:07:13.409220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:07:13.409220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:07:13.409220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:07:13.409220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:07:13.410019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:07:13.410019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:07:13.410019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:07:13.410019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:13.410019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:07:13.410019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:13.410019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:07:13.410019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:13.410019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:07:13.410019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:13.410019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:07:13.410019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:13.410019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:07:13.410019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:07:13.410019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:13.410019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:07:13.410019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:13.410019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:07:13.410019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:07:13.411183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:13.411183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:07:13.411183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:13.411183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:07:13.411183 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:07:13.411183 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:07:13.411183 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:07:13.411183 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:07:13.411569 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:07:13.411569 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:07:13.411714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:13.412456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:07:13.412456 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:13.412456 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:07:13.412456 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:07:13.412456 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:13.412456 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:07:13.412456 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:13.412456 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:07:13.413574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:13.413574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:07:13.413574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:07:13.413574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:07:13.413574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:07:13.413574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:07:13.413574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:07:13.413574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:13.413574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:07:13.413574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:13.413574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:07:13.413574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:13.413991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:07:13.413991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:13.413991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:07:13.413991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:13.413991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:07:13.413991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:07:13.413991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:13.413991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:07:13.413991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:13.413991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:07:13.413991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:07:13.413991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:13.413991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:07:13.414407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:13.414407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:07:13.414648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:13.414648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:07:13.414648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:07:13.414648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:07:13.414648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:07:13.414648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:07:13.414648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:07:13.414648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:07:13.414648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:07:13.414648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:07:13.414648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:07:13.414648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:07:13.414648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:07:13.414648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:07:13.414648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:07:13.414648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:07:13.415140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:07:13.415140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:07:13.415140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:13.415140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:07:13.415140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:13.415140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:07:13.415140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:13.415140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:07:13.415140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:13.415140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:07:13.415140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:13.415140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:07:13.415140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:07:13.415140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:13.415140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:07:13.415140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:13.415140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:07:13.415140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:07:13.415140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:13.415629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:07:13.415629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:13.415629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:07:17.848949 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:07:17.849872 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:07:17.849872 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:07:17.850981 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:07:17.890886 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:07:17.891229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:17.892297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:07:17.892297 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:17.892297 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:07:17.892297 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:07:17.892297 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:17.892297 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:07:17.892297 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:17.892297 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:07:17.893658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.043 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:17.893658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:07:17.893658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:07:17.893658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:07:17.893658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:07:17.893658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:07:17.893658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:07:17.893658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:17.893658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:07:17.893658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:17.893658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:07:17.893658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:17.894862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:07:17.894862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:17.894862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:07:17.894862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:17.894862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:07:17.894862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:07:17.894862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:17.894862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:07:17.894862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:17.894862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:07:17.894862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:07:17.894862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:17.894862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:07:17.895579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:17.895579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:07:17.895579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:17.895579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:07:17.895579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:07:17.895579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:07:17.895579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:07:17.895579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:07:17.895579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:07:17.895579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:07:17.895579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:07:17.895579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:07:17.895579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:07:17.895579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:07:17.895579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:07:17.896271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:07:17.896271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:07:17.896271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:07:17.896271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:07:17.896271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:07:17.896271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:17.896271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:07:17.896271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:17.896271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:07:17.896271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:17.896271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:07:17.896271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:17.896271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:07:17.896271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:17.896271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:07:17.896271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:07:17.896271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:17.896271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:07:17.896271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:17.897051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:07:17.897051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:07:17.897051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:17.897051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:07:17.897051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:17.897051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:07:18.366799 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:07:18.367717 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:07:18.368048 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:07:18.369656 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:07:18.370128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:18.371254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:07:18.371254 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:18.371254 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:07:18.371254 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:07:18.371254 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:18.371254 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:07:18.371254 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:18.371254 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:07:18.373557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:18.373557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:07:18.373557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:07:18.373557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:07:18.373557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:07:18.373557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:07:18.373557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:07:18.373557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:18.373557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:07:18.373557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:18.373557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:07:18.373557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:18.374793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:07:18.374793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:18.374793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:07:18.374793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:18.374793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:07:18.374793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:07:18.374793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:18.374793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:07:18.374793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:18.374793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:07:18.374793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:07:18.374793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:18.374793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:07:18.375467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:18.375467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:07:18.376630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:18.376630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:07:18.376630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:07:18.376630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:07:18.376630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:07:18.376630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:07:18.376630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:07:18.376630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:07:18.376630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:07:18.376630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:07:18.376630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:07:18.376630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:07:18.376630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:07:18.376630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:07:18.376630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:07:18.376630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:07:18.377125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:07:18.377125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:07:18.377125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:18.377125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:07:18.377125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:18.377125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:07:18.377125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:18.377125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:07:18.377125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:18.377125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:07:18.377125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:18.377125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:07:18.377125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:07:18.377125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:18.377125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:07:18.377125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:18.377125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:07:18.377125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:07:18.377125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:18.377590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:07:18.377590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:18.377590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:07:18.377590 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:07:18.377590 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:07:18.377590 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:07:18.378403 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:07:18.378956 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:07:18.379185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:18.379976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:07:18.379976 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:18.379976 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:07:18.379976 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:07:18.379976 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:18.379976 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:07:18.379976 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:18.379976 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:07:18.380716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:18.380716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:07:18.380716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:07:18.380716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:07:18.380716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:18.380716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:07:18.380716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:18.380716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:07:18.380716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:18.380716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:07:18.380716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:18.380716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:07:18.380716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:18.380716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:07:18.380716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:07:18.381179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:18.381179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:07:18.381179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:18.381179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:07:18.381179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:07:18.381179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:18.381179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:07:18.381179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:18.381179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:07:18.381431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:18.381431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:07:18.381431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:07:18.381431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:07:18.381431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:07:18.381431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:07:18.381431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:07:18.381431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:07:18.381431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:07:18.381431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:07:18.381431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:18.381431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:07:18.381846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:18.381846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:07:18.381846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:18.381846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:07:18.381846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:18.381846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:07:18.381846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:18.381846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:07:18.381846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:07:18.381846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:18.381846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:07:18.381846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:18.381846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:07:18.381846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:07:18.382236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:18.382236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:07:18.382236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:18.382236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:07:18.382236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:18.382236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:07:18.382236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:07:18.382236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:07:18.382236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:07:18.382236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:07:18.382236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:07:18.382236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:07:18.382236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:07:18.382236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:07:18.382735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:07:18.382735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:07:18.382735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:07:18.382735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:07:18.382735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:07:18.382735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:07:18.382735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:07:18.382735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:07:18.382735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:07:18.382735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:07:18.382735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:07:18.382735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:18.382735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:07:18.382735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:18.382735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:07:18.382735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:18.382735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:07:18.382735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:18.382735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:07:18.383324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:18.383324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:07:18.383324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:07:18.383324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:18.383324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:07:18.383324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:18.383324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:07:18.383324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:07:18.383324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:18.383324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:07:18.383324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:18.383324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:07:18.402073 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:07:18.402550 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:07:18.402846 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:07:18.403137 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:07:18.403803 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:07:18.404195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:18.405129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:07:18.405129 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:18.405129 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:07:18.405129 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:07:18.405129 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:18.405129 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:07:18.405129 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:18.405129 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:07:18.406579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:18.406579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:07:18.406579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:07:18.406579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:07:18.406579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:18.406579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:07:18.406579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:18.406579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:07:18.406579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:18.406579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:07:18.406579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:18.406579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:07:18.406579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:18.406579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:07:18.406579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:07:18.407569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:18.407569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:07:18.407569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:18.407569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:07:18.407569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:07:18.407569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:18.407569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:07:18.407569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:18.407569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:07:18.408591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:18.408591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:07:18.408591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:07:18.408591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:07:18.408591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:07:18.408591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:07:18.408591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:07:18.408591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:07:18.408591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:07:18.408591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:18.408591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:07:18.409395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:18.409395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:07:18.409395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:18.409395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:07:18.409395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:18.409395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:07:18.409395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:18.409395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:07:18.409395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:07:18.409395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:18.409395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:07:18.409395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:18.409395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:07:18.409395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:07:18.410326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:18.410326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:07:18.410326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:18.410326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:07:18.411101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:18.411101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:07:18.411101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:07:18.411101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:07:18.411101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:07:18.411101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:07:18.411101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:07:18.411101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:07:18.411101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:07:18.411101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:07:18.411101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:07:18.411101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:07:18.411101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:07:18.411101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:07:18.411101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:07:18.411101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:07:18.411101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:07:18.412238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:07:18.412238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:07:18.412238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:07:18.412238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:18.412238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:07:18.412238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:18.412238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:07:18.412238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:18.412238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:07:18.412238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:18.412238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:07:18.412238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:18.412238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:07:18.412238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:07:18.412238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:18.412238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:07:18.412238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:18.412238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:07:18.412238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:07:18.412666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:18.412666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:07:18.412666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:18.412666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:07:18.412666 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:07:18.412666 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:07:18.412666 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:07:18.412666 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:07:18.412865 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:07:18.413198 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:07:18.413378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:18.414118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:07:18.414118 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:18.414118 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:07:18.414118 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:07:18.414118 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:18.414118 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:07:18.414118 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:18.414118 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:07:18.414926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:18.414926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:07:18.414926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:07:18.414926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:07:18.414926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:07:18.414926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:07:18.414926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:07:18.414926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:18.414926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:07:18.414926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:18.414926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:07:18.414926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:18.415715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:07:18.415715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:18.415715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:07:18.415715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:18.415715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:07:18.415715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:07:18.415715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:18.415715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:07:18.415715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:18.415715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:07:18.415715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:07:18.415715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:18.415715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:07:18.416335 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:18.416335 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:07:18.416335 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:18.416335 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:07:18.416335 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:07:18.416335 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:07:18.416335 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:07:18.416335 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:07:18.416335 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:07:18.416335 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:07:18.416335 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:07:18.416335 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:07:18.416335 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:07:18.416335 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:07:18.416335 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:07:18.417037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:07:18.417037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:07:18.417037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:07:18.417037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:07:18.417037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:07:18.417037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:18.417037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:07:18.417037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:18.417037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:07:18.417037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:18.417037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:07:18.417037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:18.417037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:07:18.417037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:18.417037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:07:18.417037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:07:18.417037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:18.417037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:07:18.417037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:18.417904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:07:18.417904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:07:18.417904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:18.417904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:07:18.417904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:18.417904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:07:22.849641 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:07:22.851011 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:07:22.851011 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:07:22.852607 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:07:22.881601 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:07:22.882094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:22.882993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:07:22.882993 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:22.882993 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:07:22.882993 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:07:22.882993 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:22.882993 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:07:22.882993 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:22.882993 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:07:22.883927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.032 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:22.883927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:07:22.883927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:07:22.883927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:07:22.883927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:07:22.883927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:07:22.883927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:07:22.883927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:22.883927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:07:22.883927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:22.883927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:07:22.883927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:22.884561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:07:22.884561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:22.884561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:07:22.884561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:22.884561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:07:22.884561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:07:22.884561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:22.884561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:07:22.884561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:22.884561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:07:22.884561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:07:22.884561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:22.884561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:07:22.885112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:22.885112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:07:22.885112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:22.885112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:07:22.885112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:07:22.885112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:07:22.885112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:07:22.885112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:07:22.885112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:07:22.885112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:07:22.885112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:07:22.885112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:07:22.885112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:07:22.885112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:07:22.885112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:07:22.886514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:07:22.886514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:07:22.886514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:07:22.886514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:07:22.886514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:07:22.886514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:22.886514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:07:22.886514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:22.886514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:07:22.886514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:22.886514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:07:22.886514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:22.886514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:07:22.886514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:22.886514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:07:22.886514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:07:22.886514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:22.886514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:07:22.886514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:22.887225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:07:22.887225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:07:22.887225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:22.887225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:07:22.887225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:22.887225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:07:23.367729 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:07:23.368328 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:07:23.368459 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:07:23.369120 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:07:23.369316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:23.369961 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:07:23.369961 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:23.369961 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:07:23.369961 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:07:23.369961 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:23.369961 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:07:23.369961 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:23.369961 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:07:23.371016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:23.371016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:07:23.371016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:07:23.371016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:07:23.371016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:07:23.371016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:07:23.371016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:07:23.371016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:23.371016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:07:23.371016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:23.371016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:07:23.371016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:23.371509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:07:23.371509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:23.371509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:07:23.371509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:23.371509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:07:23.371509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:07:23.371509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:23.371509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:07:23.371509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:23.371509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:07:23.371509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:07:23.371509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:23.371509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:07:23.371899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:23.371899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:07:23.372088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:23.372088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:07:23.372088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:07:23.372088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:07:23.372088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:07:23.372088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:07:23.372088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:07:23.372088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:07:23.372088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:07:23.372088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:07:23.372088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:07:23.372088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:07:23.372088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:07:23.372088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:07:23.372088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:07:23.372088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:07:23.372523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:07:23.372523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:07:23.372523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:23.372523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:07:23.372523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:23.372523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:07:23.372523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:23.372523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:07:23.372523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:23.372523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:07:23.372523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:23.372523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:07:23.372523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:07:23.372523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:23.372523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:07:23.372523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:23.372523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:07:23.372523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:07:23.372523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:23.373114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:07:23.373114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:23.373114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:07:23.377480 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:07:23.377795 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:07:23.377911 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:07:23.378042 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:07:23.378452 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:07:23.378691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:23.379255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:07:23.379255 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:23.379255 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:07:23.379255 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:07:23.379255 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:23.379255 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:07:23.379255 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:23.379255 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:07:23.380431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:23.380431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:07:23.380431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:07:23.380431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:07:23.380431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:23.380431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:07:23.380431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:23.380431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:07:23.380431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:23.380431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:07:23.380431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:23.380431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:07:23.380431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:23.380431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:07:23.380431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:07:23.381192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:23.381192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:07:23.381192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:23.381192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:07:23.381192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:07:23.381192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:23.381192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:07:23.381192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:23.381192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:07:23.381886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:23.381886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:07:23.381886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:07:23.381886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:07:23.381886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:07:23.381886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:07:23.381886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:07:23.381886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:07:23.381886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:07:23.381886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:07:23.381886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:23.381886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:07:23.382285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:23.382285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:07:23.382285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:23.382285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:07:23.382285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:23.382285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:07:23.382285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:23.382285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:07:23.382285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:07:23.382285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:23.382285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:07:23.382285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:23.382285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:07:23.382285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:07:23.382666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:23.382666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:07:23.382666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:23.382666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:07:23.383838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:23.383838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:07:23.383838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:07:23.383838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:07:23.383838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:07:23.383838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:07:23.383838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:07:23.383838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:07:23.383838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:07:23.383838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:07:23.383838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:07:23.383838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:07:23.383838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:07:23.383838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:07:23.383838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:07:23.383838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:07:23.384283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:07:23.384283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:07:23.384283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:07:23.384283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:07:23.384283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:07:23.384283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:23.384283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:07:23.384283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:23.384283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:07:23.384283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:23.384283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:07:23.384283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:23.384283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:07:23.384283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:23.384283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:07:23.384283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:07:23.384283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:23.384283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:07:23.384815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:23.384815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:07:23.384815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:07:23.384815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:23.384815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:07:23.384815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:23.384815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:07:23.402537 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:07:23.402993 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:07:23.403168 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:07:23.403367 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:07:23.403965 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:07:23.404416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:23.405260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:07:23.405260 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:23.405260 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:07:23.405260 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:07:23.405260 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:23.405260 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:07:23.405260 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:23.405260 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:07:23.406479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:23.406479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:07:23.406479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:07:23.406479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:07:23.406479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:23.406479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:07:23.406479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:23.406479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:07:23.406479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:23.406479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:07:23.406479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:23.406479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:07:23.406479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:23.406479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:07:23.406479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:07:23.407482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:23.407482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:07:23.407482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:23.407482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:07:23.407482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:07:23.407482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:23.407482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:07:23.407482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:23.407482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:07:23.408154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:23.408154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:07:23.408154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:07:23.408154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:07:23.408154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:07:23.408154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:07:23.408154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:07:23.408154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:07:23.408154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:07:23.408154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:23.408154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:07:23.409138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:23.409138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:07:23.409138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:23.409138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:07:23.409138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:23.409138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:07:23.409138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:23.409138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:07:23.409138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:07:23.409138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:23.409138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:07:23.409138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:23.409138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:07:23.409138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:07:23.409844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:23.409844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:07:23.409844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:23.409844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:07:23.409844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:23.409844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:07:23.409844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:07:23.409844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:07:23.409844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:07:23.409844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:07:23.409844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:07:23.409844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:07:23.409844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:07:23.409844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:07:23.410474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:07:23.410474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:07:23.410474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:07:23.410474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:07:23.410474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:07:23.410474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:07:23.410474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:07:23.410474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:07:23.410474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:07:23.410474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:07:23.410474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:23.410474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:07:23.410474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:23.410474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:07:23.410474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:23.410474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:07:23.410474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:23.410474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:07:23.410474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:23.411439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:07:23.411439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:07:23.411439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:23.411439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:07:23.411439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:23.411439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:07:23.411439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:07:23.411439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:23.411439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:07:23.411439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:23.411439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:07:23.411903 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:07:23.412258 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:07:23.412419 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:07:23.412636 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:07:23.412981 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:07:23.413210 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:07:23.413332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:23.413826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:07:23.413826 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:23.413826 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:07:23.413826 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:07:23.413826 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:23.413826 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:07:23.413826 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:23.413826 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:07:23.414881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:23.414881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:07:23.414881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:07:23.414881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:07:23.414881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:07:23.414881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:07:23.414881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:07:23.414881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:23.414881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:07:23.414881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:23.414881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:07:23.414881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:23.415425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:07:23.415425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:23.415425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:07:23.415425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:23.415425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:07:23.415425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:07:23.415425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:23.415425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:07:23.415425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:23.415425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:07:23.415425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:07:23.415425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:23.415425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:07:23.416022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:23.416022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:07:23.416336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:23.416336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:07:23.416336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:07:23.416336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:07:23.416336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:07:23.416336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:07:23.416336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:07:23.416336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:07:23.416336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:07:23.416336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:07:23.416336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:07:23.416336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:07:23.416336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:07:23.416336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:07:23.416336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:07:23.416336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:07:23.416848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:07:23.416848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:07:23.416848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:23.416848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:07:23.416848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:23.416848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:07:23.416848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:23.416848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:07:23.416848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:23.416848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:07:23.416848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:23.416848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:07:23.416848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:07:23.416848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:23.416848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:07:23.416848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:23.416848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:07:23.416848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:07:23.416848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:23.417399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:07:23.417399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:23.417399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:07:27.854450 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:07:27.855551 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:07:27.855551 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:07:27.856891 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:07:27.897935 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:07:27.898358 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:27.899460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:07:27.899460 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:27.899460 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:07:27.899460 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:07:27.899460 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:27.899460 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:07:27.899460 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:27.899460 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:07:27.900277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.044 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:27.900277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:07:27.900277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:07:27.900277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:07:27.900277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:07:27.900277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:07:27.900277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:07:27.900277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:27.900277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:07:27.900277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:27.900277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:07:27.900277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:27.901529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:07:27.901529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:27.901529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:07:27.901529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:27.901529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:07:27.901529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:07:27.901529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:27.901529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:07:27.901529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:27.901529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:07:27.901529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:07:27.901529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:27.901529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:07:27.902514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:27.902514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:07:27.902514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:27.902514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:07:27.902514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:07:27.902514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:07:27.902514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:07:27.902514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:07:27.902514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:07:27.902514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:07:27.902514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:07:27.902514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:07:27.902514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:07:27.902514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:07:27.902514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:07:27.903211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:07:27.903211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:07:27.903211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:07:27.903211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:07:27.903211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:07:27.903211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:27.903211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:07:27.903211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:27.903211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:07:27.903211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:27.903211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:07:27.903211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:27.903211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:07:27.903211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:27.903211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:07:27.903211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:07:27.903211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:27.903211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:07:27.903211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:27.904818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:07:27.904818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:07:27.904818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:27.904818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:07:27.904818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:27.904818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:07:28.369116 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:07:28.369950 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:07:28.370146 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:07:28.371505 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:07:28.371999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:28.373017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:07:28.373017 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:28.373017 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:07:28.373017 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:07:28.373017 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:28.373017 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:07:28.373017 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:28.373017 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:07:28.374782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:28.374782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:07:28.374782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:07:28.374782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:07:28.374782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:07:28.374782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:07:28.374782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:07:28.374782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:28.374782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:07:28.374782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:28.374782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:07:28.374782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:28.375326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:07:28.375326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:28.375326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:07:28.375326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:28.375326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:07:28.375326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:07:28.375326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:28.375326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:07:28.375326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:28.375326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:07:28.375326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:07:28.375326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:28.375326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:07:28.375935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:28.375935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:07:28.376517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:28.376517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:07:28.376517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:07:28.376517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:07:28.376517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:07:28.376517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:07:28.376517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:07:28.376517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:07:28.376517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:07:28.376517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:07:28.376517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:07:28.376517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:07:28.376517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:07:28.376517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:07:28.376517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:07:28.376517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:07:28.377450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:07:28.377450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:07:28.377450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:28.377450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:07:28.377450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:28.377450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:07:28.377450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:28.377450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:07:28.377450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:28.377450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:07:28.377450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:28.377450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:07:28.377450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:07:28.377450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:28.377450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:07:28.377450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:28.377450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:07:28.377450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:07:28.377450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:28.378527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:07:28.378527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:28.378527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:07:28.378527 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:07:28.378775 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:07:28.378876 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:07:28.379054 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:07:28.379636 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:07:28.380072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:28.380871 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:07:28.380871 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:28.380871 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:07:28.380871 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:07:28.380871 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:28.380871 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:07:28.380871 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:28.380871 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:07:28.381640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:28.381640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:07:28.381640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:07:28.381640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:07:28.381640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:28.381640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:07:28.381640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:28.381640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:07:28.381640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:28.381640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:07:28.381640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:28.381640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:07:28.381640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:28.381640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:07:28.381640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:07:28.382075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:28.382075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:07:28.382075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:28.382075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:07:28.382075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:07:28.382075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:28.382075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:07:28.382075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:28.382075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:07:28.382549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:28.382549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:07:28.382549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:07:28.382549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:07:28.382549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:07:28.382549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:07:28.382549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:07:28.382549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:07:28.382549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:07:28.382549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:07:28.382549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:28.382549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:07:28.383239 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:28.383239 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:07:28.383239 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:28.383239 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:07:28.383239 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:28.383239 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:07:28.383239 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:28.383239 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:07:28.383239 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:07:28.383239 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:28.383239 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:07:28.383239 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:28.383239 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:07:28.383239 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:07:28.384064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:28.384064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:07:28.384064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:28.384064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:07:28.384064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:28.384064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:07:28.384064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:07:28.384064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:07:28.384064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:07:28.384064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:07:28.384064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:07:28.384064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:07:28.384064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:07:28.384064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:07:28.384422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:07:28.384422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:07:28.384422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:07:28.384422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:07:28.384422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:07:28.384422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:07:28.384422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:07:28.384422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:07:28.384422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:07:28.384422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:07:28.384422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:07:28.384422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:28.384422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:07:28.384422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:28.384422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:07:28.384422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:28.384422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:07:28.384422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:28.384422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:07:28.384982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:28.384982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:07:28.384982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:07:28.384982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:28.384982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:07:28.384982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:28.384982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:07:28.384982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:07:28.384982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:28.384982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:07:28.384982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:28.384982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:07:28.403407 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:07:28.404022 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:07:28.404227 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:07:28.404379 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:07:28.405085 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:07:28.405435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:28.406331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:07:28.406331 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:28.406331 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:07:28.406331 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:07:28.406331 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:28.406331 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:07:28.406331 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:28.406331 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:07:28.407730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:28.407730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:07:28.407730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:07:28.407730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:07:28.407730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:28.407730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:07:28.407730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:28.407730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:07:28.407730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:28.407730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:07:28.407730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:28.407730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:07:28.407730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:28.407730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:07:28.407730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:07:28.409029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:28.409029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:07:28.409029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:28.409029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:07:28.409029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:07:28.409029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:28.409029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:07:28.409029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:28.409029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:07:28.409374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:28.409374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:07:28.409374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:07:28.409374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:07:28.409374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:07:28.409374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:07:28.409374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:07:28.409374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:07:28.409374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:07:28.409374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:28.409374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:07:28.409882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:28.409882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:07:28.409882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:28.409882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:07:28.409882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:28.409882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:07:28.409882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:28.409882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:07:28.409882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:07:28.409882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:28.409882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:07:28.409882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:28.409882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:07:28.409882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:07:28.410549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:28.410549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:07:28.410549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:28.410549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:07:28.411100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:28.411100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:07:28.411100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:07:28.411100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:07:28.411100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:07:28.411100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:07:28.411100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:07:28.411100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:07:28.411100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:07:28.411100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:07:28.411100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:07:28.411100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:07:28.411100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:07:28.411100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:07:28.411100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:07:28.411100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:07:28.411100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:07:28.412718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:07:28.412718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:07:28.412718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:07:28.412718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:28.412718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:07:28.412718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:28.412718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:07:28.412718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:28.412718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:07:28.412718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:28.412718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:07:28.412718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:28.412718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:07:28.412718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:07:28.412718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:28.412718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:07:28.412718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:28.412718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:07:28.412718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:07:28.413578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:28.413578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:07:28.413578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:28.413578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:07:28.413578 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:07:28.413578 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:07:28.413578 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:07:28.413578 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:07:28.413950 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:07:28.414298 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:07:28.414473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:28.415289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:07:28.415289 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:28.415289 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:07:28.415289 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:07:28.415289 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:28.415289 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:07:28.415289 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:28.415289 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:07:28.416157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:28.416157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:07:28.416157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:07:28.416157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:07:28.416157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:07:28.416157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:07:28.416157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:07:28.416157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:28.416157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:07:28.416157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:28.416157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:07:28.416157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:28.417030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:07:28.417030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:28.417030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:07:28.417030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:28.417030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:07:28.417030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:07:28.417030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:28.417030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:07:28.417030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:28.417030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:07:28.417030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:07:28.417030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:28.417030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:07:28.417772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:28.417772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:07:28.417772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:28.417772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:07:28.417772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:07:28.417772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:07:28.417772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:07:28.417772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:07:28.417772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:07:28.417772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:07:28.417772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:07:28.417772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:07:28.417772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:07:28.417772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:07:28.417772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:07:28.418333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:07:28.418333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:07:28.418333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:07:28.418333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:07:28.418333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:07:28.418333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:28.418333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:07:28.418333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:28.418333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:07:28.418333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:28.418333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:07:28.418333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:28.418333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:07:28.418333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:28.418333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:07:28.418333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:07:28.418333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:28.418333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:07:28.418333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:28.418803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:07:28.418803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:07:28.418803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:28.418803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:07:28.418803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:28.418803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:07:32.855228 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:07:32.856281 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:07:32.856281 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:07:32.857723 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:07:32.886769 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:07:32.887382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:32.888553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:07:32.888553 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:32.888553 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:07:32.888553 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:07:32.888553 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:32.888553 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:07:32.888553 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:32.888553 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:07:32.889771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.032 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:32.889771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:07:32.889771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:07:32.889771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:07:32.889771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:07:32.889771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:07:32.889771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:07:32.889771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:32.889771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:07:32.889771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:32.889771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:07:32.889771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:32.890454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:07:32.890454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:32.890454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:07:32.890454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:32.890454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:07:32.890454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:07:32.890454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:32.890454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:07:32.890454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:32.890454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:07:32.890454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:07:32.890454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:32.890454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:07:32.891038 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:32.891038 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:07:32.891038 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:32.891038 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:07:32.891038 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:07:32.891038 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:07:32.891038 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:07:32.891038 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:07:32.891038 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:07:32.891038 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:07:32.891038 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:07:32.891038 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:07:32.891038 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:07:32.891038 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:07:32.891038 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:07:32.892522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:07:32.892522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:07:32.892522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:07:32.892522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:07:32.892522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:07:32.892522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:32.892522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:07:32.892522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:32.892522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:07:32.892522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:32.892522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:07:32.892522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:32.892522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:07:32.892522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:32.892522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:07:32.892522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:07:32.892522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:32.892522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:07:32.892522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:32.893932 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:07:32.893932 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:07:32.893932 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:32.893932 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:07:32.893932 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:32.893932 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:07:33.370255 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:07:33.371190 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:07:33.371408 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:07:33.372515 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:07:33.372910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:33.373846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:07:33.373846 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:33.373846 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:07:33.373846 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:07:33.373846 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:33.373846 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:07:33.373846 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:33.373846 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:07:33.375784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:33.375784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:07:33.375784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:07:33.375784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:07:33.375784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:07:33.375784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:07:33.375784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:07:33.375784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:33.375784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:07:33.375784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:33.375784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:07:33.375784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:33.376330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:07:33.376330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:33.376330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:07:33.376330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:33.376330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:07:33.376330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:07:33.376330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:33.376330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:07:33.376330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:33.376330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:07:33.376330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:07:33.376330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:33.376330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:07:33.376858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:33.376858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:07:33.376858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:33.376858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:07:33.376858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:07:33.376858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:07:33.376858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:07:33.376858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:07:33.376858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:07:33.376858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:07:33.376858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:07:33.376858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:07:33.376858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:07:33.376858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:07:33.376858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:07:33.377482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:07:33.377482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:07:33.377482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:07:33.377482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:07:33.377482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:07:33.377482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:33.377482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:07:33.377482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:33.377482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:07:33.377482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:33.377482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:07:33.377482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:33.377482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:07:33.377482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:33.377482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:07:33.377482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:07:33.377482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:33.377482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:07:33.377482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:33.378360 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:07:33.378360 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:07:33.378360 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:33.378360 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:07:33.378360 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:33.378360 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:07:33.378635 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:07:33.378929 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:07:33.379094 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:07:33.379201 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:07:33.379681 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:07:33.379927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:33.381536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:07:33.381536 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:33.381536 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:07:33.381536 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:07:33.381536 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:33.381536 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:07:33.381536 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:33.381536 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:07:33.382438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:33.382438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:07:33.382438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:07:33.382438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:07:33.382438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:33.382438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:07:33.382438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:33.382438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:07:33.382438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:33.382438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:07:33.382438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:33.382438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:07:33.382438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:33.382438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:07:33.382438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:07:33.383830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:33.383830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:07:33.383830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:33.383830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:07:33.383830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:07:33.383830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:33.383830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:07:33.383830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:33.383830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:07:33.383830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:33.383830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:07:33.383830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:07:33.383830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:07:33.383830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:07:33.384207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:07:33.384207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:07:33.384207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:07:33.384207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:07:33.384207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:07:33.384207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:33.384207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:07:33.384207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:33.384207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:07:33.384207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:33.384207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:07:33.384207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:33.384207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:07:33.384529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:33.384529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:07:33.384529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:07:33.384529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:33.384529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:07:33.384529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:33.384529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:07:33.384529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:07:33.384529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:33.384529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:07:33.384529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:33.384529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:07:33.384838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:33.384838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:07:33.384838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:07:33.384838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:07:33.384838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:07:33.384838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:07:33.384838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:07:33.384838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:07:33.384838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:07:33.384838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:07:33.384838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:07:33.384838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:07:33.384838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:07:33.384838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:07:33.384838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:07:33.384838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:07:33.387003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:07:33.387003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:07:33.387003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:07:33.387003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:07:33.387003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:07:33.387003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:33.387003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:07:33.387003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:33.387003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:07:33.387003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:33.387003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:07:33.387003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:33.387003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:07:33.387003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:33.387003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:07:33.387003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:07:33.387003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:33.387003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:07:33.388077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:33.388077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:07:33.388077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:07:33.388077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:33.388077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:07:33.388077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:33.388077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:07:33.404068 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:07:33.404577 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:07:33.404841 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:07:33.405072 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:07:33.405627 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:07:33.406066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:33.406912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:07:33.406912 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:33.406912 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:07:33.406912 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:07:33.406912 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:33.406912 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:07:33.406912 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:33.406912 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:07:33.408289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:33.408289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:07:33.408289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:07:33.408289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:07:33.408289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:33.408289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:07:33.408289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:33.408289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:07:33.408289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:33.408289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:07:33.408289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:33.408289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:07:33.408289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:33.408289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:07:33.408289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:07:33.409990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:33.409990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:07:33.409990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:33.409990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:07:33.409990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:07:33.409990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:33.409990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:07:33.409990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:33.409990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:07:33.409990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:33.409990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:07:33.409990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:07:33.409990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:07:33.409990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:07:33.411164 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:07:33.411164 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:07:33.411164 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:07:33.411164 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:07:33.411164 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:33.411164 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:07:33.411164 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:33.411164 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:07:33.411164 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:33.411164 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:07:33.411164 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:33.411164 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:07:33.411164 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:33.411746 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:07:33.411746 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:07:33.411746 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:33.411746 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:07:33.411746 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:33.411746 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:07:33.411746 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:07:33.411746 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:33.411746 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:07:33.411746 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:33.411746 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:07:33.412221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:33.412221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:07:33.412221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:07:33.412221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:07:33.412221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:07:33.412221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:07:33.412221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:07:33.412221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:07:33.412221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:07:33.412221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:07:33.412221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:07:33.412221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:07:33.412221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:07:33.412221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:07:33.412221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:07:33.412221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:07:33.412221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:07:33.413280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:07:33.413280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:07:33.413280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:07:33.413280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:33.413280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:07:33.413280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:33.413280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:07:33.413280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:33.413280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:07:33.413280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:33.413280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:07:33.413280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:33.413280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:07:33.413280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:07:33.413280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:33.413280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:07:33.413280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:33.413280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:07:33.413280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:07:33.415621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:33.415621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:07:33.415621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:33.415621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:07:33.415621 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:07:33.415621 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:07:33.415621 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:07:33.415621 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:07:33.416482 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:07:33.416482 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:07:33.416482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:33.416482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:07:33.416482 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:33.416482 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:07:33.416482 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:07:33.416482 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:33.416482 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:07:33.416482 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:33.416482 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:07:33.417625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:33.417625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:07:33.417625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:07:33.417625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:07:33.417625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:07:33.417625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:07:33.417625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:07:33.417625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:33.417625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:07:33.417625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:33.417625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:07:33.417625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:33.418045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:07:33.418045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:33.418045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:07:33.418045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:33.418045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:07:33.418045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:07:33.418045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:33.418045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:07:33.418045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:33.418045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:07:33.418045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:07:33.418045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:33.418045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:07:33.418447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:33.418447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:07:33.418846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:33.418846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:07:33.418846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:07:33.418846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:07:33.418846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:07:33.418846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:07:33.418846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:07:33.418846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:07:33.418846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:07:33.418846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:07:33.418846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:07:33.418846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:07:33.418846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:07:33.418846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:07:33.418846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:07:33.418846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:07:33.419368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:07:33.419368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:07:33.419368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:33.419368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:07:33.419368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:33.419368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:07:33.419368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:33.419368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:07:33.419368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:33.419368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:07:33.419368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:33.419368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:07:33.419368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:07:33.419368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:33.419368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:07:33.419368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:33.419368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:07:33.419368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:07:33.419368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:33.420006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:07:33.420006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:33.420006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:07:37.855956 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:07:37.856895 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:07:37.857048 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:07:37.858388 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:07:37.881673 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG dbcounter [-] [106314] Writing DB stats neutron:SELECT=248 {{(pid=106314) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 09 09:07:37.888562 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:07:37.888999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:37.889944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:07:37.889944 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:37.889944 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:07:37.889944 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:07:37.889944 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:37.889944 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:07:37.889944 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:37.889944 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:07:37.890839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.033 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:37.890839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:07:37.890839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:07:37.890839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:07:37.890839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:07:37.890839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:07:37.890839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:07:37.890839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:37.890839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:07:37.890839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:37.890839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:07:37.890839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:37.891707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:07:37.891707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:37.891707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:07:37.891707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:37.891707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:07:37.891707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:07:37.891707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:37.891707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:07:37.891707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:37.891707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:07:37.891707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:07:37.891707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:37.891707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:07:37.893027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:37.893027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:07:37.893027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:37.893027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:07:37.893027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:07:37.893027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:07:37.893027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:07:37.893027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:07:37.893027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:07:37.893027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:07:37.893027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:07:37.893027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:07:37.893027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:07:37.893027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:07:37.893027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:07:37.894008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:07:37.894008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:07:37.894008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:07:37.894008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:07:37.894008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:07:37.894008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:37.894008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:07:37.894008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:37.894008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:07:37.894008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:37.894008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:07:37.894008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:37.894008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:07:37.894008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:37.894008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:07:37.894008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:07:37.894008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:37.894008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:07:37.894008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:37.894576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:07:37.894576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:07:37.894576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:37.894576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:07:37.894576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:37.894576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:07:38.371086 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:07:38.371687 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:07:38.371967 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:07:38.373103 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:07:38.373436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:38.374312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:07:38.374312 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:38.374312 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:07:38.374312 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:07:38.374312 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:38.374312 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:07:38.374312 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:38.374312 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:07:38.376059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:38.376059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:07:38.376059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:07:38.376059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:07:38.376059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:07:38.376059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:07:38.376059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:07:38.376059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:38.376059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:07:38.376059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:38.376059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:07:38.376059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:38.376691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:07:38.376691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:38.376691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:07:38.376691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:38.376691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:07:38.376691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:07:38.376691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:38.376691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:07:38.376691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:38.376691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:07:38.376691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:07:38.376691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:38.376691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:07:38.377381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:38.377381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:07:38.377796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:38.377796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:07:38.377796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:07:38.377796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:07:38.377796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:07:38.377796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:07:38.377796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:07:38.377796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:07:38.377796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:07:38.377796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:07:38.377796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:07:38.377796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:07:38.377796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:07:38.377796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:07:38.377796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:07:38.377796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:07:38.378586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:07:38.378586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:07:38.378586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:38.378586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:07:38.378586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:38.378586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:07:38.378586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:38.378586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:07:38.378586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:38.378586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:07:38.378586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:38.378586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:07:38.378586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:07:38.378586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:38.378586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:07:38.378586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:38.378586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:07:38.378586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:07:38.378586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:38.379487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:07:38.379487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:38.379487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:07:38.379487 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:07:38.379487 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:07:38.379815 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:07:38.379884 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:07:38.380317 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:07:38.380615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:38.381233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:07:38.381233 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:38.381233 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:07:38.381233 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:07:38.381233 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:38.381233 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:07:38.381233 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:38.381233 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:07:38.382569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:38.382569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:07:38.382569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:07:38.382569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:07:38.382569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:38.382569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:07:38.382569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:38.382569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:07:38.382569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:38.382569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:07:38.382569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:38.382569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:07:38.382569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:38.382569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:07:38.382569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:07:38.383079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:38.383079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:07:38.383079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:38.383079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:07:38.383079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:07:38.383079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:38.383079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:07:38.383079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:38.383079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:07:38.383880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:38.383880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:07:38.383880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:07:38.383880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:07:38.383880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:07:38.383880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:07:38.383880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:07:38.383880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:07:38.383880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:07:38.383880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:07:38.383880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:38.383880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:07:38.384275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:38.384275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:07:38.384275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:38.384275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:07:38.384275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:38.384275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:07:38.384275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:38.384275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:07:38.384275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:07:38.384275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:38.384275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:07:38.384275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:38.384275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:07:38.384275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:07:38.385037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:38.385037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:07:38.385037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:38.385037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:07:38.385037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:38.385037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:07:38.385037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:07:38.385037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:07:38.385037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:07:38.385037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:07:38.385037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:07:38.385037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:07:38.385037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:07:38.385037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:07:38.385440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:07:38.385440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:07:38.385440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:07:38.385440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:07:38.385440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:07:38.385440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:07:38.385440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:07:38.385440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:07:38.385440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:07:38.385440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:07:38.385440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:07:38.385440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:38.385440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:07:38.385440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:38.385440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:07:38.385440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:38.385440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:07:38.385440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:38.385440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:07:38.385980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:38.385980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:07:38.385980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:07:38.385980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:38.385980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:07:38.385980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:38.385980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:07:38.385980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:07:38.385980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:38.385980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:07:38.385980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:38.385980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:07:38.404598 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:07:38.405133 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:07:38.405391 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:07:38.405678 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:07:38.406371 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:07:38.406776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:38.407782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:07:38.407782 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:38.407782 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:07:38.407782 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:07:38.407782 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:38.407782 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:07:38.407782 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:38.407782 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:07:38.409085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:38.409085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:07:38.409085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:07:38.409085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:07:38.409085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:38.409085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:07:38.409085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:38.409085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:07:38.409085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:38.409085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:07:38.409085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:38.409085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:07:38.409085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:38.409085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:07:38.409085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:07:38.409816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:38.409816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:07:38.409816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:38.409816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:07:38.409816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:07:38.409816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:38.409816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:07:38.409816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:38.409816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:07:38.411548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:38.411548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:07:38.411548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:07:38.411548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:07:38.411548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:07:38.411548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:07:38.411548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:07:38.411548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:07:38.411548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:07:38.411548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:38.411548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:07:38.412190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:38.412190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:07:38.412190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:38.412190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:07:38.412190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:38.412190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:07:38.412190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:38.412190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:07:38.412190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:07:38.412190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:38.412190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:07:38.412190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:38.412190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:07:38.412190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:07:38.412890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:38.412890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:07:38.412890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:38.412890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:07:38.412890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:38.412890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:07:38.412890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:07:38.412890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:07:38.412890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:07:38.412890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:07:38.412890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:07:38.412890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:07:38.412890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:07:38.412890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:07:38.413725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:07:38.413725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:07:38.413725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:07:38.413725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:07:38.413725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:07:38.413725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:07:38.413725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:07:38.413725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:07:38.413725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:07:38.413725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:07:38.413725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:38.413725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:07:38.413725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:38.413725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:07:38.413725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:38.413725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:07:38.413725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:38.413725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:07:38.413725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:38.414731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:07:38.414731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:07:38.414731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:38.414731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:07:38.414731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:38.414731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:07:38.414731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:07:38.414731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:38.414731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:07:38.414731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:38.414731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:07:38.414731 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:07:38.414731 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:07:38.415647 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:07:38.415647 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:07:38.415647 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:07:38.415647 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:07:38.415647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:38.416148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:07:38.416148 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:38.416148 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:07:38.416148 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:07:38.416148 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:38.416148 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:07:38.416148 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:38.416148 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:07:38.417567 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:38.417567 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:07:38.417567 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:07:38.417567 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:07:38.417567 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:07:38.417567 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:07:38.417567 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:07:38.417567 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:38.417567 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:07:38.417567 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:38.417567 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:07:38.417567 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:38.417966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:07:38.417966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:38.417966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:07:38.417966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:38.417966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:07:38.417966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:07:38.417966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:38.417966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:07:38.417966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:38.417966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:07:38.417966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:07:38.417966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:38.417966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:07:38.418397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:38.418397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:07:38.419241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:38.419241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:07:38.419241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:07:38.419241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:07:38.419241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:07:38.419241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:07:38.419241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:07:38.419241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:07:38.419241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:07:38.419241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:07:38.419241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:07:38.419241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:07:38.419241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:07:38.419241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:07:38.419241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:07:38.419241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:07:38.419719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:07:38.419719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:07:38.419719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:38.419719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:07:38.419719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:38.419719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:07:38.419719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:38.419719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:07:38.419719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:38.419719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:07:38.419719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:38.419719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:07:38.419719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:07:38.419719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:38.419719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:07:38.419719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:38.419719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:07:38.419719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:07:38.419719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:38.420345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:07:38.420345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:38.420345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:07:42.856958 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:07:42.857952 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:07:42.857952 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:07:42.859095 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:07:42.895573 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:07:42.895941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:42.896892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:07:42.896892 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:42.896892 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:07:42.896892 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:07:42.896892 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:42.896892 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:07:42.896892 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:42.896892 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:07:42.898006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:42.898006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:07:42.898006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:07:42.898006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:07:42.898006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:07:42.898006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:07:42.898006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:07:42.898006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:42.898006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:07:42.898006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:42.898006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:07:42.898006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:42.898540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:07:42.898540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:42.898540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:07:42.898540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:42.898540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:07:42.898540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:07:42.898540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:42.898540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:07:42.898540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:42.898540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:07:42.898540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:07:42.898540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:42.898540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:07:42.899111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:42.899111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:07:42.899218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:42.899218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:07:42.899218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:07:42.899218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:07:42.899218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:07:42.899218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:07:42.899218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:07:42.899218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:07:42.899218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:07:42.899218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:07:42.899218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:07:42.899218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:07:42.899218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:07:42.899218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:07:42.899218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:07:42.899218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:07:42.899883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:07:42.899883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:07:42.899883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:42.899883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:07:42.899883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:42.899883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:07:42.899883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:42.899883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:07:42.899883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:42.899883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:07:42.899883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:42.899883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:07:42.899883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:07:42.899883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:42.899883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:07:42.899883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:42.899883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:07:42.899883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:07:42.899883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:42.900629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:07:42.900629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:42.900629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:07:43.372067 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:07:43.372776 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:07:43.372968 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:07:43.374352 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:07:43.374803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:43.375947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:07:43.375947 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:43.375947 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:07:43.375947 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:07:43.375947 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:43.375947 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:07:43.375947 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:43.375947 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:07:43.377617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:43.377617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:07:43.377617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:07:43.377617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:07:43.377617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:07:43.377617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:07:43.377617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:07:43.377617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:43.377617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:07:43.377617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:43.377617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:07:43.377617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:43.377940 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:07:43.377940 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:43.377940 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:07:43.377940 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:43.377940 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:07:43.377940 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:07:43.377940 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:43.377940 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:07:43.377940 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:43.377940 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:07:43.377940 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:07:43.377940 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:43.377940 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:07:43.378193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:43.378193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:07:43.379032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:43.379032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:07:43.379032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:07:43.379032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:07:43.379032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:07:43.379032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:07:43.379032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:07:43.379032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:07:43.379032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:07:43.379032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:07:43.379032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:07:43.379032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:07:43.379032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:07:43.379032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:07:43.379032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:07:43.379032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:07:43.379563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:07:43.379563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:07:43.379563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:43.379563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:07:43.379563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:43.379563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:07:43.379563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:43.379563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:07:43.379563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:43.379563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:07:43.379563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:43.379563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:07:43.379563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:07:43.379563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:43.379563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:07:43.379563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:43.379563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:07:43.379563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:07:43.379563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:43.379950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:07:43.379950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:43.379950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:07:43.379950 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:07:43.379950 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:07:43.380164 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:07:43.380164 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:07:43.380675 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:07:43.381024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:43.381787 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:07:43.381787 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:43.381787 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:07:43.381787 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:07:43.381787 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:43.381787 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:07:43.381787 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:43.381787 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:07:43.382695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:43.382695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:07:43.382695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:07:43.382695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:07:43.382695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:43.382695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:07:43.382695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:43.382695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:07:43.382695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:43.382695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:07:43.382695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:43.382695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:07:43.382695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:43.382695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:07:43.382695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:07:43.383051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:43.383051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:07:43.383051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:43.383051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:07:43.383051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:07:43.383051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:43.383051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:07:43.383051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:43.383051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:07:43.383590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:43.383590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:07:43.383590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:07:43.383590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:07:43.383590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:07:43.383590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:07:43.383590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:07:43.383590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:07:43.383590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:07:43.383590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:07:43.383590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:43.383590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:07:43.383968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:43.383968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:07:43.383968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:43.383968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:07:43.383968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:43.383968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:07:43.383968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:43.383968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:07:43.383968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:07:43.383968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:43.383968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:07:43.383968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:43.383968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:07:43.383968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:07:43.384298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:43.384298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:07:43.384298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:43.384298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:07:43.385652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:43.385652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:07:43.385652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:07:43.385652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:07:43.385652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:07:43.385652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:07:43.385652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:07:43.385652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:07:43.385652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:07:43.385652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:07:43.385652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:07:43.385652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:07:43.385652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:07:43.385652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:07:43.385652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:07:43.385652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:07:43.386119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:07:43.386119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:07:43.386119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:07:43.386119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:07:43.386119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:07:43.386119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:43.386119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:07:43.386119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:43.386119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:07:43.386119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:43.386119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:07:43.386119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:43.386119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:07:43.386119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:43.386119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:07:43.386119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:07:43.386119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:43.386119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:07:43.386577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:43.386577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:07:43.386577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:07:43.386577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:43.386577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:07:43.386577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:43.386577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:07:43.405216 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:07:43.405738 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:07:43.405962 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:07:43.406194 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:07:43.406740 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:07:43.407144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:43.407986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:07:43.407986 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:43.407986 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:07:43.407986 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:07:43.407986 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:43.407986 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:07:43.407986 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:43.407986 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:07:43.409245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:43.409245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:07:43.409245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:07:43.409245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:07:43.409245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:43.409245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:07:43.409245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:43.409245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:07:43.409245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:43.409245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:07:43.409245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:43.409245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:07:43.409245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:43.409245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:07:43.409245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:07:43.410150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:43.410150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:07:43.410150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:43.410150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:07:43.410150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:07:43.410150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:43.410150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:07:43.410150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:43.410150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:07:43.410676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:43.410676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:07:43.410676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:07:43.410676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:07:43.410676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:07:43.410676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:07:43.410676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:07:43.410676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:07:43.410676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:07:43.410676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:43.410676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:07:43.411279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:43.411279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:07:43.411279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:43.411279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:07:43.411279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:43.411279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:07:43.411279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:43.411279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:07:43.411279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:07:43.411279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:43.411279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:07:43.411279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:43.411279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:07:43.411279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:07:43.412003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:43.412003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:07:43.412003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:43.412003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:07:43.412003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:43.412003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:07:43.412003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:07:43.412003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:07:43.412003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:07:43.412003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:07:43.412003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:07:43.412003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:07:43.412003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:07:43.412003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:07:43.412694 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:07:43.412694 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:07:43.412694 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:07:43.412694 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:07:43.412694 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:07:43.412694 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:07:43.412694 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:07:43.412694 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:07:43.412694 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:07:43.412694 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:07:43.412694 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:43.412694 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:07:43.412694 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:43.412694 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:07:43.412694 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:43.412694 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:07:43.412694 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:43.412694 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:07:43.412694 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:43.413598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:07:43.413598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:07:43.413598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:43.413598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:07:43.413598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:43.413598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:07:43.413598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:07:43.413598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:43.413598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:07:43.413598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:43.413598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:07:43.414392 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:07:43.414663 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:07:43.414779 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:07:43.414937 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:07:43.415408 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:07:43.415821 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:07:43.416026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:43.416910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:07:43.416910 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:43.416910 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:07:43.416910 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:07:43.416910 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:43.416910 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:07:43.416910 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:43.416910 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:07:43.417759 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:43.417759 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:07:43.417759 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:07:43.417759 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:07:43.417759 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:07:43.417759 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:07:43.417759 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:07:43.417759 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:43.417759 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:07:43.417759 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:43.417759 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:07:43.417759 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:43.418174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:07:43.418174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:43.418174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:07:43.418174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:43.418174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:07:43.418174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:07:43.418174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:43.418174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:07:43.418174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:43.418174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:07:43.418174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:07:43.418174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:43.418174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:07:43.418544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:43.418544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:07:43.418761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:43.418761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:07:43.418761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:07:43.418761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:07:43.418761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:07:43.418761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:07:43.418761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:07:43.418761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:07:43.418761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:07:43.418761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:07:43.418761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:07:43.418761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:07:43.418761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:07:43.418761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:07:43.418761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:07:43.418761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:07:43.419279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:07:43.419279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:07:43.419279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:43.419279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:07:43.419279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:43.419279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:07:43.419279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:43.419279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:07:43.419279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:43.419279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:07:43.419279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:43.419279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:07:43.419279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:07:43.419279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:43.419279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:07:43.419279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:43.419279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:07:43.419279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:07:43.419279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:43.419834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:07:43.419834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:43.419834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:07:47.857797 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:07:47.858645 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:07:47.858911 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:07:47.860361 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:07:47.905977 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:07:47.910807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:47.910807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:07:47.910807 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:47.910807 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:07:47.910807 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:07:47.910807 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:47.910807 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:07:47.910807 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:47.910807 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:07:47.910807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.049 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:47.910807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:07:47.910807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:07:47.910807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:07:47.910807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:07:47.910807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:07:47.911804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:07:47.911804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:47.911804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:07:47.911804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:47.911804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:07:47.911804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:47.911804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:07:47.911804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:47.911804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:07:47.911804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:47.911804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:07:47.911804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:07:47.911804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:47.911804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:07:47.912768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:47.912768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:07:47.912768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:07:47.912768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:47.912768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:07:47.912768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:47.912768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:07:47.912768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:47.912768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:07:47.912768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:07:47.912768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:07:47.912768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:07:47.912768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:07:47.913605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:07:47.913605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:07:47.913605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:07:47.913605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:07:47.913605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:07:47.913605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:07:47.913605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:07:47.913605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:07:47.913605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:07:47.913605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:07:47.913605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:07:47.913605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:07:47.913605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:47.913605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:07:47.913605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:47.913605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:07:47.913605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:47.913605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:07:47.913605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:47.913605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:07:47.914174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:47.914174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:07:47.914174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:07:47.914174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:47.914174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:07:47.914174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:47.914174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:07:47.914174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:07:47.914174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:47.914174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:07:47.914174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:47.914174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:07:48.372847 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:07:48.373706 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:07:48.374078 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:07:48.375452 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:07:48.375862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:48.376955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:07:48.376955 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:48.376955 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:07:48.376955 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:07:48.376955 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:48.376955 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:07:48.376955 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:48.376955 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:07:48.378702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:48.378702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:07:48.378702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:07:48.378702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:07:48.378702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:07:48.378702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:07:48.378702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:07:48.378702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:48.378702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:07:48.378702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:48.378702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:07:48.378702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:48.379244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:07:48.379244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:48.379244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:07:48.379244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:48.379244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:07:48.379244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:07:48.379244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:48.379244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:07:48.379244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:48.379244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:07:48.379244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:07:48.379244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:48.379244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:07:48.379767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:48.379767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:07:48.381401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:48.381401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:07:48.381401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:07:48.381401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:07:48.381401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:07:48.381401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:07:48.381401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:07:48.381401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:07:48.381401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:07:48.381401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:07:48.381401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:07:48.381401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:07:48.381401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:07:48.381401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:07:48.381401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:07:48.381401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:07:48.382116 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:07:48.382116 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:07:48.382116 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:48.382116 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:07:48.382116 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:48.382116 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:07:48.382116 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:48.382116 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:07:48.382116 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:48.382116 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:07:48.382116 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:48.382116 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:07:48.382116 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:07:48.382116 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:48.382116 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:07:48.382116 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:48.382116 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:07:48.382116 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:07:48.382116 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:48.382700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:07:48.382700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:48.382700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:07:48.383464 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:07:48.383878 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:07:48.384043 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:07:48.384190 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:07:48.384736 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:07:48.385089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:48.385966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:07:48.385966 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:48.385966 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:07:48.385966 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:07:48.385966 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:48.385966 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:07:48.385966 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:48.385966 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:07:48.386831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:48.386831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:07:48.386831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:07:48.386831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:07:48.386831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:48.386831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:07:48.386831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:48.386831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:07:48.386831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:48.386831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:07:48.386831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:48.386831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:07:48.386831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:48.386831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:07:48.386831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:07:48.387235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:48.387235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:07:48.387235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:48.387235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:07:48.387235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:07:48.387235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:48.387235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:07:48.387235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:48.387235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:07:48.387672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:48.387672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:07:48.387672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:07:48.387672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:07:48.387672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:07:48.387672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:07:48.387672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:07:48.387672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:07:48.387672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:07:48.387672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:07:48.387672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:48.387672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:07:48.388025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:48.388025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:07:48.388025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:48.388025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:07:48.388025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:48.388025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:07:48.388025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:48.388025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:07:48.388025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:07:48.388025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:48.388025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:07:48.388025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:48.388025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:07:48.388025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:07:48.388380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:48.388380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:07:48.388380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:48.388380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:07:48.388727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:48.388727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:07:48.388727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:07:48.388727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:07:48.388727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:07:48.388727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:07:48.388727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:07:48.388727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:07:48.388727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:07:48.388727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:07:48.388727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:07:48.388727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:07:48.388727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:07:48.388727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:07:48.388727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:07:48.388727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:07:48.389208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:07:48.389208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:07:48.389208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:07:48.389208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:07:48.389208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:07:48.389208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:48.389208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:07:48.389208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:48.389208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:07:48.389208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:48.389208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:07:48.389208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:48.389208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:07:48.389208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:48.389208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:07:48.389208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:07:48.389208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:48.389208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:07:48.389680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:48.389680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:07:48.389680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:07:48.389680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:48.389680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:07:48.389680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:48.389680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:07:48.405717 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:07:48.406396 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:07:48.406542 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:07:48.406711 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:07:48.407310 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:07:48.407669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:48.408602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:07:48.408602 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:48.408602 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:07:48.408602 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:07:48.408602 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:48.408602 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:07:48.408602 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:48.408602 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:07:48.410129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:48.410129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:07:48.410129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:07:48.410129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:07:48.410129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:48.410129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:07:48.410129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:48.410129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:07:48.410129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:48.410129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:07:48.410129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:48.410129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:07:48.410129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:48.410129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:07:48.410129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:07:48.410848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:48.410848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:07:48.410848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:48.410848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:07:48.410848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:07:48.410848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:48.410848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:07:48.410848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:48.410848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:07:48.411253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:48.411253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:07:48.411253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:07:48.411253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:07:48.411253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:07:48.411253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:07:48.411253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:07:48.411253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:07:48.411253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:07:48.411253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:48.411253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:07:48.411743 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:48.411743 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:07:48.411743 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:48.411743 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:07:48.411743 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:48.411743 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:07:48.411743 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:48.411743 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:07:48.411743 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:07:48.411743 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:48.411743 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:07:48.411743 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:48.411743 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:07:48.411743 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:07:48.412350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:48.412350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:07:48.412350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:48.412350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:07:48.412350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:48.412350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:07:48.412350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:07:48.412350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:07:48.412350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:07:48.412350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:07:48.412350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:07:48.412350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:07:48.412350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:07:48.412350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:07:48.412977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:07:48.412977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:07:48.412977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:07:48.412977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:07:48.412977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:07:48.412977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:07:48.412977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:07:48.412977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:07:48.412977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:07:48.412977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:07:48.412977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:48.412977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:07:48.412977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:48.412977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:07:48.412977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:48.412977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:07:48.412977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:48.412977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:07:48.412977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:48.413838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:07:48.413838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:07:48.413838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:48.413838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:07:48.413838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:48.413838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:07:48.413838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:07:48.413838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:48.413838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:07:48.413838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:48.413838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:07:48.414827 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:07:48.415150 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:07:48.415257 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:07:48.415420 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:07:48.415977 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:07:48.416303 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:07:48.416493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:48.417258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:07:48.417258 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:48.417258 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:07:48.417258 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:07:48.417258 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:48.417258 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:07:48.417258 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:48.417258 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:07:48.418192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:48.418192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:07:48.418192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:07:48.418192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:07:48.418192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:07:48.418192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:07:48.418192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:07:48.418192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:48.418192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:07:48.418192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:48.418192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:07:48.418192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:48.418542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:07:48.418542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:48.418542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:07:48.418542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:48.418542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:07:48.418542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:07:48.418542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:48.418542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:07:48.418542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:48.418542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:07:48.418542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:07:48.418542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:48.418542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:07:48.418929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:48.418929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:07:48.419214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:48.419214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:07:48.419214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:07:48.419214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:07:48.419214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:07:48.419214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:07:48.419214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:07:48.419214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:07:48.419214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:07:48.419214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:07:48.419214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:07:48.419214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:07:48.419214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:07:48.419214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:07:48.419214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:07:48.419214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:07:48.419655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:07:48.419655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:07:48.419655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:48.419655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:07:48.419655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:48.419655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:07:48.419655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:48.419655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:07:48.419655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:48.419655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:07:48.419655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:48.419655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:07:48.419655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:07:48.419655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:48.419655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:07:48.419655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:48.419655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:07:48.419655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:07:48.419655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:48.420202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:07:48.420202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:48.420202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:07:52.858505 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:07:52.861250 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:07:52.861250 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:07:52.861250 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:07:52.889829 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:07:52.890105 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:52.891834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:07:52.891834 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:52.891834 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:07:52.891834 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:07:52.891834 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:52.891834 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:07:52.891834 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:52.891834 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:07:52.892285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.031 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:52.892285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:07:52.892285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:07:52.892285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:07:52.892285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:07:52.892285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:07:52.892285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:07:52.892285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:52.892285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:07:52.892285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:52.892285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:07:52.892285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:52.892810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:07:52.892810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:52.892810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:07:52.892810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:52.892810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:07:52.892810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:07:52.892810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:52.892810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:07:52.892810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:52.892810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:07:52.892810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:07:52.892810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:52.892810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:07:52.893238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:52.893238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:07:52.895464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:52.895464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:07:52.895464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:07:52.895464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:07:52.895464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:07:52.895464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:07:52.895464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:07:52.895464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:07:52.895464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:07:52.895464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:07:52.895464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:07:52.895464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:07:52.895464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:07:52.895464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:07:52.895464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:07:52.895464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:07:52.896102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:07:52.896102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:07:52.896102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:52.896102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:07:52.896102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:52.896102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:07:52.896102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:52.896102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:07:52.896102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:52.896102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:07:52.896102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:52.896102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:07:52.896102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:07:52.896102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:52.896102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:07:52.896102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:52.896102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:07:52.896102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:07:52.896102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:52.896771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:07:52.896771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:52.896771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:07:53.373521 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:07:53.374447 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:07:53.374873 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:07:53.376160 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:07:53.376512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:53.377417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:07:53.377417 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:53.377417 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:07:53.377417 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:07:53.377417 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:53.377417 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:07:53.377417 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:53.377417 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:07:53.379175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:53.379175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:07:53.379175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:07:53.379175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:07:53.379175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:07:53.379175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:07:53.379175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:07:53.379175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:53.379175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:07:53.379175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:53.379175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:07:53.379175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:53.379905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:07:53.379905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:53.379905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:07:53.379905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:53.379905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:07:53.379905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:07:53.379905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:53.379905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:07:53.379905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:53.379905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:07:53.379905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:07:53.379905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:53.379905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:07:53.380537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:53.380537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:07:53.381534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:53.381534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:07:53.381534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:07:53.381534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:07:53.381534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:07:53.381534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:07:53.381534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:07:53.381534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:07:53.381534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:07:53.381534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:07:53.381534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:07:53.381534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:07:53.381534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:07:53.381534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:07:53.381534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:07:53.381534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:07:53.382827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:07:53.382827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:07:53.382827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:53.382827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:07:53.382827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:53.382827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:07:53.382827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:53.382827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:07:53.382827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:53.382827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:07:53.382827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:53.382827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:07:53.382827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:07:53.382827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:53.382827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:07:53.382827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:53.382827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:07:53.382827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:07:53.382827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:53.383657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:07:53.383657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:53.383657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:07:53.383960 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:07:53.384540 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:07:53.384895 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:07:53.385337 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:07:53.386211 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:07:53.386672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:53.388126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:07:53.388126 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:53.388126 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:07:53.388126 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:07:53.388126 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:53.388126 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:07:53.388126 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:53.388126 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:07:53.388974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:53.388974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:07:53.388974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:07:53.388974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:07:53.388974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:53.388974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:07:53.388974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:53.388974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:07:53.388974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:53.388974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:07:53.388974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:53.388974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:07:53.388974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:53.388974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:07:53.388974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:07:53.389639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:53.389639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:07:53.389639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:53.389639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:07:53.389639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:07:53.389639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:53.389639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:07:53.389639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:53.389639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:07:53.389901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:53.389901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:07:53.389901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:07:53.389901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:07:53.389901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:07:53.389901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:07:53.389901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:07:53.389901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:07:53.389901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:07:53.389901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:07:53.389901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:53.389901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:07:53.390220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:53.390220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:07:53.390220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:53.390220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:07:53.390220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:53.390220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:07:53.390220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:53.390220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:07:53.390220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:07:53.390220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:53.390220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:07:53.390220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:53.390220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:07:53.390220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:07:53.390612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:53.390612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:07:53.390612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:53.390612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:07:53.391459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:53.391459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:07:53.391459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:07:53.391459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:07:53.391459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:07:53.391459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:07:53.391459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:07:53.391459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:07:53.391459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:07:53.391459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:07:53.391459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:07:53.391459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:07:53.391459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:07:53.391459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:07:53.391459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:07:53.391459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:07:53.392095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:07:53.392095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:07:53.392095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:07:53.392095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:07:53.392095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:07:53.392095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:53.392095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:07:53.392095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:53.392095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:07:53.392095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:53.392095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:07:53.392095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:53.392095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:07:53.392095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:53.392095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:07:53.392095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:07:53.392095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:53.392095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:07:53.392922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:53.392922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:07:53.392922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:07:53.392922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:53.392922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:07:53.392922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:53.392922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:07:53.406259 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:07:53.406633 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:07:53.406789 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:07:53.406929 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:07:53.407683 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:07:53.408201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:53.409105 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:07:53.409105 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:53.409105 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:07:53.409105 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:07:53.409105 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:53.409105 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:07:53.409105 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:53.409105 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:07:53.409991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:53.409991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:07:53.409991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:07:53.409991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:07:53.409991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:53.409991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:07:53.409991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:53.409991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:07:53.409991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:53.409991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:07:53.409991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:53.409991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:07:53.409991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:53.409991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:07:53.409991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:07:53.410698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:53.410698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:07:53.410698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:53.410698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:07:53.410698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:07:53.410698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:53.410698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:07:53.410698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:53.410698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:07:53.411182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:53.411182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:07:53.411182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:07:53.411182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:07:53.411182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:07:53.411182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:07:53.411182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:07:53.411182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:07:53.411182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:07:53.411182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:53.411182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:07:53.411724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:53.411724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:07:53.411724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:53.411724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:07:53.411724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:53.411724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:07:53.411724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:53.411724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:07:53.411724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:07:53.411724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:53.411724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:07:53.411724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:53.411724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:07:53.411724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:07:53.412406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:53.412406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:07:53.412406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:53.412406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:07:53.412406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:53.412406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:07:53.412406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:07:53.412406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:07:53.412406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:07:53.412406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:07:53.412406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:07:53.412406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:07:53.412406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:07:53.412406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:07:53.413182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:07:53.413182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:07:53.413182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:07:53.413182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:07:53.413182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:07:53.413182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:07:53.413182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:07:53.413182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:07:53.413182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:07:53.413182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:07:53.413182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:53.413182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:07:53.413182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:53.413182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:07:53.413182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:53.413182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:07:53.413182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:53.413182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:07:53.413182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:53.414298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:07:53.414298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:07:53.414298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:53.414298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:07:53.414298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:53.414298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:07:53.414298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:07:53.414298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:53.414298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:07:53.414298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:53.414298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:07:53.415611 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:07:53.415869 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:07:53.415945 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:07:53.416115 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:07:53.416638 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:07:53.417062 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:07:53.417281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:53.418096 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:07:53.418096 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:53.418096 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:07:53.418096 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:07:53.418096 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:53.418096 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:07:53.418096 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:53.418096 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:07:53.419038 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:53.419038 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:07:53.419038 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:07:53.419038 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:07:53.419038 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:07:53.419038 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:07:53.419038 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:07:53.419038 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:53.419038 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:07:53.419038 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:53.419038 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:07:53.419038 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:53.419403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:07:53.419403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:53.419403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:07:53.419403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:53.419403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:07:53.419403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:07:53.419403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:53.419403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:07:53.419403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:53.419403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:07:53.419403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:07:53.419403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:53.419403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:07:53.419779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:53.419779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:07:53.419998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:53.419998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:07:53.419998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:07:53.419998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:07:53.419998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:07:53.419998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:07:53.419998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:07:53.419998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:07:53.419998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:07:53.419998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:07:53.419998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:07:53.419998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:07:53.419998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:07:53.419998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:07:53.419998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:07:53.419998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:07:53.420506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:07:53.420506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:07:53.420506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:53.420506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:07:53.420506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:53.420506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:07:53.420506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:53.420506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:07:53.420506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:53.420506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:07:53.420506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:53.420506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:07:53.420506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:07:53.420506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:53.420506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:07:53.420506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:53.420506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:07:53.420506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:07:53.420506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:53.421079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:07:53.421079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:53.421079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:07:57.859948 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:07:57.861136 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:07:57.861136 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:07:57.862202 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:07:57.916242 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:07:57.916683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:57.917802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:07:57.917802 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:57.917802 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:07:57.917802 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:07:57.917802 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:57.917802 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:07:57.917802 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:57.917802 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:07:57.918805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.057 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:57.918805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:07:57.918805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:07:57.918805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:07:57.918805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:07:57.918805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:07:57.918805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:07:57.918805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:57.918805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:07:57.918805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:57.918805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:07:57.918805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:57.919488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:07:57.919488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:57.919488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:07:57.919488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:57.919488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:07:57.919488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:07:57.919488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:57.919488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:07:57.919488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:57.919488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:07:57.919488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:07:57.919488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:57.919488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:07:57.920160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:57.920160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:07:57.920160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:57.920160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:07:57.920160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:07:57.920160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:07:57.920160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:07:57.920160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:07:57.920160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:07:57.920160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:07:57.920160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:07:57.920160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:07:57.920160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:07:57.920160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:07:57.920160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:07:57.920816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:07:57.920816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:07:57.920816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:07:57.920816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:07:57.920816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:07:57.920816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:57.920816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:07:57.920816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:57.920816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:07:57.920816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:57.920816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:07:57.920816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:57.920816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:07:57.920816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:57.920816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:07:57.920816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:07:57.920816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:57.920816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:07:57.920816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:57.922531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:07:57.922531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:07:57.922531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:57.922531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:07:57.922531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:57.922531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:07:58.374655 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:07:58.375563 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:07:58.375845 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:07:58.377298 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:07:58.377724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:58.378921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:07:58.378921 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:58.378921 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:07:58.378921 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:07:58.378921 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:58.378921 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:07:58.378921 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:58.378921 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:07:58.380638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:58.380638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:07:58.380638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:07:58.380638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:07:58.380638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:07:58.380638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:07:58.380638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:07:58.380638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:58.380638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:07:58.380638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:58.380638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:07:58.380638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:58.381337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:07:58.381337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:58.381337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:07:58.381337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:58.381337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:07:58.381337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:07:58.381337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:58.381337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:07:58.381337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:58.381337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:07:58.381337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:07:58.381337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:58.381337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:07:58.381975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:58.381975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:07:58.382717 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:58.382717 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:07:58.382717 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:07:58.382717 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:07:58.382717 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:07:58.382717 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:07:58.382717 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:07:58.382717 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:07:58.382717 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:07:58.382717 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:07:58.382717 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:07:58.382717 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:07:58.382717 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:07:58.382717 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:07:58.382717 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:07:58.382717 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:07:58.383549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:07:58.383549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:07:58.383549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:58.383549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:07:58.383549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:58.383549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:07:58.383549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:58.383549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:07:58.383549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:58.383549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:07:58.383549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:58.383549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:07:58.383549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:07:58.383549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:58.383549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:07:58.383549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:58.383549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:07:58.383549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:07:58.383549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:58.384452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:07:58.384452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:58.384452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:07:58.384764 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:07:58.385335 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:07:58.385517 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:07:58.385707 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:07:58.387369 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:07:58.387781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:58.388798 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:07:58.388798 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:58.388798 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:07:58.388798 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:07:58.388798 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:58.388798 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:07:58.388798 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:58.388798 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:07:58.389653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:58.389653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:07:58.389653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:07:58.389653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:07:58.389653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:58.389653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:07:58.389653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:58.389653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:07:58.389653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:58.389653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:07:58.389653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:58.389653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:07:58.389653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:58.389653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:07:58.389653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:07:58.390094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:58.390094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:07:58.390094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:58.390094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:07:58.390094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:07:58.390094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:58.390094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:07:58.390094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:58.390094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:07:58.390575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:58.390575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:07:58.390575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:07:58.390575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:07:58.390575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:07:58.390575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:07:58.390575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:07:58.390575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:07:58.390575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:07:58.390575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:07:58.390575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:58.390575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:07:58.390947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:58.390947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:07:58.390947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:58.390947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:07:58.390947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:58.390947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:07:58.390947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:58.390947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:07:58.390947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:07:58.390947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:58.390947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:07:58.390947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:58.390947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:07:58.390947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:07:58.391413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:58.391413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:07:58.391413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:58.391413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:07:58.391769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:58.391769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:07:58.391769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:07:58.391769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:07:58.391769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:07:58.391769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:07:58.391769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:07:58.391769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:07:58.391769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:07:58.391769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:07:58.391769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:07:58.391769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:07:58.391769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:07:58.391769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:07:58.391769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:07:58.391769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:07:58.392227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:07:58.392227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:07:58.392227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:07:58.392227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:07:58.392227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:07:58.392227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:58.392227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:07:58.392227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:58.392227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:07:58.392227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:58.392227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:07:58.392227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:58.392227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:07:58.392227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:58.392227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:07:58.392227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:07:58.392227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:58.392227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:07:58.392753 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:58.392753 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:07:58.392753 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:07:58.392753 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:58.392753 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:07:58.392753 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:58.392753 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:07:58.406888 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:07:58.407382 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:07:58.407487 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:07:58.407637 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:07:58.408270 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:07:58.408659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:58.409531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:07:58.409531 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:58.409531 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:07:58.409531 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:07:58.409531 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:58.409531 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:07:58.409531 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:58.409531 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:07:58.410785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:58.410785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:07:58.410785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:07:58.410785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:07:58.410785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:58.410785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:07:58.410785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:58.410785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:07:58.410785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:58.410785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:07:58.410785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:58.410785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:07:58.410785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:58.410785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:07:58.410785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:07:58.411581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:58.411581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:07:58.411581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:58.411581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:07:58.411581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:07:58.411581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:58.411581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:07:58.411581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:58.411581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:07:58.412245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:58.412245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:07:58.412245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:07:58.412245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:07:58.412245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:07:58.412245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:07:58.412245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:07:58.412245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:07:58.412245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:07:58.412245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:58.412245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:07:58.412793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:58.412793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:07:58.412793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:58.412793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:07:58.412793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:58.412793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:07:58.412793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:58.412793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:07:58.412793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:07:58.412793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:58.412793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:07:58.412793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:58.412793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:07:58.412793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:07:58.413425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:58.413425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:07:58.413425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:58.413425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:07:58.414057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:58.414057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:07:58.414057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:07:58.414057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:07:58.414057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:07:58.414057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:07:58.414057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:07:58.414057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:07:58.414057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:07:58.414057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:07:58.414057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:07:58.414057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:07:58.414057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:07:58.414057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:07:58.414057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:07:58.414057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:07:58.414057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:07:58.414883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:07:58.414883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:07:58.414883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:07:58.414883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:58.414883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:07:58.414883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:58.414883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:07:58.414883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:58.414883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:07:58.414883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:58.414883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:07:58.414883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:58.414883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:07:58.414883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:07:58.414883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:58.414883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:07:58.414883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:58.414883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:07:58.414883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:07:58.415724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:58.415724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:07:58.415724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:58.415724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:07:58.415937 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:07:58.416303 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:07:58.416466 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:07:58.416697 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:07:58.417090 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:07:58.417359 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:07:58.417555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:58.418235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:07:58.418235 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:58.418235 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:07:58.418235 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:07:58.418235 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:58.418235 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:07:58.418235 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:58.418235 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:07:58.419219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:58.419219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:07:58.419219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:07:58.419219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:07:58.419219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:07:58.419219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:07:58.419219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:07:58.419219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:58.419219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:07:58.419219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:58.419219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:07:58.419219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:58.419860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:07:58.419860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:58.419860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:07:58.419860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:58.419860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:07:58.419860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:07:58.419860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:58.419860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:07:58.419860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:58.419860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:07:58.419860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:07:58.419860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:58.419860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:07:58.420447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:58.420447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:07:58.420447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:58.420447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:07:58.420447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:07:58.420447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:07:58.420447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:07:58.420447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:07:58.420447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:07:58.420447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:07:58.420447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:07:58.420447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:07:58.420447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:07:58.420447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:07:58.420447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:07:58.421144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:07:58.421144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:07:58.421144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:07:58.421144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:07:58.421144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:07:58.421144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:58.421144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:07:58.421144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:07:58.421144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:07:58.421144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:07:58.421144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:07:58.421144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:07:58.421144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:07:58.421144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:07:58.421144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:07:58.421144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:07:58.421144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:07:58.421144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:07:58.421144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:07:58.422080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:07:58.422080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:07:58.422080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:07:58.422080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:07:58.422080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:07:58.422080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:08:02.860771 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:08:02.861673 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:08:02.861934 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:08:02.863327 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:08:02.901199 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:08:02.901612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:02.902776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:08:02.902776 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:02.902776 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:08:02.902776 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:08:02.902776 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:02.902776 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:08:02.902776 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:02.902776 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:08:02.903761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:02.903761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:08:02.903761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:08:02.903761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:08:02.903761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:08:02.903761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:08:02.903761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:08:02.903761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:02.903761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:08:02.903761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:02.903761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:08:02.903761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:02.904336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:08:02.904336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:02.904336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:08:02.904336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:02.904336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:08:02.904336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:08:02.904336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:02.904336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:08:02.904336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:02.904336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:08:02.904336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:08:02.904336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:02.904336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:08:02.905624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:02.905624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:08:02.905624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:02.905624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:08:02.905624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:08:02.905624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:08:02.905624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:08:02.905624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:08:02.905624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:08:02.905624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:08:02.905624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:08:02.905624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:08:02.905624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:08:02.905624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:08:02.905624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:08:02.906624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:08:02.906624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:08:02.906624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:08:02.906624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:08:02.906624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:08:02.906624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:02.906624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:08:02.906624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:02.906624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:08:02.906624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:02.906624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:08:02.906624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:02.906624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:08:02.906624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:02.906624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:08:02.906624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:08:02.906624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:02.906624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:08:02.906624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:02.907452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:08:02.907452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:08:02.907452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:02.907452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:08:02.907452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:02.907452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:08:03.375589 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:08:03.376400 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:08:03.376591 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:08:03.378057 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:08:03.378493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:03.379597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:08:03.379597 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:03.379597 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:08:03.379597 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:08:03.379597 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:03.379597 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:08:03.379597 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:03.379597 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:08:03.381511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:03.381511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:08:03.381511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:08:03.381511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:08:03.381511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:08:03.381511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:08:03.381511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:08:03.381511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:03.381511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:08:03.381511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:03.381511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:08:03.381511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:03.382193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:08:03.382193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:03.382193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:08:03.382193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:03.382193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:08:03.382193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:08:03.382193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:03.382193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:08:03.382193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:03.382193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:08:03.382193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:08:03.382193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:03.382193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:08:03.382767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:03.382767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:08:03.383654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:03.383654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:08:03.383654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:08:03.383654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:08:03.383654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:08:03.383654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:08:03.383654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:08:03.383654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:08:03.383654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:08:03.383654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:08:03.383654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:08:03.383654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:08:03.383654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:08:03.383654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:08:03.383654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:08:03.383654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:08:03.385565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:08:03.385565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:08:03.385565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:03.385565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:08:03.385565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:03.385565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:08:03.385565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:03.385565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:08:03.385565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:03.385565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:08:03.385565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:03.385565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:08:03.385565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:08:03.385565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:03.385565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:08:03.385565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:03.385565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:08:03.385565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:08:03.385565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:03.386492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:08:03.386492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:03.386492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:08:03.386492 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:08:03.386492 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:08:03.386492 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:08:03.386492 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:08:03.386874 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:08:03.387067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:03.387631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:08:03.387631 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:03.387631 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:08:03.387631 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:08:03.387631 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:03.387631 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:08:03.387631 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:03.387631 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:08:03.388416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:03.388416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:08:03.388416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:08:03.388416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:08:03.388416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:03.388416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:08:03.388416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:03.388416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:08:03.388416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:03.388416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:08:03.388416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:03.388416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:08:03.388416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:03.388416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:08:03.388416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:08:03.389005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:03.389005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:08:03.389005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:03.389005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:08:03.389005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:08:03.389005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:03.389005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:08:03.389005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:03.389005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:08:03.389338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:03.389338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:08:03.389338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:08:03.389338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:08:03.389338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:08:03.389338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:08:03.389338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:08:03.389338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:08:03.389338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:08:03.389338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:08:03.389338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:03.389338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:08:03.389732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:03.389732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:08:03.389732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:03.389732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:08:03.389732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:03.389732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:08:03.389732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:03.389732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:08:03.389732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:08:03.389732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:03.389732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:08:03.389732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:03.389732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:08:03.389732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:08:03.390181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:03.390181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:08:03.390181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:03.390181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:08:03.390310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:03.390310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:08:03.390310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:08:03.390310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:08:03.390310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:08:03.390310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:08:03.390310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:08:03.390310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:08:03.390310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:08:03.390310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:08:03.390310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:08:03.390310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:08:03.390310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:08:03.390310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:08:03.390310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:08:03.390310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:08:03.390841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:08:03.390841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:08:03.390841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:08:03.390841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:08:03.390841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:08:03.390841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:03.390841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:08:03.390841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:03.390841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:08:03.390841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:03.390841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:08:03.390841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:03.390841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:08:03.390841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:03.390841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:08:03.390841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:08:03.390841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:03.390841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:08:03.391419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:03.391419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:08:03.391419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:08:03.391419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:03.391419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:08:03.391419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:03.391419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:08:03.407591 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:08:03.407936 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:08:03.408021 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:08:03.408187 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:08:03.408725 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:08:03.409175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:03.410041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:08:03.410041 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:03.410041 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:08:03.410041 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:08:03.410041 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:03.410041 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:08:03.410041 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:03.410041 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:08:03.411246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:03.411246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:08:03.411246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:08:03.411246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:08:03.411246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:03.411246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:08:03.411246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:03.411246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:08:03.411246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:03.411246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:08:03.411246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:03.411246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:08:03.411246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:03.411246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:08:03.411246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:08:03.412246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:03.412246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:08:03.412246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:03.412246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:08:03.412246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:08:03.412246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:03.412246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:08:03.412246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:03.412246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:08:03.412679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:03.412679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:08:03.412679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:08:03.412679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:08:03.412679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:08:03.412679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:08:03.412679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:08:03.412679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:08:03.412679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:08:03.412679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:03.412679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:08:03.413281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:03.413281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:08:03.413281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:03.413281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:08:03.413281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:03.413281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:08:03.413281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:03.413281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:08:03.413281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:08:03.413281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:03.413281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:08:03.413281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:03.413281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:08:03.413281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:08:03.413951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:03.413951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:08:03.413951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:03.413951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:08:03.414338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:03.414338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:08:03.414338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:08:03.414338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:08:03.414338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:08:03.414338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:08:03.414338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:08:03.414338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:08:03.414338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:08:03.414338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:08:03.414338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:08:03.414338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:08:03.414338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:08:03.414338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:08:03.414338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:08:03.414338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:08:03.414338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:08:03.415216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:08:03.415216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:08:03.415216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:08:03.415216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:03.415216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:08:03.415216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:03.415216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:08:03.415216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:03.415216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:08:03.415216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:03.415216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:08:03.415216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:03.415216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:08:03.415216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:08:03.415216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:03.415216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:08:03.415216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:03.415216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:08:03.415216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:08:03.416237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:03.416237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:08:03.416237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:03.416237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:08:03.416456 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:08:03.416956 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:08:03.417106 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:08:03.417350 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:08:03.417853 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:08:03.418232 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:08:03.418435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:03.419188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:08:03.419188 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:03.419188 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:08:03.419188 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:08:03.419188 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:03.419188 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:08:03.419188 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:03.419188 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:08:03.420076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:03.420076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:08:03.420076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:08:03.420076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:08:03.420076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:08:03.420076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:08:03.420076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:08:03.420076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:03.420076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:08:03.420076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:03.420076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:08:03.420076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:03.420491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:08:03.420491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:03.420491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:08:03.420491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:03.420491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:08:03.420491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:08:03.420491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:03.420491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:08:03.420491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:03.420491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:08:03.420491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:08:03.420491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:03.420491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:08:03.420911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:03.420911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:08:03.421035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:03.421035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:08:03.421035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:08:03.421035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:08:03.421035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:08:03.421035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:08:03.421035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:08:03.421035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:08:03.421035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:08:03.421035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:08:03.421035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:08:03.421035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:08:03.421035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:08:03.421035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:08:03.421035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:08:03.421035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:08:03.421690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:08:03.421690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:08:03.421690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:03.421690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:08:03.421690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:03.421690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:08:03.421690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:03.421690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:08:03.421690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:03.421690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:08:03.421690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:03.421690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:08:03.421690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:08:03.421690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:03.421690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:08:03.421690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:03.421690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:08:03.421690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:08:03.421690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:03.422291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:08:03.422291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:03.422291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:08:07.861620 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:08:07.862300 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:08:07.862456 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:08:07.863518 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:08:07.919114 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:08:07.919551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:07.920695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:08:07.920695 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:07.920695 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:08:07.920695 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:08:07.920695 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:07.920695 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:08:07.920695 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:07.920695 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:08:07.921935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.058 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:07.921935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:08:07.921935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:08:07.921935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:08:07.921935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:08:07.921935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:08:07.921935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:08:07.921935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:07.921935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:08:07.921935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:07.921935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:08:07.921935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:07.922467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:08:07.922467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:07.922467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:08:07.922467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:07.922467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:08:07.922467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:08:07.922467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:07.922467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:08:07.922467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:07.922467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:08:07.922467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:08:07.922467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:07.922467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:08:07.923030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:07.923030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:08:07.923134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:07.923134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:08:07.923134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:08:07.923134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:08:07.923134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:08:07.923134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:08:07.923134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:08:07.923134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:08:07.923134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:08:07.923134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:08:07.923134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:08:07.923134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:08:07.923134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:08:07.923134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:08:07.923134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:08:07.923134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:08:07.923862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:08:07.923862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:08:07.923862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:07.923862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:08:07.923862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:07.923862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:08:07.923862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:07.923862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:08:07.923862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:07.923862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:08:07.923862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:07.923862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:08:07.923862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:08:07.923862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:07.923862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:08:07.923862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:07.923862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:08:07.923862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:08:07.923862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:07.924646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:08:07.924646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:07.924646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:08:08.376351 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:08:08.377237 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:08:08.377443 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:08:08.378885 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:08:08.379269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:08.380274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:08:08.380274 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:08.380274 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:08:08.380274 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:08:08.380274 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:08.380274 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:08:08.380274 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:08.380274 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:08:08.382037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:08.382037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:08:08.382037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:08:08.382037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:08:08.382037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:08:08.382037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:08:08.382037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:08:08.382037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:08.382037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:08:08.382037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:08.382037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:08:08.382037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:08.382635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:08:08.382635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:08.382635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:08:08.382635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:08.382635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:08:08.382635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:08:08.382635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:08.382635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:08:08.382635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:08.382635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:08:08.382635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:08:08.382635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:08.382635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:08:08.383275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:08.383275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:08:08.383958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:08.383958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:08:08.383958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:08:08.383958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:08:08.383958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:08:08.383958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:08:08.383958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:08:08.383958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:08:08.383958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:08:08.383958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:08:08.383958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:08:08.383958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:08:08.383958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:08:08.383958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:08:08.383958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:08:08.383958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:08:08.384693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:08:08.384693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:08:08.384693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:08.384693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:08:08.384693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:08.384693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:08:08.384693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:08.384693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:08:08.384693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:08.384693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:08:08.384693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:08.384693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:08:08.384693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:08:08.384693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:08.384693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:08:08.384693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:08.384693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:08:08.384693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:08:08.384693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:08.385596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:08:08.385596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:08.385596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:08:08.386123 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:08:08.386566 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:08:08.386685 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:08:08.386792 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:08:08.387357 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:08:08.387701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:08.388554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:08:08.388554 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:08.388554 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:08:08.388554 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:08:08.388554 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:08.388554 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:08:08.388554 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:08.388554 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:08:08.389351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:08.389351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:08:08.389351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:08:08.389351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:08:08.389351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:08.389351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:08:08.389351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:08.389351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:08:08.389351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:08.389351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:08:08.389351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:08.389351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:08:08.389351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:08.389351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:08:08.389351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:08:08.389787 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:08.389787 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:08:08.389787 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:08.389787 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:08:08.389787 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:08:08.389787 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:08.389787 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:08:08.389787 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:08.389787 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:08:08.390200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:08.390200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:08:08.390200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:08:08.390200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:08:08.390200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:08:08.390200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:08:08.390200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:08:08.390200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:08:08.390200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:08:08.390200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:08:08.390200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:08.390200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:08:08.390543 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:08.390543 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:08:08.390543 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:08.390543 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:08:08.390543 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:08.390543 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:08:08.390543 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:08.390543 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:08:08.390543 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:08:08.390543 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:08.390543 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:08:08.390543 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:08.390543 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:08:08.390543 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:08:08.390947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:08.390947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:08:08.390947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:08.390947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:08:08.391213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:08.391213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:08:08.391213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:08:08.391213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:08:08.391213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:08:08.391213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:08:08.391213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:08:08.391213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:08:08.391213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:08:08.391213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:08:08.391213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:08:08.391213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:08:08.391213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:08:08.391213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:08:08.391213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:08:08.391213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:08:08.391650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:08:08.391650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:08:08.391650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:08:08.391650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:08:08.391650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:08:08.391650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:08.391650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:08:08.391650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:08.391650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:08:08.391650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:08.391650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:08:08.391650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:08.391650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:08:08.391650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:08.391650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:08:08.391650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:08:08.391650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:08.391650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:08:08.392177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:08.392177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:08:08.392177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:08:08.392177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:08.392177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:08:08.392177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:08.392177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:08:08.408148 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:08:08.408575 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:08:08.408958 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:08:08.409154 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:08:08.409761 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:08:08.410177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:08.411055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:08:08.411055 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:08.411055 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:08:08.411055 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:08:08.411055 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:08.411055 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:08:08.411055 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:08.411055 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:08:08.412299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:08.412299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:08:08.412299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:08:08.412299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:08:08.412299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:08.412299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:08:08.412299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:08.412299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:08:08.412299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:08.412299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:08:08.412299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:08.412299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:08:08.412299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:08.412299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:08:08.412299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:08:08.412782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:08.412782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:08:08.412782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:08.412782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:08:08.412782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:08:08.412782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:08.412782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:08:08.412782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:08.412782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:08:08.413265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:08.413265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:08:08.413265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:08:08.413265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:08:08.413265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:08:08.413265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:08:08.413265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:08:08.413265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:08:08.413265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:08:08.413265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:08.413265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:08:08.413604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:08.413604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:08:08.413604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:08.413604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:08:08.413604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:08.413604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:08:08.413604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:08.413604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:08:08.413604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:08:08.413604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:08.413604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:08:08.413604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:08.413604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:08:08.413604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:08:08.413975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:08.413975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:08:08.413975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:08.413975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:08:08.414391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:08.414391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:08:08.414391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:08:08.414391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:08:08.414391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:08:08.414391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:08:08.414391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:08:08.414391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:08:08.414391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:08:08.414391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:08:08.414391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:08:08.414391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:08:08.414391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:08:08.414391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:08:08.414391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:08:08.414391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:08:08.414391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:08:08.414896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:08:08.414896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:08:08.414896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:08:08.414896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:08.414896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:08:08.414896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:08.414896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:08:08.414896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:08.414896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:08:08.414896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:08.414896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:08:08.414896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:08.414896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:08:08.414896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:08:08.414896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:08.414896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:08:08.414896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:08.414896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:08:08.414896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:08:08.415393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:08.415393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:08:08.415393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:08.415393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:08:08.416995 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:08:08.417357 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:08:08.417464 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:08:08.417626 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:08:08.418042 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:08:08.418282 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:08:08.418420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:08.419087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:08:08.419087 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:08.419087 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:08:08.419087 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:08:08.419087 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:08.419087 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:08:08.419087 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:08.419087 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:08:08.420025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:08.420025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:08:08.420025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:08:08.420025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:08:08.420025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:08:08.420025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:08:08.420025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:08:08.420025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:08.420025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:08:08.420025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:08.420025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:08:08.420025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:08.420386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:08:08.420386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:08.420386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:08:08.420386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:08.420386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:08:08.420386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:08:08.420386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:08.420386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:08:08.420386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:08.420386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:08:08.420386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:08:08.420386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:08.420386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:08:08.420761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:08.420761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:08:08.421010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:08.421010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:08:08.421010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:08:08.421010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:08:08.421010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:08:08.421010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:08:08.421010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:08:08.421010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:08:08.421010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:08:08.421010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:08:08.421010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:08:08.421010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:08:08.421010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:08:08.421010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:08:08.421010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:08:08.421010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:08:08.421452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:08:08.421452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:08:08.421452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:08.421452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:08:08.421452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:08.421452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:08:08.421452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:08.421452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:08:08.421452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:08.421452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:08:08.421452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:08.421452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:08:08.421452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:08:08.421452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:08.421452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:08:08.421452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:08.421452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:08:08.421452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:08:08.421452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:08.421990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:08:08.421990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:08.421990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:08:12.862442 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:08:12.863392 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:08:12.863645 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:08:12.865248 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:08:12.900890 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:08:12.901471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:12.902693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:08:12.902693 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:12.902693 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:08:12.902693 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:08:12.902693 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:12.902693 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:08:12.902693 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:12.902693 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:08:12.904027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:12.904027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:08:12.904027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:08:12.904027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:08:12.904027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:08:12.904027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:08:12.904027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:08:12.904027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:12.904027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:08:12.904027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:12.904027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:08:12.904027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:12.904822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:08:12.904822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:12.904822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:08:12.904822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:12.904822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:08:12.904822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:08:12.904822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:12.904822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:08:12.904822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:12.904822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:08:12.904822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:08:12.904822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:12.904822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:08:12.905503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:12.905503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:08:12.905503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:12.905503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:08:12.905503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:08:12.905503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:08:12.905503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:08:12.905503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:08:12.905503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:08:12.905503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:08:12.905503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:08:12.905503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:08:12.905503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:08:12.905503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:08:12.905503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:08:12.906264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:08:12.906264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:08:12.906264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:08:12.906264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:08:12.906264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:08:12.906264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:12.906264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:08:12.906264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:12.906264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:08:12.906264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:12.906264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:08:12.906264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:12.906264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:08:12.906264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:12.906264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:08:12.906264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:08:12.906264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:12.906264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:08:12.906264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:12.906972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:08:12.906972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:08:12.906972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:12.906972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:08:12.906972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:12.906972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:08:13.377112 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:08:13.377927 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:08:13.378137 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:08:13.379349 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:08:13.379717 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:13.380895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:08:13.380895 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:13.380895 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:08:13.380895 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:08:13.380895 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:13.380895 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:08:13.380895 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:13.380895 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:08:13.382544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:13.382544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:08:13.382544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:08:13.382544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:08:13.382544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:08:13.382544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:08:13.382544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:08:13.382544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:13.382544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:08:13.382544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:13.382544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:08:13.382544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:13.383216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:08:13.383216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:13.383216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:08:13.383216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:13.383216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:08:13.383216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:08:13.383216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:13.383216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:08:13.383216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:13.383216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:08:13.383216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:08:13.383216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:13.383216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:08:13.383817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:13.383817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:08:13.384362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:13.384362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:08:13.384362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:08:13.384362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:08:13.384362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:08:13.384362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:08:13.384362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:08:13.384362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:08:13.384362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:08:13.384362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:08:13.384362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:08:13.384362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:08:13.384362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:08:13.384362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:08:13.384362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:08:13.384362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:08:13.385459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:08:13.385459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:08:13.385459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:13.385459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:08:13.385459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:13.385459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:08:13.385459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:13.385459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:08:13.385459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:13.385459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:08:13.385459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:13.385459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:08:13.385459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:08:13.385459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:13.385459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:08:13.385459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:13.385459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:08:13.385459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:08:13.385459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:13.386391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:08:13.386391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:13.386391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:08:13.386956 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:08:13.387250 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:08:13.387464 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:08:13.387837 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:08:13.388302 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:08:13.389216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:13.389560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:08:13.389560 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:13.389560 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:08:13.389560 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:08:13.389560 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:13.389560 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:08:13.389560 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:13.389560 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:08:13.390377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:13.390377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:08:13.390377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:08:13.390377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:08:13.390377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:13.390377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:08:13.390377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:13.390377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:08:13.390377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:13.390377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:08:13.390377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:13.390377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:08:13.390377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:13.390377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:08:13.390377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:08:13.390928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:13.390928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:08:13.390928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:13.390928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:08:13.390928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:08:13.390928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:13.390928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:08:13.390928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:13.390928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:08:13.391272 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:13.391272 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:08:13.391272 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:08:13.391272 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:08:13.391272 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:08:13.391272 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:08:13.391272 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:08:13.391272 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:08:13.391272 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:08:13.391272 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:08:13.391272 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:13.391272 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:08:13.391815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:13.391815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:08:13.391815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:13.391815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:08:13.391815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:13.391815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:08:13.391815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:13.391815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:08:13.391815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:08:13.391815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:13.391815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:08:13.391815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:13.391815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:08:13.391815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:08:13.392235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:13.392235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:08:13.392235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:13.392235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:08:13.392367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:13.392367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:08:13.392367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:08:13.392367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:08:13.392367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:08:13.392367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:08:13.392367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:08:13.392367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:08:13.392367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:08:13.392367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:08:13.392367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:08:13.392367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:08:13.392367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:08:13.392367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:08:13.392367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:08:13.392367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:08:13.392887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:08:13.392887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:08:13.392887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:08:13.392887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:08:13.392887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:08:13.392887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:13.392887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:08:13.392887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:13.392887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:08:13.392887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:13.392887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:08:13.392887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:13.392887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:08:13.392887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:13.392887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:08:13.392887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:08:13.392887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:13.392887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:08:13.394140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:13.394140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:08:13.394140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:08:13.394140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:13.394140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:08:13.394140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:13.394140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:08:13.408559 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:08:13.408978 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:08:13.409173 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:08:13.409395 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:08:13.410882 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:08:13.410882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:13.411437 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:08:13.411437 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:13.411437 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:08:13.411437 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:08:13.411437 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:13.411437 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:08:13.411437 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:13.411437 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:08:13.412711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:13.412711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:08:13.412711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:08:13.412711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:08:13.412711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:13.412711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:08:13.412711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:13.412711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:08:13.412711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:13.412711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:08:13.412711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:13.412711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:08:13.412711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:13.412711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:08:13.412711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:08:13.413579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:13.413579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:08:13.413579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:13.413579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:08:13.413579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:08:13.413579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:13.413579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:08:13.413579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:13.413579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:08:13.414166 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:13.414166 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:08:13.414166 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:08:13.414166 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:08:13.414166 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:08:13.414166 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:08:13.414166 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:08:13.414166 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:08:13.414166 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:08:13.414166 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:13.414166 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:08:13.414932 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:13.414932 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:08:13.414932 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:13.414932 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:08:13.414932 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:13.414932 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:08:13.414932 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:13.414932 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:08:13.414932 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:08:13.414932 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:13.414932 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:08:13.414932 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:13.414932 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:08:13.414932 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:08:13.415805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:13.415805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:08:13.415805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:13.415805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:08:13.416071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:13.416071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:08:13.416071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:08:13.416071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:08:13.416071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:08:13.416071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:08:13.416071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:08:13.416071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:08:13.416071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:08:13.416071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:08:13.416071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:08:13.416071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:08:13.416071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:08:13.416071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:08:13.416071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:08:13.416071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:08:13.416071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:08:13.417148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:08:13.417148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:08:13.417148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:08:13.417148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:13.417148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:08:13.417148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:13.417148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:08:13.417148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:13.417148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:08:13.417148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:13.417148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:08:13.417148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:13.417148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:08:13.417148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:08:13.417148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:13.417148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:08:13.417148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:13.417148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:08:13.417148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:08:13.418543 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:13.418543 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:08:13.418543 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:13.418543 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:08:13.418543 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:08:13.418543 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:08:13.418543 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:08:13.418543 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:08:13.418997 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:08:13.419244 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:08:13.419444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:13.420140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:08:13.420140 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:13.420140 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:08:13.420140 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:08:13.420140 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:13.420140 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:08:13.420140 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:13.420140 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:08:13.421039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:13.421039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:08:13.421039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:08:13.421039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:08:13.421039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:08:13.421039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:08:13.421039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:08:13.421039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:13.421039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:08:13.421039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:13.421039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:08:13.421039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:13.421656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:08:13.421656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:13.421656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:08:13.421656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:13.421656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:08:13.421656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:08:13.421656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:13.421656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:08:13.421656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:13.421656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:08:13.421656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:08:13.421656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:13.421656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:08:13.421989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:13.421989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:08:13.421989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:13.421989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:08:13.421989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:08:13.421989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:08:13.421989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:08:13.421989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:08:13.421989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:08:13.421989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:08:13.421989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:08:13.421989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:08:13.421989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:08:13.421989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:08:13.421989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:08:13.422597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:08:13.422597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:08:13.422597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:08:13.422597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:08:13.422597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:08:13.422597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:13.422597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:08:13.422597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:13.422597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:08:13.422597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:13.422597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:08:13.422597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:13.422597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:08:13.422597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:13.422597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:08:13.422597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:08:13.422597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:13.422597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:08:13.422597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:13.423010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:08:13.423010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:08:13.423010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:13.423010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:08:13.423010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:13.423010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:08:17.863287 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:08:17.864792 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:08:17.864792 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:08:17.866445 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:08:17.901142 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:08:17.901589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:17.902680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:08:17.902680 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:17.902680 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:08:17.902680 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:08:17.902680 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:17.902680 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:08:17.902680 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:17.902680 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:08:17.903732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:17.903732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:08:17.903732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:08:17.903732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:08:17.903732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:08:17.903732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:08:17.903732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:08:17.903732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:17.903732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:08:17.903732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:17.903732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:08:17.903732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:17.904359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:08:17.904359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:17.904359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:08:17.904359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:17.904359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:08:17.904359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:08:17.904359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:17.904359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:08:17.904359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:17.904359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:08:17.904359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:08:17.904359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:17.904359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:08:17.904902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:17.904902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:08:17.904902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:17.904902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:08:17.904902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:08:17.904902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:08:17.904902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:08:17.904902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:08:17.904902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:08:17.904902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:08:17.904902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:08:17.904902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:08:17.904902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:08:17.904902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:08:17.904902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:08:17.905581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:08:17.905581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:08:17.905581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:08:17.905581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:08:17.905581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:08:17.905581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:17.905581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:08:17.905581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:17.905581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:08:17.905581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:17.905581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:08:17.905581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:17.905581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:08:17.905581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:17.905581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:08:17.905581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:08:17.905581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:17.905581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:08:17.905581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:17.906413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:08:17.906413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:08:17.906413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:17.906413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:08:17.906413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:17.906413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:08:18.377787 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:08:18.378640 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:08:18.378928 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:08:18.381426 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:08:18.381852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:18.382674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:08:18.382674 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:18.382674 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:08:18.382674 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:08:18.382674 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:18.382674 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:08:18.382674 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:18.382674 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:08:18.384615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:18.384615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:08:18.384615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:08:18.384615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:08:18.384615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:08:18.384615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:08:18.384615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:08:18.384615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:18.384615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:08:18.384615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:18.384615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:08:18.384615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:18.385350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:08:18.385350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:18.385350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:08:18.385350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:18.385350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:08:18.385350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:08:18.385350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:18.385350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:08:18.385350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:18.385350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:08:18.385350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:08:18.385350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:18.385350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:08:18.385905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:18.385905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:08:18.386813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:18.386813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:08:18.386813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:08:18.386813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:08:18.386813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:08:18.386813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:08:18.386813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:08:18.386813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:08:18.386813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:08:18.386813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:08:18.386813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:08:18.386813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:08:18.386813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:08:18.386813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:08:18.386813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:08:18.386813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:08:18.388055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:08:18.388055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:08:18.388055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:18.388055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:08:18.388055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:18.388055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:08:18.388055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:18.388055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:08:18.388055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:18.388055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:08:18.388055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:18.388055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:08:18.388055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:08:18.388055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:18.388055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:08:18.388055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:18.388055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:08:18.388055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:08:18.388055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:18.388809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:08:18.388809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:18.388809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:08:18.388809 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:08:18.388809 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:08:18.388809 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:08:18.388809 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:08:18.388809 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:08:18.389203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:18.389203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:08:18.389203 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:18.389203 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:08:18.389203 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:08:18.389203 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:18.389203 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:08:18.389203 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:18.389203 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:08:18.389892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:18.389892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:08:18.389892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:08:18.389892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:08:18.389892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:18.389892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:08:18.389892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:18.389892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:08:18.389892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:18.389892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:08:18.389892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:18.389892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:08:18.389892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:18.389892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:08:18.389892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:08:18.390634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:18.390634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:08:18.390634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:18.390634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:08:18.390634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:08:18.390634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:18.390634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:08:18.390634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:18.390634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:08:18.391050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:18.391050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:08:18.391050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:08:18.391050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:08:18.391050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:08:18.391050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:08:18.391050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:08:18.391050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:08:18.391050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:08:18.391050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:08:18.391050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:18.391050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:08:18.391574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:18.391574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:08:18.391574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:18.391574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:08:18.391574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:18.391574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:08:18.391574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:18.391574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:08:18.391574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:08:18.391574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:18.391574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:08:18.391574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:18.391574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:08:18.391574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:08:18.391978 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:18.391978 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:08:18.391978 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:18.391978 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:08:18.391978 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:18.391978 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:08:18.391978 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:08:18.391978 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:08:18.391978 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:08:18.391978 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:08:18.391978 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:08:18.391978 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:08:18.391978 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:08:18.391978 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:08:18.392242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:08:18.392242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:08:18.392242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:08:18.392242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:08:18.392242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:08:18.392242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:08:18.392242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:08:18.392242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:08:18.392242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:08:18.392242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:08:18.392242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:08:18.392242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:18.392242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:08:18.392242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:18.392242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:08:18.392242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:18.392242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:08:18.392242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:18.392242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:08:18.392635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:18.392635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:08:18.392635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:08:18.392635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:18.392635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:08:18.392635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:18.392635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:08:18.392635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:08:18.392635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:18.392635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:08:18.392635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:18.392635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:08:18.409152 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:08:18.409687 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:08:18.409917 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:08:18.410138 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:08:18.410705 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:08:18.411133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:18.412073 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:08:18.412073 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:18.412073 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:08:18.412073 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:08:18.412073 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:18.412073 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:08:18.412073 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:18.412073 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:08:18.412881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:18.412881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:08:18.412881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:08:18.412881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:08:18.412881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:18.412881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:08:18.412881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:18.412881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:08:18.412881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:18.412881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:08:18.412881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:18.412881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:08:18.412881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:18.412881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:08:18.412881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:08:18.413733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:18.413733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:08:18.413733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:18.413733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:08:18.413733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:08:18.413733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:18.413733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:08:18.413733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:18.413733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:08:18.414269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:18.414269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:08:18.414269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:08:18.414269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:08:18.414269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:08:18.414269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:08:18.414269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:08:18.414269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:08:18.414269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:08:18.414269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:18.414269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:08:18.414734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:18.414734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:08:18.414734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:18.414734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:08:18.414734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:18.414734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:08:18.414734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:18.414734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:08:18.414734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:08:18.414734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:18.414734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:08:18.414734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:18.414734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:08:18.414734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:08:18.415294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:18.415294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:08:18.415294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:18.415294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:08:18.415294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:18.415294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:08:18.415294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:08:18.415294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:08:18.415294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:08:18.415294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:08:18.415294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:08:18.415294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:08:18.415294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:08:18.415294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:08:18.415951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:08:18.415951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:08:18.415951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:08:18.415951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:08:18.415951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:08:18.415951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:08:18.415951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:08:18.415951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:08:18.415951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:08:18.415951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:08:18.415951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:18.415951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:08:18.415951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:18.415951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:08:18.415951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:18.415951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:08:18.415951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:18.415951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:08:18.415951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:18.416833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:08:18.416833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:08:18.416833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:18.416833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:08:18.416833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:18.416833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:08:18.416833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:08:18.416833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:18.416833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:08:18.416833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:18.416833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:08:18.417957 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:08:18.418263 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:08:18.418384 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:08:18.418566 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:08:18.419144 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:08:18.419517 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:08:18.419713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:18.420563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:08:18.420563 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:18.420563 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:08:18.420563 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:08:18.420563 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:18.420563 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:08:18.420563 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:18.420563 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:08:18.421778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:18.421778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:08:18.421778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:08:18.421778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:08:18.421778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:08:18.421778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:08:18.421778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:08:18.421778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:18.421778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:08:18.421778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:18.421778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:08:18.421778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:18.422158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:08:18.422158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:18.422158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:08:18.422158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:18.422158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:08:18.422158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:08:18.422158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:18.422158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:08:18.422158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:18.422158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:08:18.422158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:08:18.422158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:18.422158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:08:18.422526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:18.422526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:08:18.422930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:18.422930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:08:18.422930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:08:18.422930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:08:18.422930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:08:18.422930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:08:18.422930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:08:18.422930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:08:18.422930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:08:18.422930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:08:18.422930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:08:18.422930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:08:18.422930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:08:18.422930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:08:18.422930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:08:18.422930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:08:18.423438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:08:18.423438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:08:18.423438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:18.423438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:08:18.423438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:18.423438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:08:18.423438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:18.423438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:08:18.423438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:18.423438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:08:18.423438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:18.423438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:08:18.423438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:08:18.423438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:18.423438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:08:18.423438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:18.423438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:08:18.423438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:08:18.423438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:18.424035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:08:18.424035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:18.424035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:08:22.864429 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:08:22.865364 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:08:22.865463 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:08:22.866974 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:08:22.904260 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:08:22.904722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:22.905817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:08:22.905817 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:22.905817 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:08:22.905817 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:08:22.905817 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:22.905817 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:08:22.905817 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:22.905817 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:08:22.906828 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:22.906828 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:08:22.906828 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:08:22.906828 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:08:22.906828 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:08:22.906828 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:08:22.906828 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:08:22.906828 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:22.906828 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:08:22.906828 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:22.906828 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:08:22.906828 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:22.907378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:08:22.907378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:22.907378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:08:22.907378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:22.907378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:08:22.907378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:08:22.907378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:22.907378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:08:22.907378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:22.907378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:08:22.907378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:08:22.907378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:22.907378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:08:22.907977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:22.907977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:08:22.907977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:22.907977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:08:22.907977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:08:22.907977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:08:22.907977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:08:22.907977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:08:22.907977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:08:22.907977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:08:22.907977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:08:22.907977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:08:22.907977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:08:22.907977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:08:22.907977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:08:22.908734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:08:22.908734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:08:22.908734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:08:22.908734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:08:22.908734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:08:22.908734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:22.908734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:08:22.908734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:22.908734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:08:22.908734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:22.908734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:08:22.908734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:22.908734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:08:22.908734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:22.908734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:08:22.908734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:08:22.908734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:22.908734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:08:22.908734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:22.910438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:08:22.910438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:08:22.910438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:22.910438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:08:22.910438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:22.910438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:08:23.378634 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:08:23.379421 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:08:23.380514 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:08:23.381535 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:08:23.382037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:23.385785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:08:23.385785 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:23.385785 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:08:23.385785 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:08:23.385785 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:23.385785 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:08:23.385785 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:23.385785 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:08:23.385785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:23.385785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:08:23.385785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:08:23.385785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:08:23.385785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:08:23.385785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:08:23.385785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:08:23.386475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:23.386475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:08:23.386475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:23.386475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:08:23.386475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:23.386475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:08:23.386475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:23.386475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:08:23.386475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:23.386475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:08:23.386475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:08:23.386475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:23.386475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:08:23.386475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:23.387065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:08:23.387065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:08:23.387065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:23.387065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:08:23.387065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:23.387065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:08:23.389449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:23.389449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:08:23.389449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:08:23.389449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:08:23.389449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:08:23.389449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:08:23.389449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:08:23.389449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:08:23.389449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:08:23.389449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:08:23.389449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:08:23.389449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:08:23.389449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:08:23.389449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:08:23.389449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:08:23.389449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:08:23.393430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:08:23.393430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:08:23.393430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:23.393430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:08:23.393430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:23.393430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:08:23.393430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:23.393430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:08:23.393430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:23.393430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:08:23.393430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:23.393430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:08:23.393430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:08:23.393430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:23.393430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:08:23.393430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:23.393430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:08:23.393430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:08:23.393430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:23.394556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:08:23.394556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:23.394556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:08:23.394556 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:08:23.394556 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:08:23.394556 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:08:23.394556 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:08:23.394556 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:08:23.394927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:23.395403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:08:23.395403 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:23.395403 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:08:23.395403 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:08:23.395403 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:23.395403 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:08:23.395403 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:23.395403 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:08:23.396315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:23.396315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:08:23.396315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:08:23.396315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:08:23.396315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:23.396315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:08:23.396315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:23.396315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:08:23.396315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:23.396315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:08:23.396315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:23.396315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:08:23.396315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:23.396315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:08:23.396315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:08:23.396737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:23.396737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:08:23.396737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:23.396737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:08:23.396737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:08:23.396737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:23.396737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:08:23.396737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:23.396737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:08:23.397229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:23.397229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:08:23.397229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:08:23.397229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:08:23.397229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:08:23.397229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:08:23.397229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:08:23.397229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:08:23.397229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:08:23.397229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:08:23.397229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:23.397229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:08:23.397514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:23.397514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:08:23.397514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:23.397514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:08:23.397514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:23.397514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:08:23.397514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:23.397514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:08:23.397514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:08:23.397514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:23.397514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:08:23.397514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:23.397514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:08:23.397514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:08:23.397996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:23.397996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:08:23.397996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:23.397996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:08:23.398600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:23.398600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:08:23.398600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:08:23.398600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:08:23.398600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:08:23.398600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:08:23.398600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:08:23.398600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:08:23.398600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:08:23.398600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:08:23.398600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:08:23.398600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:08:23.398600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:08:23.398600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:08:23.398600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:08:23.398600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:08:23.399121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:08:23.399121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:08:23.399121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:08:23.399121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:08:23.399121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:08:23.399121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:23.399121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:08:23.399121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:23.399121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:08:23.399121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:23.399121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:08:23.399121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:23.399121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:08:23.399121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:23.399121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:08:23.399121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:08:23.399121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:23.399121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:08:23.399985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:23.399985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:08:23.399985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:08:23.399985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:23.399985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:08:23.399985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:23.399985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:08:23.409712 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:08:23.410080 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:08:23.410202 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:08:23.410355 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:08:23.410973 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:08:23.411436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:23.412232 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:08:23.412232 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:23.412232 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:08:23.412232 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:08:23.412232 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:23.412232 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:08:23.412232 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:23.412232 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:08:23.413092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:23.413092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:08:23.413092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:08:23.413092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:08:23.413092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:23.413092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:08:23.413092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:23.413092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:08:23.413092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:23.413092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:08:23.413092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:23.413092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:08:23.413092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:23.413092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:08:23.413092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:08:23.413855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:23.413855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:08:23.413855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:23.413855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:08:23.413855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:08:23.413855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:23.413855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:08:23.413855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:23.413855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:08:23.414290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:23.414290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:08:23.414290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:08:23.414290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:08:23.414290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:08:23.414290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:08:23.414290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:08:23.414290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:08:23.414290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:08:23.414290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:23.414290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:08:23.414905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:23.414905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:08:23.414905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:23.414905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:08:23.414905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:23.414905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:08:23.414905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:23.414905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:08:23.414905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:08:23.414905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:23.414905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:08:23.414905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:23.414905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:08:23.414905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:08:23.415689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:23.415689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:08:23.415689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:23.415689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:08:23.415689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:23.415689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:08:23.415689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:08:23.415689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:08:23.415689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:08:23.415689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:08:23.415689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:08:23.415689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:08:23.415689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:08:23.415689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:08:23.416473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:08:23.416473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:08:23.416473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:08:23.416473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:08:23.416473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:08:23.416473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:08:23.416473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:08:23.416473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:08:23.416473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:08:23.416473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:08:23.416473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:23.416473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:08:23.416473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:23.416473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:08:23.416473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:23.416473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:08:23.416473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:23.416473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:08:23.416473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:23.417123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:08:23.417123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:08:23.417123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:23.417123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:08:23.417123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:23.417123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:08:23.417123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:08:23.417123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:23.417123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:08:23.417123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:23.417123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:08:23.418379 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:08:23.418686 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:08:23.418848 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:08:23.419003 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:08:23.419474 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:08:23.419987 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:08:23.420258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:23.421205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:08:23.421205 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:23.421205 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:08:23.421205 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:08:23.421205 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:23.421205 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:08:23.421205 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:23.421205 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:08:23.422997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:23.422997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:08:23.422997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:08:23.422997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:08:23.422997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:08:23.422997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:08:23.422997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:08:23.422997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:23.422997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:08:23.422997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:23.422997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:08:23.422997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:23.423392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:08:23.423392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:23.423392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:08:23.423392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:23.423392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:08:23.423392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:08:23.423392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:23.423392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:08:23.423392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:23.423392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:08:23.423392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:08:23.423392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:23.423392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:08:23.423774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:23.423774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:08:23.425152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:23.425152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:08:23.425152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:08:23.425152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:08:23.425152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:08:23.425152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:08:23.425152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:08:23.425152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:08:23.425152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:08:23.425152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:08:23.425152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:08:23.425152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:08:23.425152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:08:23.425152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:08:23.425152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:08:23.425152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:08:23.425630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:08:23.425630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:08:23.425630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:23.425630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:08:23.425630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:23.425630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:08:23.425630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:23.425630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:08:23.425630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:23.425630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:08:23.425630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:23.425630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:08:23.425630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:08:23.425630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:23.425630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:08:23.425630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:23.425630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:08:23.425630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:08:23.425630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:23.426269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:08:23.426269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:23.426269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:08:27.865328 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:08:27.866516 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:08:27.866840 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:08:27.868064 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:08:27.904123 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:08:27.904519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:27.905419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:08:27.905419 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:27.905419 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:08:27.905419 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:08:27.905419 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:27.905419 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:08:27.905419 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:27.905419 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:08:27.906412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:27.906412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:08:27.906412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:08:27.906412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:08:27.906412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:08:27.906412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:08:27.906412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:08:27.906412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:27.906412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:08:27.906412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:27.906412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:08:27.906412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:27.907325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:08:27.907325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:27.907325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:08:27.907325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:27.907325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:08:27.907325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:08:27.907325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:27.907325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:08:27.907325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:27.907325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:08:27.907325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:08:27.907325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:27.907325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:08:27.907902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:27.907902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:08:27.907902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:27.907902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:08:27.907902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:08:27.907902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:08:27.907902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:08:27.907902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:08:27.907902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:08:27.907902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:08:27.907902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:08:27.907902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:08:27.907902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:08:27.907902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:08:27.907902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:08:27.908502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:08:27.908502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:08:27.908502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:08:27.908502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:08:27.908502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:08:27.908502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:27.908502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:08:27.908502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:27.908502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:08:27.908502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:27.908502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:08:27.908502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:27.908502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:08:27.908502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:27.908502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:08:27.908502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:08:27.908502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:27.908502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:08:27.908502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:27.909321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:08:27.909321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:08:27.909321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:27.909321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:08:27.909321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:27.909321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:08:28.379314 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:08:28.379995 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:08:28.380176 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:08:28.381220 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:08:28.381545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:28.382454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:08:28.382454 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:28.382454 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:08:28.382454 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:08:28.382454 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:28.382454 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:08:28.382454 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:28.382454 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:08:28.384093 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:28.384093 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:08:28.384093 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:08:28.384093 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:08:28.384093 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:08:28.384093 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:08:28.384093 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:08:28.384093 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:28.384093 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:08:28.384093 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:28.384093 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:08:28.384093 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:28.384609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:08:28.384609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:28.384609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:08:28.384609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:28.384609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:08:28.384609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:08:28.384609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:28.384609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:08:28.384609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:28.384609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:08:28.384609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:08:28.384609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:28.384609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:08:28.385284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:28.385284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:08:28.385901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:28.385901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:08:28.385901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:08:28.385901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:08:28.385901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:08:28.385901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:08:28.385901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:08:28.385901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:08:28.385901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:08:28.385901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:08:28.385901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:08:28.385901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:08:28.385901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:08:28.385901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:08:28.385901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:08:28.385901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:08:28.386665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:08:28.386665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:08:28.386665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:28.386665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:08:28.386665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:28.386665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:08:28.386665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:28.386665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:08:28.386665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:28.386665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:08:28.386665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:28.386665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:08:28.386665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:08:28.386665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:28.386665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:08:28.386665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:28.386665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:08:28.386665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:08:28.386665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:28.387644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:08:28.387644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:28.387644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:08:28.390284 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:08:28.390693 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:08:28.390998 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:08:28.391214 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:08:28.392041 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:08:28.392362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:28.393095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:08:28.393095 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:28.393095 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:08:28.393095 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:08:28.393095 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:28.393095 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:08:28.393095 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:28.393095 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:08:28.393881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:28.393881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:08:28.393881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:08:28.393881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:08:28.393881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:28.393881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:08:28.393881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:28.393881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:08:28.393881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:28.393881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:08:28.393881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:28.393881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:08:28.393881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:28.393881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:08:28.393881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:08:28.394291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:28.394291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:08:28.394291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:28.394291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:08:28.394291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:08:28.394291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:28.394291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:08:28.394291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:28.394291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:08:28.394826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:28.394826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:08:28.394826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:08:28.394826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:08:28.394826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:08:28.394826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:08:28.394826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:08:28.394826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:08:28.394826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:08:28.394826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:08:28.394826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:28.394826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:08:28.395609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:28.395609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:08:28.395609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:28.395609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:08:28.395609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:28.395609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:08:28.395609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:28.395609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:08:28.395609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:08:28.395609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:28.395609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:08:28.395609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:28.395609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:08:28.395609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:08:28.396089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:28.396089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:08:28.396089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:28.396089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:08:28.396089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:28.396089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:08:28.396089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:08:28.396089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:08:28.396089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:08:28.396089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:08:28.396089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:08:28.396089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:08:28.396089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:08:28.396089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:08:28.396939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:08:28.396939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:08:28.396939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:08:28.396939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:08:28.396939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:08:28.396939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:08:28.396939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:08:28.396939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:08:28.396939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:08:28.396939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:08:28.396939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:08:28.396939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:28.396939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:08:28.396939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:28.396939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:08:28.396939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:28.396939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:08:28.396939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:28.396939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:08:28.397434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:28.397434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:08:28.397434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:08:28.397434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:28.397434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:08:28.397434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:28.397434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:08:28.397434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:08:28.397434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:28.397434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:08:28.397434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:28.397434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:08:28.410205 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:08:28.410664 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:08:28.410980 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:08:28.411257 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:08:28.411766 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:08:28.412172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:28.413025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:08:28.413025 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:28.413025 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:08:28.413025 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:08:28.413025 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:28.413025 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:08:28.413025 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:28.413025 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:08:28.414015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:28.414015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:08:28.414015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:08:28.414015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:08:28.414015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:28.414015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:08:28.414015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:28.414015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:08:28.414015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:28.414015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:08:28.414015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:28.414015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:08:28.414015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:28.414015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:08:28.414015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:08:28.414644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:28.414644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:08:28.414644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:28.414644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:08:28.414644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:08:28.414644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:28.414644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:08:28.414644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:28.414644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:08:28.415184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:28.415184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:08:28.415184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:08:28.415184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:08:28.415184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:08:28.415184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:08:28.415184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:08:28.415184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:08:28.415184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:08:28.415184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:28.415184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:08:28.415662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:28.415662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:08:28.415662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:28.415662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:08:28.415662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:28.415662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:08:28.415662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:28.415662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:08:28.415662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:08:28.415662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:28.415662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:08:28.415662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:28.415662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:08:28.415662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:08:28.416478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:28.416478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:08:28.416478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:28.416478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:08:28.416478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:28.416478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:08:28.416478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:08:28.416478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:08:28.416478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:08:28.416478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:08:28.416478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:08:28.416478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:08:28.416478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:08:28.416478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:08:28.417736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:08:28.417736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:08:28.417736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:08:28.417736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:08:28.417736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:08:28.417736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:08:28.417736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:08:28.417736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:08:28.417736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:08:28.417736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:08:28.417736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:28.417736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:08:28.417736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:28.417736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:08:28.417736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:28.417736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:08:28.417736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:28.417736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:08:28.417736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:28.419552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:08:28.419552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:08:28.419552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:28.419552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:08:28.419552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:28.419552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:08:28.419552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:08:28.419552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:28.419552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:08:28.419552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:28.419552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:08:28.419552 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:08:28.419552 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:08:28.419914 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:08:28.419914 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:08:28.419914 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:08:28.420023 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:08:28.420174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:28.420612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:08:28.420612 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:28.420612 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:08:28.420612 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:08:28.420612 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:28.420612 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:08:28.420612 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:28.420612 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:08:28.421465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:28.421465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:08:28.421465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:08:28.421465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:08:28.421465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:08:28.421465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:08:28.421465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:08:28.421465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:28.421465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:08:28.421465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:28.421465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:08:28.421465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:28.422222 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:08:28.422222 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:28.422222 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:08:28.422222 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:28.422222 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:08:28.422222 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:08:28.422222 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:28.422222 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:08:28.422222 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:28.422222 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:08:28.422222 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:08:28.422222 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:28.422222 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:08:28.423002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:28.423002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:08:28.423002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:28.423002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:08:28.423002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:08:28.423002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:08:28.423002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:08:28.423002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:08:28.423002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:08:28.423002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:08:28.423002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:08:28.423002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:08:28.423002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:08:28.423002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:08:28.423002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:08:28.423712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:08:28.423712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:08:28.423712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:08:28.423712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:08:28.423712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:08:28.423712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:28.423712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:08:28.423712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:28.423712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:08:28.423712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:28.423712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:08:28.423712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:28.423712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:08:28.423712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:28.423712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:08:28.423712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:08:28.423712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:28.423712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:08:28.423712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:28.424099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:08:28.424099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:08:28.424099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:28.424099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:08:28.424099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:28.424099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:08:32.866566 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:08:32.867596 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:08:32.867596 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:08:32.869092 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:08:32.900442 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:08:32.900764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:32.901454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:08:32.901454 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:32.901454 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:08:32.901454 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:08:32.901454 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:32.901454 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:08:32.901454 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:32.901454 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:08:32.902433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:32.902433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:08:32.902433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:08:32.902433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:08:32.902433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:08:32.902433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:08:32.902433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:08:32.902433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:32.902433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:08:32.902433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:32.902433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:08:32.902433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:32.903029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:08:32.903029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:32.903029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:08:32.903029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:32.903029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:08:32.903029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:08:32.903029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:32.903029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:08:32.903029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:32.903029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:08:32.903029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:08:32.903029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:32.903029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:08:32.903559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:32.903559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:08:32.903559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:32.903559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:08:32.903559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:08:32.903559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:08:32.903559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:08:32.903559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:08:32.903559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:08:32.903559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:08:32.903559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:08:32.903559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:08:32.903559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:08:32.903559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:08:32.903559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:08:32.904200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:08:32.904200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:08:32.904200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:08:32.904200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:08:32.904200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:08:32.904200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:32.904200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:08:32.904200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:32.904200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:08:32.904200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:32.904200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:08:32.904200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:32.904200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:08:32.904200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:32.904200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:08:32.904200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:08:32.904200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:32.904200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:08:32.904200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:32.905113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:08:32.905113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:08:32.905113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:32.905113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:08:32.905113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:32.905113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:08:33.379932 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:08:33.380554 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:08:33.380778 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:08:33.381772 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:08:33.382053 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:33.382728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:08:33.382728 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:33.382728 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:08:33.382728 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:08:33.382728 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:33.382728 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:08:33.382728 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:33.382728 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:08:33.384550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:33.384550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:08:33.384550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:08:33.384550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:08:33.384550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:08:33.384550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:08:33.384550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:08:33.384550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:33.384550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:08:33.384550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:33.384550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:08:33.384550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:33.385944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:08:33.385944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:33.385944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:08:33.385944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:33.385944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:08:33.385944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:08:33.385944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:33.385944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:08:33.385944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:33.385944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:08:33.385944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:08:33.385944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:33.385944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:08:33.386891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:33.386891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:08:33.386891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:33.386891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:08:33.386891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:08:33.386891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:08:33.386891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:08:33.386891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:08:33.386891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:08:33.386891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:08:33.386891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:08:33.386891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:08:33.386891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:08:33.386891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:08:33.386891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:08:33.387359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:08:33.387359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:08:33.387359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:08:33.387359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:08:33.387359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:08:33.387359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:33.387359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:08:33.387359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:33.387359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:08:33.387359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:33.387359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:08:33.387359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:33.387359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:08:33.387359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:33.387359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:08:33.387359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:08:33.387359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:33.387359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:08:33.387359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:33.387944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:08:33.387944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:08:33.387944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:33.387944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:08:33.387944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:33.387944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:08:33.390784 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:08:33.391179 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:08:33.391371 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:08:33.391481 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:08:33.391814 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:08:33.392031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:33.392558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:08:33.392558 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:33.392558 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:08:33.392558 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:08:33.392558 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:33.392558 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:08:33.392558 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:33.392558 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:08:33.393462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:33.393462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:08:33.393462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:08:33.393462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:08:33.393462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:33.393462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:08:33.393462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:33.393462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:08:33.393462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:33.393462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:08:33.393462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:33.393462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:08:33.393462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:33.393462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:08:33.393462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:08:33.393978 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:33.393978 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:08:33.393978 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:33.393978 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:08:33.393978 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:08:33.393978 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:33.393978 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:08:33.393978 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:33.393978 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:08:33.394402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:33.394402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:08:33.394402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:08:33.394402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:08:33.394402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:08:33.394402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:08:33.394402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:08:33.394402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:08:33.394402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:08:33.394402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:08:33.394402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:33.394402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:08:33.394718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:33.394718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:08:33.394718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:33.394718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:08:33.394718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:33.394718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:08:33.394718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:33.394718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:08:33.394718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:08:33.394718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:33.394718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:08:33.394718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:33.394718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:08:33.394718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:08:33.394988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:33.394988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:08:33.394988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:33.394988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:08:33.395576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:33.395576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:08:33.395576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:08:33.395576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:08:33.395576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:08:33.395576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:08:33.395576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:08:33.395576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:08:33.395576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:08:33.395576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:08:33.395576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:08:33.395576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:08:33.395576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:08:33.395576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:08:33.395576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:08:33.395576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:08:33.395916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:08:33.395916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:08:33.395916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:08:33.395916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:08:33.395916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:08:33.395916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:33.395916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:08:33.395916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:33.395916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:08:33.395916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:33.395916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:08:33.395916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:33.395916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:08:33.395916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:33.395916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:08:33.395916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:08:33.395916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:33.395916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:08:33.396235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:33.396235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:08:33.396235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:08:33.396235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:33.396235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:08:33.396235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:33.396235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:08:33.410675 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:08:33.411197 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:08:33.411385 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:08:33.411616 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:08:33.412127 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:08:33.412467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:33.413227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:08:33.413227 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:33.413227 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:08:33.413227 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:08:33.413227 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:33.413227 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:08:33.413227 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:33.413227 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:08:33.414473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:33.414473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:08:33.414473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:08:33.414473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:08:33.414473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:33.414473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:08:33.414473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:33.414473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:08:33.414473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:33.414473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:08:33.414473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:33.414473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:08:33.414473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:33.414473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:08:33.414473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:08:33.415193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:33.415193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:08:33.415193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:33.415193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:08:33.415193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:08:33.415193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:33.415193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:08:33.415193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:33.415193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:08:33.416126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:33.416126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:08:33.416126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:08:33.416126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:08:33.416126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:08:33.416126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:08:33.416126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:08:33.416126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:08:33.416126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:08:33.416126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:33.416126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:08:33.416616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:33.416616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:08:33.416616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:33.416616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:08:33.416616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:33.416616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:08:33.416616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:33.416616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:08:33.416616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:08:33.416616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:33.416616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:08:33.416616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:33.416616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:08:33.416616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:08:33.417247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:33.417247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:08:33.417247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:33.417247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:08:33.417960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:33.417960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:08:33.417960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:08:33.417960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:08:33.417960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:08:33.417960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:08:33.417960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:08:33.417960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:08:33.417960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:08:33.417960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:08:33.417960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:08:33.417960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:08:33.417960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:08:33.417960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:08:33.417960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:08:33.417960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:08:33.417960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:08:33.418934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:08:33.418934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:08:33.418934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:08:33.418934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:33.418934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:08:33.418934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:33.418934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:08:33.418934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:33.418934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:08:33.418934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:33.418934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:08:33.418934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:33.418934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:08:33.418934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:08:33.418934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:33.418934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:08:33.418934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:33.418934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:08:33.418934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:08:33.419781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:33.419781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:08:33.419781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:33.419781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:08:33.419781 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:08:33.419781 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:08:33.420092 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:08:33.420216 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:08:33.420658 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:08:33.421185 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:08:33.421449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:33.422382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:08:33.422382 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:33.422382 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:08:33.422382 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:08:33.422382 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:33.422382 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:08:33.422382 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:33.422382 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:08:33.423685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:33.423685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:08:33.423685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:08:33.423685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:08:33.423685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:08:33.423685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:08:33.423685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:08:33.423685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:33.423685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:08:33.423685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:33.423685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:08:33.423685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:33.424066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:08:33.424066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:33.424066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:08:33.424066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:33.424066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:08:33.424066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:08:33.424066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:33.424066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:08:33.424066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:33.424066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:08:33.424066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:08:33.424066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:33.424066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:08:33.424462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:33.424462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:08:33.424679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:33.424679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:08:33.424679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:08:33.424679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:08:33.424679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:08:33.424679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:08:33.424679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:08:33.424679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:08:33.424679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:08:33.424679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:08:33.424679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:08:33.424679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:08:33.424679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:08:33.424679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:08:33.424679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:08:33.424679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:08:33.425144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:08:33.425144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:08:33.425144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:33.425144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:08:33.425144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:33.425144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:08:33.425144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:33.425144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:08:33.425144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:33.425144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:08:33.425144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:33.425144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:08:33.425144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:08:33.425144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:33.425144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:08:33.425144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:33.425144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:08:33.425144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:08:33.425144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:33.425677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:08:33.425677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:33.425677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:08:37.867342 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:08:37.870595 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:08:37.870595 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:08:37.870595 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:08:37.889802 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG dbcounter [-] [106314] Writing DB stats neutron:SELECT=232 {{(pid=106314) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 09 09:08:37.905908 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:08:37.906296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:37.907139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:08:37.907139 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:37.907139 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:08:37.907139 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:08:37.907139 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:37.907139 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:08:37.907139 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:37.907139 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:08:37.907946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:37.907946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:08:37.907946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:08:37.907946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:08:37.907946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:08:37.907946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:08:37.907946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:08:37.907946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:37.907946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:08:37.907946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:37.907946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:08:37.907946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:37.908602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:08:37.908602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:37.908602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:08:37.908602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:37.908602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:08:37.908602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:08:37.908602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:37.908602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:08:37.908602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:37.908602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:08:37.908602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:08:37.908602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:37.908602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:08:37.909261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:37.909261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:08:37.909261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:37.909261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:08:37.909261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:08:37.909261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:08:37.909261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:08:37.909261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:08:37.909261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:08:37.909261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:08:37.909261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:08:37.909261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:08:37.909261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:08:37.909261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:08:37.909261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:08:37.910106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:08:37.910106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:08:37.910106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:08:37.910106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:08:37.910106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:08:37.910106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:37.910106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:08:37.910106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:37.910106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:08:37.910106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:37.910106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:08:37.910106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:37.910106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:08:37.910106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:37.910106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:08:37.910106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:08:37.910106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:37.910106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:08:37.910106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:37.911128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:08:37.911128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:08:37.911128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:37.911128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:08:37.911128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:37.911128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:08:38.380687 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:08:38.381089 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:08:38.381298 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:08:38.381914 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:08:38.382236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:38.383165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:08:38.383165 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:38.383165 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:08:38.383165 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:08:38.383165 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:38.383165 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:08:38.383165 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:38.383165 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:08:38.384062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:38.384062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:08:38.384062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:08:38.384062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:08:38.384062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:08:38.384062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:08:38.384062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:08:38.384062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:38.384062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:08:38.384062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:38.384062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:08:38.384062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:38.385160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:08:38.385160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:38.385160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:08:38.385160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:38.385160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:08:38.385160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:08:38.385160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:38.385160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:08:38.385160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:38.385160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:08:38.385160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:08:38.385160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:38.385160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:08:38.385951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:38.385951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:08:38.385951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:38.385951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:08:38.385951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:08:38.385951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:08:38.385951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:08:38.385951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:08:38.385951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:08:38.385951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:08:38.385951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:08:38.385951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:08:38.385951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:08:38.385951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:08:38.385951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:08:38.386542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:08:38.386542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:08:38.386542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:08:38.386542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:08:38.386542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:08:38.386542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:38.386542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:08:38.386542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:38.386542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:08:38.386542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:38.386542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:08:38.386542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:38.386542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:08:38.386542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:38.386542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:08:38.386542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:08:38.386542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:38.386542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:08:38.386542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:38.387303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:08:38.387303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:08:38.387303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:38.387303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:08:38.387303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:38.387303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:08:38.391701 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:08:38.392539 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:08:38.392539 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:08:38.392539 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:08:38.393255 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:08:38.393696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:38.394863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:08:38.394863 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:38.394863 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:08:38.394863 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:08:38.394863 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:38.394863 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:08:38.394863 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:38.394863 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:08:38.396281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:38.396281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:08:38.396281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:08:38.396281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:08:38.396281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:38.396281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:08:38.396281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:38.396281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:08:38.396281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:38.396281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:08:38.396281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:38.396281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:08:38.396281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:38.396281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:08:38.396281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:08:38.396985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:38.396985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:08:38.396985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:38.396985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:08:38.396985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:08:38.396985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:38.396985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:08:38.396985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:38.396985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:08:38.397891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:38.397891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:08:38.397891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:08:38.397891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:08:38.397891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:08:38.397891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:08:38.397891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:08:38.397891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:08:38.397891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:08:38.397891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:08:38.397891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:38.397891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:08:38.398523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:38.398523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:08:38.398523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:38.398523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:08:38.398523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:38.398523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:08:38.398523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:38.398523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:08:38.398523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:08:38.398523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:38.398523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:08:38.398523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:38.398523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:08:38.398523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:08:38.398946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:38.398946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:08:38.398946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:38.398946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:08:38.399685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:38.399685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:08:38.399685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:08:38.399685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:08:38.399685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:08:38.399685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:08:38.399685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:08:38.399685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:08:38.399685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:08:38.399685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:08:38.399685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:08:38.399685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:08:38.399685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:08:38.399685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:08:38.399685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:08:38.399685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:08:38.400205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:08:38.400205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:08:38.400205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:08:38.400205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:08:38.400205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:08:38.400205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:38.400205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:08:38.400205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:38.400205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:08:38.400205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:38.400205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:08:38.400205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:38.400205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:08:38.400205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:38.400205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:08:38.400205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:08:38.400205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:38.400205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:08:38.400692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:38.400692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:08:38.400692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:08:38.400692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:38.400692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:08:38.400692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:38.400692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:08:38.411429 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:08:38.411834 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:08:38.411949 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:08:38.412104 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:08:38.412772 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:08:38.413230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:38.414424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:08:38.414424 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:38.414424 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:08:38.414424 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:08:38.414424 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:38.414424 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:08:38.414424 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:38.414424 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:08:38.415342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:38.415342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:08:38.415342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:08:38.415342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:08:38.415342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:38.415342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:08:38.415342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:38.415342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:08:38.415342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:38.415342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:08:38.415342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:38.415342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:08:38.415342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:38.415342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:08:38.415342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:08:38.415941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:38.415941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:08:38.415941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:38.415941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:08:38.415941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:08:38.415941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:38.415941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:08:38.415941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:38.415941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:08:38.416765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:38.416765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:08:38.416765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:08:38.416765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:08:38.416765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:08:38.416765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:08:38.416765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:08:38.416765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:08:38.416765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:08:38.416765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:38.416765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:08:38.417283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:38.417283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:08:38.417283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:38.417283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:08:38.417283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:38.417283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:08:38.417283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:38.417283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:08:38.417283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:08:38.417283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:38.417283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:08:38.417283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:38.417283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:08:38.417283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:08:38.418044 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:38.418044 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:08:38.418044 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:38.418044 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:08:38.418615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:38.418615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:08:38.418615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:08:38.418615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:08:38.418615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:08:38.418615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:08:38.418615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:08:38.418615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:08:38.418615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:08:38.418615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:08:38.418615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:08:38.418615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:08:38.418615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:08:38.418615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:08:38.418615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:08:38.418615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:08:38.418615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:08:38.419722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:08:38.419722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:08:38.419722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:08:38.419722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:38.419722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:08:38.419722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:38.419722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:08:38.419722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:38.419722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:08:38.419722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:38.419722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:08:38.419722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:38.419722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:08:38.419722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:08:38.419722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:38.419722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:08:38.419722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:38.419722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:08:38.419722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:08:38.420698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:38.420698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:08:38.420698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:38.420698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:08:38.420698 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:08:38.420698 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:08:38.420698 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:08:38.421629 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:08:38.421629 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:08:38.421629 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:08:38.422065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:38.422558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:08:38.422558 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:38.422558 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:08:38.422558 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:08:38.422558 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:38.422558 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:08:38.422558 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:38.422558 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:08:38.423589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:38.423589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:08:38.423589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:08:38.423589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:08:38.423589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:08:38.423589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:08:38.423589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:08:38.423589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:38.423589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:08:38.423589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:38.423589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:08:38.423589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:38.424003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:08:38.424003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:38.424003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:08:38.424003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:38.424003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:08:38.424003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:08:38.424003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:38.424003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:08:38.424003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:38.424003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:08:38.424003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:08:38.424003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:38.424003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:08:38.424404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:38.424404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:08:38.424719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:38.424719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:08:38.424719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:08:38.424719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:08:38.424719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:08:38.424719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:08:38.424719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:08:38.424719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:08:38.424719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:08:38.424719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:08:38.424719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:08:38.424719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:08:38.424719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:08:38.424719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:08:38.424719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:08:38.424719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:08:38.425202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:08:38.425202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:08:38.425202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:38.425202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:08:38.425202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:38.425202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:08:38.425202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:38.425202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:08:38.425202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:38.425202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:08:38.425202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:38.425202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:08:38.425202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:08:38.425202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:38.425202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:08:38.425202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:38.425202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:08:38.425202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:08:38.425202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:38.425703 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:08:38.425703 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:38.425703 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:08:42.868704 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:08:42.869445 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:08:42.869445 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:08:42.870689 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:08:42.897088 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:08:42.897437 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:42.898143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:08:42.898143 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:42.898143 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:08:42.898143 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:08:42.898143 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:42.898143 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:08:42.898143 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:42.898143 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:08:42.898907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.029 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:42.898907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:08:42.898907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:08:42.898907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:08:42.898907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:08:42.898907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:08:42.898907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:08:42.898907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:42.898907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:08:42.898907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:42.898907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:08:42.898907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:42.899432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:08:42.899432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:42.899432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:08:42.899432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:42.899432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:08:42.899432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:08:42.899432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:42.899432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:08:42.899432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:42.899432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:08:42.899432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:08:42.899432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:42.899432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:08:42.900068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:42.900068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:08:42.900068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:42.900068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:08:42.900068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:08:42.900068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:08:42.900068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:08:42.900068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:08:42.900068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:08:42.900068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:08:42.900068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:08:42.900068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:08:42.900068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:08:42.900068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:08:42.900068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:08:42.900710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:08:42.900710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:08:42.900710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:08:42.900710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:08:42.900710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:08:42.900710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:42.900710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:08:42.900710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:42.900710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:08:42.900710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:42.900710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:08:42.900710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:42.900710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:08:42.900710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:42.900710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:08:42.900710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:08:42.900710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:42.900710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:08:42.900710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:42.901503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:08:42.901503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:08:42.901503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:42.901503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:08:42.901503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:42.901503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:08:43.381568 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:08:43.382449 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:08:43.382647 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:08:43.383871 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:08:43.384242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:43.385202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:08:43.385202 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:43.385202 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:08:43.385202 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:08:43.385202 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:43.385202 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:08:43.385202 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:43.385202 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:08:43.386963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:43.386963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:08:43.386963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:08:43.386963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:08:43.386963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:08:43.386963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:08:43.386963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:08:43.386963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:43.386963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:08:43.386963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:43.386963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:08:43.386963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:43.387593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:08:43.387593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:43.387593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:08:43.387593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:43.387593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:08:43.387593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:08:43.387593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:43.387593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:08:43.387593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:43.387593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:08:43.387593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:08:43.387593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:43.387593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:08:43.388214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:43.388214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:08:43.388214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:43.388214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:08:43.388214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:08:43.388214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:08:43.388214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:08:43.388214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:08:43.388214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:08:43.388214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:08:43.388214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:08:43.388214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:08:43.388214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:08:43.388214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:08:43.388214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:08:43.388930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:08:43.388930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:08:43.388930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:08:43.388930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:08:43.388930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:08:43.388930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:43.388930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:08:43.388930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:43.388930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:08:43.388930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:43.388930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:08:43.388930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:43.388930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:08:43.388930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:43.388930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:08:43.388930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:08:43.388930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:43.388930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:08:43.388930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:43.389999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:08:43.389999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:08:43.389999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:43.389999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:08:43.389999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:43.389999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:08:43.392185 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:08:43.393021 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:08:43.393239 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:08:43.393449 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:08:43.394102 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:08:43.394435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:43.395345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:08:43.395345 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:43.395345 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:08:43.395345 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:08:43.395345 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:43.395345 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:08:43.395345 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:43.395345 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:08:43.396763 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:43.396763 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:08:43.396763 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:08:43.396763 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:08:43.396763 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:43.396763 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:08:43.396763 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:43.396763 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:08:43.396763 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:43.396763 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:08:43.396763 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:43.396763 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:08:43.396763 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:43.396763 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:08:43.396763 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:08:43.397245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:43.397245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:08:43.397245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:43.397245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:08:43.397245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:08:43.397245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:43.397245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:08:43.397245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:43.397245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:08:43.398210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:43.398210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:08:43.398210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:08:43.398210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:08:43.398210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:08:43.398210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:08:43.398210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:08:43.398210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:08:43.398210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:08:43.398210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:08:43.398210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:43.398210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:08:43.398558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:43.398558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:08:43.398558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:43.398558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:08:43.398558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:43.398558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:08:43.398558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:43.398558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:08:43.398558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:08:43.398558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:43.398558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:08:43.398558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:43.398558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:08:43.398558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:08:43.398965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:43.398965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:08:43.398965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:43.398965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:08:43.399830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:43.399830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:08:43.399830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:08:43.399830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:08:43.399830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:08:43.399830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:08:43.399830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:08:43.399830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:08:43.399830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:08:43.399830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:08:43.399830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:08:43.399830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:08:43.399830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:08:43.399830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:08:43.399830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:08:43.399830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:08:43.400321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:08:43.400321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:08:43.400321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:08:43.400321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:08:43.400321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:08:43.400321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:43.400321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:08:43.400321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:43.400321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:08:43.400321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:43.400321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:08:43.400321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:43.400321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:08:43.400321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:43.400321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:08:43.400321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:08:43.400321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:43.400321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:08:43.400822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:43.400822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:08:43.400822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:08:43.400822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:43.400822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:08:43.400822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:43.400822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:08:43.412328 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:08:43.412710 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:08:43.412930 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:08:43.413135 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:08:43.413696 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:08:43.414201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:43.415262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:08:43.415262 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:43.415262 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:08:43.415262 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:08:43.415262 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:43.415262 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:08:43.415262 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:43.415262 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:08:43.416451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:43.416451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:08:43.416451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:08:43.416451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:08:43.416451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:43.416451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:08:43.416451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:43.416451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:08:43.416451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:43.416451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:08:43.416451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:43.416451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:08:43.416451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:43.416451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:08:43.416451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:08:43.417082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:43.417082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:08:43.417082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:43.417082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:08:43.417082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:08:43.417082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:43.417082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:08:43.417082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:43.417082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:08:43.417858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:43.417858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:08:43.417858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:08:43.417858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:08:43.417858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:08:43.417858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:08:43.417858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:08:43.417858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:08:43.417858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:08:43.417858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:43.417858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:08:43.418363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:43.418363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:08:43.418363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:43.418363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:08:43.418363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:43.418363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:08:43.418363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:43.418363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:08:43.418363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:08:43.418363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:43.418363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:08:43.418363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:43.418363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:08:43.418363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:08:43.418982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:43.418982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:08:43.418982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:43.418982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:08:43.419574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:43.419574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:08:43.419574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:08:43.419574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:08:43.419574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:08:43.419574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:08:43.419574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:08:43.419574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:08:43.419574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:08:43.419574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:08:43.419574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:08:43.419574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:08:43.419574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:08:43.419574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:08:43.419574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:08:43.419574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:08:43.419574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:08:43.420304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:08:43.420304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:08:43.420304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:08:43.420304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:43.420304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:08:43.420304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:43.420304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:08:43.420304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:43.420304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:08:43.420304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:43.420304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:08:43.420304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:43.420304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:08:43.420304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:08:43.420304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:43.420304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:08:43.420304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:43.420304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:08:43.420304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:08:43.421095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:43.421095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:08:43.421095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:43.421095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:08:43.421095 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:08:43.421323 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:08:43.421486 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:08:43.421720 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:08:43.422246 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:08:43.422495 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:08:43.422638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:43.423291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:08:43.423291 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:43.423291 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:08:43.423291 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:08:43.423291 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:43.423291 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:08:43.423291 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:43.423291 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:08:43.424158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:43.424158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:08:43.424158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:08:43.424158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:08:43.424158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:08:43.424158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:08:43.424158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:08:43.424158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:43.424158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:08:43.424158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:43.424158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:08:43.424158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:43.424481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:08:43.424481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:43.424481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:08:43.424481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:43.424481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:08:43.424481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:08:43.424481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:43.424481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:08:43.424481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:43.424481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:08:43.424481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:08:43.424481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:43.424481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:08:43.424810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:43.424810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:08:43.425124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:43.425124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:08:43.425124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:08:43.425124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:08:43.425124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:08:43.425124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:08:43.425124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:08:43.425124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:08:43.425124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:08:43.425124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:08:43.425124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:08:43.425124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:08:43.425124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:08:43.425124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:08:43.425124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:08:43.425124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:08:43.425528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:08:43.425528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:08:43.425528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:43.425528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:08:43.425528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:43.425528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:08:43.425528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:43.425528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:08:43.425528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:43.425528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:08:43.425528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:43.425528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:08:43.425528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:08:43.425528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:43.425528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:08:43.425528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:43.425528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:08:43.425528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:08:43.425528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:43.426039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:08:43.426039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:43.426039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:08:47.869729 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:08:47.871171 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:08:47.871404 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:08:47.873056 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:08:47.904575 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:08:47.905066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:47.905998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:08:47.905998 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:47.905998 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:08:47.905998 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:08:47.905998 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:47.905998 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:08:47.905998 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:47.905998 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:08:47.906730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:47.906730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:08:47.906730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:08:47.906730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:08:47.906730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:08:47.906730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:08:47.906730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:08:47.906730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:47.906730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:08:47.906730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:47.906730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:08:47.906730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:47.907155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:08:47.907155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:47.907155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:08:47.907155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:47.907155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:08:47.907155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:08:47.907155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:47.907155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:08:47.907155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:47.907155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:08:47.907155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:08:47.907155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:47.907155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:08:47.907517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:47.907517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:08:47.907517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:47.907517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:08:47.907517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:08:47.907517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:08:47.907517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:08:47.907517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:08:47.907517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:08:47.907517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:08:47.907517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:08:47.907517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:08:47.907517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:08:47.907517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:08:47.907517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:08:47.908023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:08:47.908023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:08:47.908023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:08:47.908023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:08:47.908023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:08:47.908023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:47.908023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:08:47.908023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:47.908023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:08:47.908023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:47.908023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:08:47.908023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:47.908023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:08:47.908023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:47.908023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:08:47.908023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:08:47.908023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:47.908023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:08:47.908023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:47.908545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:08:47.908545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:08:47.908545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:47.908545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:08:47.908545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:47.908545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:08:48.382690 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:08:48.383468 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:08:48.383740 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:08:48.384912 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:08:48.385246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:48.386072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:08:48.386072 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:48.386072 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:08:48.386072 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:08:48.386072 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:48.386072 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:08:48.386072 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:48.386072 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:08:48.388681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:48.388681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:08:48.388681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:08:48.388681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:08:48.388681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:08:48.388681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:08:48.388681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:08:48.388681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:48.388681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:08:48.388681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:48.388681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:08:48.388681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:48.389516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:08:48.389516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:48.389516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:08:48.389516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:48.389516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:08:48.389516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:08:48.389516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:48.389516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:08:48.389516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:48.389516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:08:48.389516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:08:48.389516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:48.389516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:08:48.390192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:48.390192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:08:48.390192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:48.390192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:08:48.390192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:08:48.390192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:08:48.390192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:08:48.390192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:08:48.390192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:08:48.390192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:08:48.390192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:08:48.390192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:08:48.390192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:08:48.390192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:08:48.390192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:08:48.390873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:08:48.390873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:08:48.390873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:08:48.390873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:08:48.390873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:08:48.390873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:48.390873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:08:48.390873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:48.390873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:08:48.390873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:48.390873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:08:48.390873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:48.390873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:08:48.390873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:48.390873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:08:48.390873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:08:48.390873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:48.390873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:08:48.390873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:48.391965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:08:48.391965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:08:48.391965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:48.391965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:08:48.391965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:48.391965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:08:48.392879 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:08:48.393199 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:08:48.393336 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:08:48.393573 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:08:48.394149 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:08:48.394454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:48.395262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:08:48.395262 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:48.395262 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:08:48.395262 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:08:48.395262 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:48.395262 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:08:48.395262 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:48.395262 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:08:48.395989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:48.395989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:08:48.395989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:08:48.395989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:08:48.395989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:48.395989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:08:48.395989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:48.395989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:08:48.395989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:48.395989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:08:48.395989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:48.395989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:08:48.395989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:48.395989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:08:48.395989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:08:48.396451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:48.396451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:08:48.396451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:48.396451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:08:48.396451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:08:48.396451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:48.396451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:08:48.396451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:48.396451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:08:48.396855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:48.396855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:08:48.396855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:08:48.396855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:08:48.396855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:08:48.396855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:08:48.396855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:08:48.396855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:08:48.396855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:08:48.396855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:08:48.396855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:48.396855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:08:48.397234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:48.397234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:08:48.397234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:48.397234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:08:48.397234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:48.397234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:08:48.397234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:48.397234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:08:48.397234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:08:48.397234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:48.397234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:08:48.397234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:48.397234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:08:48.397234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:08:48.397636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:48.397636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:08:48.397636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:48.397636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:08:48.398143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:48.398143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:08:48.398143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:08:48.398143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:08:48.398143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:08:48.398143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:08:48.398143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:08:48.398143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:08:48.398143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:08:48.398143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:08:48.398143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:08:48.398143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:08:48.398143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:08:48.398143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:08:48.398143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:08:48.398143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:08:48.398595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:08:48.398595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:08:48.398595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:08:48.398595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:08:48.398595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:08:48.398595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:48.398595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:08:48.398595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:48.398595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:08:48.398595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:48.398595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:08:48.398595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:48.398595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:08:48.398595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:48.398595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:08:48.398595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:08:48.398595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:48.398595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:08:48.399145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:48.399145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:08:48.399145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:08:48.399145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:48.399145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:08:48.399145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:48.399145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:08:48.412854 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:08:48.413256 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:08:48.413446 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:08:48.413712 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:08:48.414221 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:08:48.414637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:48.415377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:08:48.415377 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:48.415377 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:08:48.415377 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:08:48.415377 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:48.415377 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:08:48.415377 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:48.415377 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:08:48.416567 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:48.416567 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:08:48.416567 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:08:48.416567 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:08:48.416567 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:48.416567 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:08:48.416567 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:48.416567 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:08:48.416567 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:48.416567 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:08:48.416567 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:48.416567 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:08:48.416567 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:48.416567 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:08:48.416567 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:08:48.417300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:48.417300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:08:48.417300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:48.417300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:08:48.417300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:08:48.417300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:48.417300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:08:48.417300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:48.417300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:08:48.418332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:48.418332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:08:48.418332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:08:48.418332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:08:48.418332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:08:48.418332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:08:48.418332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:08:48.418332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:08:48.418332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:08:48.418332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:48.418332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:08:48.418920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:48.418920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:08:48.418920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:48.418920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:08:48.418920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:48.418920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:08:48.418920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:48.418920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:08:48.418920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:08:48.418920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:48.418920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:08:48.418920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:48.418920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:08:48.418920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:08:48.419647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:48.419647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:08:48.419647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:48.419647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:08:48.421208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:48.421208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:08:48.421208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:08:48.421208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:08:48.421208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:08:48.421208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:08:48.421208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:08:48.421208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:08:48.421208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:08:48.421208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:08:48.421208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:08:48.421208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:08:48.421208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:08:48.421208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:08:48.421208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:08:48.421208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:08:48.421208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:08:48.421932 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:08:48.421932 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:08:48.421932 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:08:48.421932 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:48.421932 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:08:48.421932 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:48.421932 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:08:48.421932 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:48.421932 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:08:48.421932 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:48.421932 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:08:48.421932 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:48.421932 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:08:48.421932 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:08:48.421932 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:48.421932 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:08:48.421932 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:48.421932 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:08:48.421932 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:08:48.422531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:48.422531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:08:48.422531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:48.422531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:08:48.422531 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:08:48.422531 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:08:48.422531 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:08:48.422827 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:08:48.423430 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:08:48.423932 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:08:48.424208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:48.425128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:08:48.425128 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:48.425128 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:08:48.425128 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:08:48.425128 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:48.425128 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:08:48.425128 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:48.425128 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:08:48.425963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:48.425963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:08:48.425963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:08:48.425963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:08:48.425963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:08:48.425963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:08:48.425963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:08:48.425963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:48.425963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:08:48.425963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:48.425963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:08:48.425963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:48.426566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:08:48.426566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:48.426566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:08:48.426566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:48.426566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:08:48.426566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:08:48.426566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:48.426566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:08:48.426566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:48.426566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:08:48.426566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:08:48.426566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:48.426566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:08:48.427169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:48.427169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:08:48.427169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:48.427169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:08:48.427169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:08:48.427169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:08:48.427169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:08:48.427169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:08:48.427169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:08:48.427169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:08:48.427169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:08:48.427169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:08:48.427169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:08:48.427169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:08:48.427169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:08:48.427949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:08:48.427949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:08:48.427949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:08:48.427949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:08:48.427949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:08:48.427949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:48.427949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:08:48.427949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:48.427949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:08:48.427949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:48.427949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:08:48.427949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:48.427949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:08:48.427949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:48.427949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:08:48.427949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:08:48.427949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:48.427949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:08:48.427949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:48.428807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:08:48.428807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:08:48.428807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:48.428807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:08:48.428807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:48.428807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:08:52.871056 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:08:52.871788 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:08:52.871927 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:08:52.873247 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:08:52.902184 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:08:52.902326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:52.902777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:08:52.902777 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:52.902777 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:08:52.902777 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:08:52.902777 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:52.902777 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:08:52.902777 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:52.902777 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:08:52.903454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.031 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:52.903454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:08:52.903454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:08:52.903454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:08:52.903454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:08:52.903454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:08:52.903454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:08:52.903454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:52.903454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:08:52.903454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:52.903454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:08:52.903454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:52.904403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:08:52.904403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:52.904403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:08:52.904403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:52.904403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:08:52.904403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:08:52.904403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:52.904403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:08:52.904403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:52.904403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:08:52.904403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:08:52.904403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:52.904403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:08:52.905298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:52.905298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:08:52.905298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:52.905298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:08:52.905298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:08:52.905298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:08:52.905298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:08:52.905298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:08:52.905298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:08:52.905298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:08:52.905298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:08:52.905298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:08:52.905298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:08:52.905298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:08:52.905298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:08:52.906341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:08:52.906341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:08:52.906341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:08:52.906341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:08:52.906341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:08:52.906341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:52.906341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:08:52.906341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:52.906341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:08:52.906341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:52.906341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:08:52.906341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:52.906341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:08:52.906341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:52.906341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:08:52.906341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:08:52.906341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:52.906341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:08:52.906341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:52.907602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:08:52.907602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:08:52.907602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:52.907602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:08:52.907602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:52.907602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:08:53.383444 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:08:53.384402 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:08:53.384609 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:08:53.385648 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:08:53.385854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:53.386833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:08:53.386833 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:53.386833 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:08:53.386833 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:08:53.386833 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:53.386833 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:08:53.386833 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:53.386833 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:08:53.388780 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:53.388780 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:08:53.388780 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:08:53.388780 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:08:53.388780 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:08:53.388780 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:08:53.388780 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:08:53.388780 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:53.388780 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:08:53.388780 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:53.388780 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:08:53.388780 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:53.389928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:08:53.389928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:53.389928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:08:53.389928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:53.389928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:08:53.389928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:08:53.389928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:53.389928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:08:53.389928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:53.389928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:08:53.389928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:08:53.389928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:53.389928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:08:53.390555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:53.390555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:08:53.390555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:53.390555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:08:53.390555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:08:53.390555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:08:53.390555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:08:53.390555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:08:53.390555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:08:53.390555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:08:53.390555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:08:53.390555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:08:53.390555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:08:53.390555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:08:53.390555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:08:53.391250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:08:53.391250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:08:53.391250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:08:53.391250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:08:53.391250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:08:53.391250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:53.391250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:08:53.391250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:53.391250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:08:53.391250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:53.391250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:08:53.391250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:53.391250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:08:53.391250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:53.391250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:08:53.391250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:08:53.391250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:53.391250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:08:53.391250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:53.392064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:08:53.392064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:08:53.392064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:53.392064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:08:53.392064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:53.392064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:08:53.393324 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:08:53.393690 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:08:53.393828 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:08:53.393950 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:08:53.394373 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:08:53.394546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:53.395098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:08:53.395098 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:53.395098 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:08:53.395098 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:08:53.395098 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:53.395098 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:08:53.395098 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:53.395098 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:08:53.395956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:53.395956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:08:53.395956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:08:53.395956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:08:53.395956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:53.395956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:08:53.395956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:53.395956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:08:53.395956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:53.395956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:08:53.395956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:53.395956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:08:53.395956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:53.395956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:08:53.395956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:08:53.396423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:53.396423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:08:53.396423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:53.396423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:08:53.396423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:08:53.396423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:53.396423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:08:53.396423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:53.396423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:08:53.396791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:53.396791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:08:53.396791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:08:53.396791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:08:53.396791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:08:53.396791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:08:53.396791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:08:53.396791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:08:53.396791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:08:53.396791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:08:53.396791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:53.396791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:08:53.397344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:53.397344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:08:53.397344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:53.397344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:08:53.397344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:53.397344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:08:53.397344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:53.397344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:08:53.397344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:08:53.397344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:53.397344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:08:53.397344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:53.397344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:08:53.397344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:08:53.397697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:53.397697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:08:53.397697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:53.397697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:08:53.398145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:53.398145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:08:53.398145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:08:53.398145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:08:53.398145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:08:53.398145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:08:53.398145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:08:53.398145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:08:53.398145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:08:53.398145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:08:53.398145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:08:53.398145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:08:53.398145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:08:53.398145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:08:53.398145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:08:53.398145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:08:53.398826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:08:53.398826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:08:53.398826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:08:53.398826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:08:53.398826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:08:53.398826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:53.398826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:08:53.398826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:53.398826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:08:53.398826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:53.398826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:08:53.398826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:53.398826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:08:53.398826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:53.398826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:08:53.398826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:08:53.398826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:53.398826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:08:53.399887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:53.399887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:08:53.399887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:08:53.399887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:53.399887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:08:53.399887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:53.399887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:08:53.413355 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:08:53.413866 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:08:53.414107 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:08:53.414382 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:08:53.415120 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:08:53.415371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:53.416137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:08:53.416137 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:53.416137 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:08:53.416137 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:08:53.416137 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:53.416137 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:08:53.416137 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:53.416137 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:08:53.417517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:53.417517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:08:53.417517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:08:53.417517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:08:53.417517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:53.417517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:08:53.417517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:53.417517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:08:53.417517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:53.417517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:08:53.417517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:53.417517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:08:53.417517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:53.417517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:08:53.417517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:08:53.418180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:53.418180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:08:53.418180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:53.418180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:08:53.418180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:08:53.418180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:53.418180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:08:53.418180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:53.418180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:08:53.418689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:53.418689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:08:53.418689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:08:53.418689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:08:53.418689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:08:53.418689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:08:53.418689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:08:53.418689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:08:53.418689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:08:53.418689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:53.418689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:08:53.419203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:53.419203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:08:53.419203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:53.419203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:08:53.419203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:53.419203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:08:53.419203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:53.419203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:08:53.419203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:08:53.419203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:53.419203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:08:53.419203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:53.419203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:08:53.419203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:08:53.419807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:53.419807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:08:53.419807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:53.419807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:08:53.419807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:53.419807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:08:53.419807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:08:53.419807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:08:53.419807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:08:53.419807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:08:53.419807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:08:53.419807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:08:53.419807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:08:53.419807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:08:53.420445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:08:53.420445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:08:53.420445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:08:53.420445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:08:53.420445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:08:53.420445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:08:53.420445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:08:53.420445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:08:53.420445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:08:53.420445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:08:53.420445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:53.420445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:08:53.420445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:53.420445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:08:53.420445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:53.420445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:08:53.420445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:53.420445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:08:53.420445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:53.421355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:08:53.421355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:08:53.421355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:53.421355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:08:53.421355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:53.421355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:08:53.421355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:08:53.421355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:53.421355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:08:53.421355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:53.421355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:08:53.422213 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:08:53.422433 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:08:53.422533 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:08:53.422695 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:08:53.423028 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:08:53.423243 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:08:53.423331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:53.423891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:08:53.423891 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:53.423891 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:08:53.423891 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:08:53.423891 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:53.423891 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:08:53.423891 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:53.423891 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:08:53.424774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:53.424774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:08:53.424774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:08:53.424774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:08:53.424774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:08:53.424774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:08:53.424774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:08:53.424774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:53.424774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:08:53.424774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:53.424774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:08:53.424774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:53.425409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:08:53.425409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:53.425409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:08:53.425409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:53.425409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:08:53.425409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:08:53.425409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:53.425409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:08:53.425409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:53.425409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:08:53.425409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:08:53.425409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:53.425409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:08:53.426024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:53.426024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:08:53.426024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:53.426024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:08:53.426024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:08:53.426024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:08:53.426024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:08:53.426024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:08:53.426024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:08:53.426024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:08:53.426024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:08:53.426024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:08:53.426024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:08:53.426024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:08:53.426024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:08:53.426727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:08:53.426727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:08:53.426727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:08:53.426727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:08:53.426727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:08:53.426727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:53.426727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:08:53.426727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:53.426727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:08:53.426727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:53.426727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:08:53.426727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:53.426727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:08:53.426727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:53.426727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:08:53.426727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:08:53.426727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:53.426727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:08:53.426727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:53.427683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:08:53.427683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:08:53.427683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:53.427683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:08:53.427683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:53.427683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:08:57.871999 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:08:57.873469 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:08:57.873469 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:08:57.875457 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:08:57.911972 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:08:57.913013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:57.913427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:08:57.913427 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:57.913427 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:08:57.913427 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:08:57.913427 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:57.913427 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:08:57.913427 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:57.913427 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:08:57.914200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:57.914200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:08:57.914200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:08:57.914200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:08:57.914200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:08:57.914200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:08:57.914200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:08:57.914200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:57.914200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:08:57.914200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:57.914200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:08:57.914200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:57.914809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:08:57.914809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:57.914809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:08:57.914809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:57.914809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:08:57.914809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:08:57.914809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:57.914809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:08:57.914809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:57.914809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:08:57.914809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:08:57.914809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:57.914809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:08:57.915344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:57.915344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:08:57.915344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:57.915344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:08:57.915344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:08:57.915344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:08:57.915344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:08:57.915344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:08:57.915344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:08:57.915344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:08:57.915344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:08:57.915344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:08:57.915344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:08:57.915344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:08:57.915344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:08:57.916089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:08:57.916089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:08:57.916089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:08:57.916089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:08:57.916089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:08:57.916089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:57.916089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:08:57.916089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:57.916089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:08:57.916089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:57.916089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:08:57.916089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:57.916089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:08:57.916089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:57.916089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:08:57.916089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:08:57.916089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:57.916089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:08:57.916089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:57.917068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:08:57.917068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:08:57.917068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:57.917068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:08:57.917068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:57.917068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:08:58.384254 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:08:58.384975 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:08:58.385243 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:08:58.386304 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:08:58.386621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:58.387722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:08:58.387722 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:58.387722 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:08:58.387722 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:08:58.387722 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:58.387722 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:08:58.387722 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:58.387722 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:08:58.389447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:58.389447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:08:58.389447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:08:58.389447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:08:58.389447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:08:58.389447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:08:58.389447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:08:58.389447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:58.389447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:08:58.389447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:58.389447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:08:58.389447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:58.390038 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:08:58.390038 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:58.390038 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:08:58.390038 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:58.390038 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:08:58.390038 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:08:58.390038 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:58.390038 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:08:58.390038 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:58.390038 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:08:58.390038 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:08:58.390038 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:58.390038 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:08:58.390454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:58.390454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:08:58.391537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:58.391537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:08:58.391537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:08:58.391537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:08:58.391537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:08:58.391537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:08:58.391537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:08:58.391537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:08:58.391537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:08:58.391537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:08:58.391537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:08:58.391537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:08:58.391537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:08:58.391537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:08:58.391537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:08:58.391537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:08:58.391986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:08:58.391986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:08:58.391986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:58.391986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:08:58.391986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:58.391986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:08:58.391986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:58.391986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:08:58.391986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:58.391986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:08:58.391986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:58.391986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:08:58.391986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:08:58.391986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:58.391986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:08:58.391986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:58.391986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:08:58.391986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:08:58.391986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:58.392542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:08:58.392542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:58.392542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:08:58.393863 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:08:58.394167 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:08:58.394280 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:08:58.394393 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:08:58.394837 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:08:58.395110 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:58.395601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:08:58.395601 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:58.395601 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:08:58.395601 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:08:58.395601 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:58.395601 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:08:58.395601 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:58.395601 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:08:58.396402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:58.396402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:08:58.396402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:08:58.396402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:08:58.396402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:58.396402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:08:58.396402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:58.396402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:08:58.396402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:58.396402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:08:58.396402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:58.396402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:08:58.396402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:58.396402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:08:58.396402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:08:58.397004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:58.397004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:08:58.397004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:58.397004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:08:58.397004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:08:58.397004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:58.397004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:08:58.397004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:58.397004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:08:58.397361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:58.397361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:08:58.397361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:08:58.397361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:08:58.397361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:08:58.397361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:08:58.397361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:08:58.397361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:08:58.397361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:08:58.397361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:08:58.397361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:58.397361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:08:58.397827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:58.397827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:08:58.397827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:58.397827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:08:58.397827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:58.397827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:08:58.397827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:58.397827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:08:58.397827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:08:58.397827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:58.397827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:08:58.397827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:58.397827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:08:58.397827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:08:58.398337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:58.398337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:08:58.398337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:58.398337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:08:58.398337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:58.398337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:08:58.398337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:08:58.398337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:08:58.398337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:08:58.398337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:08:58.398337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:08:58.398337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:08:58.398337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:08:58.398337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:08:58.398896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:08:58.398896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:08:58.398896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:08:58.398896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:08:58.398896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:08:58.398896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:08:58.398896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:08:58.398896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:08:58.398896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:08:58.398896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:08:58.398896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:08:58.398896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:58.398896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:08:58.398896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:58.398896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:08:58.398896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:58.398896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:08:58.398896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:58.398896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:08:58.399657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:58.399657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:08:58.399657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:08:58.399657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:58.399657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:08:58.399657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:58.399657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:08:58.399657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:08:58.399657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:58.399657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:08:58.399657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:58.399657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:08:58.413898 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:08:58.414335 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:08:58.414596 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:08:58.414893 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:08:58.415329 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:08:58.415685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:58.416332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:08:58.416332 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:58.416332 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:08:58.416332 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:08:58.416332 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:58.416332 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:08:58.416332 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:58.416332 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:08:58.417508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:58.417508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:08:58.417508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:08:58.417508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:08:58.417508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:58.417508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:08:58.417508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:58.417508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:08:58.417508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:58.417508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:08:58.417508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:58.417508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:08:58.417508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:58.417508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:08:58.417508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:08:58.418427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:58.418427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:08:58.418427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:58.418427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:08:58.418427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:08:58.418427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:58.418427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:08:58.418427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:58.418427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:08:58.419154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:58.419154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:08:58.419154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:08:58.419154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:08:58.419154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:08:58.419154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:08:58.419154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:08:58.419154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:08:58.419154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:08:58.419154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:58.419154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:08:58.420033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:58.420033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:08:58.420033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:58.420033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:08:58.420033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:58.420033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:08:58.420033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:58.420033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:08:58.420033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:08:58.420033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:58.420033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:08:58.420033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:58.420033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:08:58.420033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:08:58.420979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:58.420979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:08:58.420979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:58.420979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:08:58.420979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:58.420979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:08:58.420979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:08:58.420979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:08:58.420979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:08:58.420979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:08:58.420979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:08:58.420979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:08:58.420979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:08:58.420979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:08:58.422097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:08:58.422097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:08:58.422097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:08:58.422097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:08:58.422097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:08:58.422097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:08:58.422097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:08:58.422097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:08:58.422097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:08:58.422097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:08:58.422097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:58.422097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:08:58.422097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:58.422097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:08:58.422097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:58.422097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:08:58.422097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:58.422097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:08:58.422097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:58.423666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:08:58.423666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:08:58.423666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:58.423666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:08:58.423666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:58.423666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:08:58.423666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:08:58.423666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:58.423666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:08:58.423666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:58.423666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:08:58.423666 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:08:58.423666 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:08:58.424375 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:08:58.424375 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:08:58.424375 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:08:58.424375 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:08:58.424375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:58.425133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:08:58.425133 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:58.425133 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:08:58.425133 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:08:58.425133 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:58.425133 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:08:58.425133 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:58.425133 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:08:58.426065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:58.426065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:08:58.426065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:08:58.426065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:08:58.426065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:08:58.426065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:08:58.426065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:08:58.426065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:58.426065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:08:58.426065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:58.426065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:08:58.426065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:58.426575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:08:58.426575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:58.426575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:08:58.426575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:58.426575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:08:58.426575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:08:58.426575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:58.426575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:08:58.426575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:58.426575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:08:58.426575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:08:58.426575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:58.426575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:08:58.427157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:58.427157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:08:58.427157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:58.427157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:08:58.427157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:08:58.427157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:08:58.427157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:08:58.427157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:08:58.427157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:08:58.427157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:08:58.427157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:08:58.427157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:08:58.427157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:08:58.427157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:08:58.427157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:08:58.427739 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:08:58.427739 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:08:58.427739 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:08:58.427739 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:08:58.427739 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:08:58.427739 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:58.427739 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:08:58.427739 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:08:58.427739 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:08:58.427739 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:08:58.427739 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:08:58.427739 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:08:58.427739 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:08:58.427739 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:08:58.427739 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:08:58.427739 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:08:58.427739 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:08:58.427739 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:08:58.427739 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:08:58.428498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:08:58.428498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:08:58.428498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:08:58.428498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:08:58.428498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:08:58.428498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:09:02.872803 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:09:02.873974 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:09:02.873974 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:09:02.874947 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:09:02.911443 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:09:02.912040 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:02.913365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:09:02.913365 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:02.913365 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:09:02.913365 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:09:02.913365 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:02.913365 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:09:02.913365 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:02.913365 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:09:02.915273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:02.915273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:09:02.915273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:09:02.915273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:09:02.915273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:09:02.915273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:09:02.915273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:09:02.915273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:02.915273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:09:02.915273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:02.915273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:09:02.915273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:02.916702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:09:02.916702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:02.916702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:09:02.916702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:02.916702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:09:02.916702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:09:02.916702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:02.916702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:09:02.916702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:02.916702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:09:02.916702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:09:02.916702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:02.916702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:09:02.918126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:02.918126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:09:02.918126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:02.918126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:09:02.918126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:09:02.918126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:09:02.918126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:09:02.918126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:09:02.918126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:09:02.918126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:09:02.918126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:09:02.918126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:09:02.918126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:09:02.918126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:09:02.918126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:09:02.919545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:09:02.919545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:09:02.919545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:09:02.919545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:09:02.919545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:09:02.919545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:02.919545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:09:02.919545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:02.919545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:09:02.919545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:02.919545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:09:02.919545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:02.919545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:09:02.919545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:02.919545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:09:02.919545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:09:02.919545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:02.919545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:09:02.919545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:02.921249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:09:02.921249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:09:02.921249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:02.921249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:09:02.921249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:02.921249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:09:03.384997 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:09:03.385483 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:09:03.385766 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:09:03.386932 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:09:03.387273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:03.388331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:09:03.388331 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:03.388331 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:09:03.388331 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:09:03.388331 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:03.388331 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:09:03.388331 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:03.388331 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:09:03.389979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:03.389979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:09:03.389979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:09:03.389979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:09:03.389979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:09:03.389979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:09:03.389979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:09:03.389979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:03.389979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:09:03.389979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:03.389979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:09:03.389979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:03.390569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:09:03.390569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:03.390569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:09:03.390569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:03.390569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:09:03.390569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:09:03.390569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:03.390569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:09:03.390569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:03.390569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:09:03.390569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:09:03.390569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:03.390569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:09:03.391103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:03.391103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:09:03.392403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:03.392403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:09:03.392403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:09:03.392403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:09:03.392403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:09:03.392403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:09:03.392403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:09:03.392403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:09:03.392403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:09:03.392403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:09:03.392403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:09:03.392403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:09:03.392403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:09:03.392403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:09:03.392403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:09:03.392403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:09:03.393188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:09:03.393188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:09:03.393188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:03.393188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:09:03.393188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:03.393188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:09:03.393188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:03.393188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:09:03.393188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:03.393188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:09:03.393188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:03.393188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:09:03.393188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:09:03.393188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:03.393188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:09:03.393188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:03.393188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:09:03.393188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:09:03.393188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:03.394136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:09:03.394136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:03.394136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:09:03.394278 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:09:03.394644 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:09:03.394910 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:09:03.395183 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:09:03.395873 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:09:03.396185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:03.397107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:09:03.397107 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:03.397107 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:09:03.397107 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:09:03.397107 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:03.397107 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:09:03.397107 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:03.397107 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:09:03.398067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:03.398067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:09:03.398067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:09:03.398067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:09:03.398067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:03.398067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:09:03.398067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:03.398067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:09:03.398067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:03.398067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:09:03.398067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:03.398067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:09:03.398067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:03.398067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:09:03.398067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:09:03.398892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:03.398892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:09:03.398892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:03.398892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:09:03.398892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:09:03.398892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:03.398892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:09:03.398892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:03.398892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:09:03.399306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:03.399306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:09:03.399306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:09:03.399306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:09:03.399306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:09:03.399306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:09:03.399306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:09:03.399306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:09:03.399306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:09:03.399306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:09:03.399306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:03.399306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:09:03.399950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:03.399950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:09:03.399950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:03.399950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:09:03.399950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:03.399950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:09:03.399950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:03.399950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:09:03.399950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:09:03.399950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:03.399950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:09:03.399950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:03.399950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:09:03.399950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:09:03.400541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:03.400541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:09:03.400541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:03.400541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:09:03.400735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:03.400735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:09:03.400735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:09:03.400735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:09:03.400735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:09:03.400735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:09:03.400735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:09:03.400735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:09:03.400735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:09:03.400735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:09:03.400735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:09:03.400735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:09:03.400735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:09:03.400735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:09:03.400735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:09:03.400735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:09:03.401513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:09:03.401513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:09:03.401513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:09:03.401513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:09:03.401513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:09:03.401513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:03.401513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:09:03.401513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:03.401513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:09:03.401513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:03.401513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:09:03.401513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:03.401513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:09:03.401513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:03.401513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:09:03.401513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:09:03.401513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:03.401513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:09:03.402029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:03.402029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:09:03.402029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:09:03.402029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:03.402029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:09:03.402029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:03.402029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:09:03.414411 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:09:03.414884 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:09:03.415123 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:09:03.415392 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:09:03.416016 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:09:03.416376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:03.418870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:09:03.418870 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:03.418870 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:09:03.418870 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:09:03.418870 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:03.418870 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:09:03.418870 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:03.418870 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:09:03.418870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:03.418870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:09:03.418870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:09:03.418870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:09:03.418870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:03.418870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:09:03.418870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:03.418870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:09:03.418870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:03.418870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:09:03.419840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:03.419840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:09:03.419840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:03.419840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:09:03.419840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:09:03.419840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:03.419840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:09:03.419840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:03.419840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:09:03.419840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:09:03.419840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:03.419840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:09:03.419840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:03.419840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:09:03.420524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:03.420524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:09:03.420524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:09:03.420524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:09:03.420524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:09:03.420524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:09:03.420524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:09:03.420524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:09:03.420524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:09:03.420524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:03.420524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:09:03.421064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:03.421064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:09:03.421064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:03.421064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:09:03.421064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:03.421064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:09:03.421064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:03.421064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:09:03.421064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:09:03.421064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:03.421064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:09:03.421064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:03.421064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:09:03.421064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:09:03.421859 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:03.421859 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:09:03.421859 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:03.421859 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:09:03.422304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:03.422304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:09:03.422304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:09:03.422304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:09:03.422304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:09:03.422304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:09:03.422304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:09:03.422304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:09:03.422304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:09:03.422304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:09:03.422304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:09:03.422304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:09:03.422304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:09:03.422304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:09:03.422304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:09:03.422304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:09:03.422304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:09:03.423659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:09:03.423659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:09:03.423659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:09:03.423659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:03.423659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:09:03.423659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:03.423659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:09:03.423659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:03.423659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:09:03.423659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:03.423659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:09:03.423659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:03.423659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:09:03.423659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:09:03.423659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:03.423659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:09:03.423659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:03.423659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:09:03.423659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:09:03.424540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:03.424540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:09:03.424540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:03.424540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:09:03.424540 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:09:03.424540 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:09:03.424540 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:09:03.424540 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:09:03.424873 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:09:03.424873 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:09:03.425065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:03.425738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:09:03.425738 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:03.425738 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:09:03.425738 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:09:03.425738 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:03.425738 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:09:03.425738 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:03.425738 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:09:03.426565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:03.426565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:09:03.426565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:09:03.426565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:09:03.426565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:09:03.426565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:09:03.426565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:09:03.426565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:03.426565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:09:03.426565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:03.426565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:09:03.426565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:03.426946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:09:03.426946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:03.426946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:09:03.426946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:03.426946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:09:03.426946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:09:03.426946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:03.426946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:09:03.426946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:03.426946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:09:03.426946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:09:03.426946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:03.426946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:09:03.427348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:03.427348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:09:03.427619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:03.427619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:09:03.427619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:09:03.427619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:09:03.427619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:09:03.427619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:09:03.427619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:09:03.427619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:09:03.427619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:09:03.427619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:09:03.427619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:09:03.427619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:09:03.427619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:09:03.427619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:09:03.427619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:09:03.427619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:09:03.428107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:09:03.428107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:09:03.428107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:03.428107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:09:03.428107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:03.428107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:09:03.428107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:03.428107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:09:03.428107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:03.428107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:09:03.428107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:03.428107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:09:03.428107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:09:03.428107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:03.428107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:09:03.428107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:03.428107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:09:03.428107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:09:03.428107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:03.428590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:09:03.428590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:03.428590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:09:07.873868 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:09:07.874670 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:09:07.874670 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:09:07.876376 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:09:07.910272 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:09:07.910683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:07.911651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:09:07.911651 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:07.911651 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:09:07.911651 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:09:07.911651 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:07.911651 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:09:07.911651 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:07.911651 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:09:07.912416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:07.912416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:09:07.912416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:09:07.912416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:09:07.912416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:09:07.912416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:09:07.912416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:09:07.912416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:07.912416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:09:07.912416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:07.912416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:09:07.912416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:07.913148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:09:07.913148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:07.913148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:09:07.913148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:07.913148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:09:07.913148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:09:07.913148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:07.913148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:09:07.913148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:07.913148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:09:07.913148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:09:07.913148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:07.913148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:09:07.913952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:07.913952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:09:07.913952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:07.913952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:09:07.913952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:09:07.913952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:09:07.913952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:09:07.913952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:09:07.913952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:09:07.913952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:09:07.913952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:09:07.913952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:09:07.913952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:09:07.913952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:09:07.913952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:09:07.914705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:09:07.914705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:09:07.914705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:09:07.914705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:09:07.914705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:09:07.914705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:07.914705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:09:07.914705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:07.914705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:09:07.914705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:07.914705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:09:07.914705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:07.914705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:09:07.914705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:07.914705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:09:07.914705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:09:07.914705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:07.914705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:09:07.914705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:07.915596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:09:07.915596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:09:07.915596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:07.915596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:09:07.915596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:07.915596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:09:08.385790 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:09:08.386465 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:09:08.386681 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:09:08.388100 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:09:08.388420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:08.389598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:09:08.389598 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:08.389598 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:09:08.389598 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:09:08.389598 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:08.389598 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:09:08.389598 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:08.389598 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:09:08.391876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:08.391876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:09:08.391876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:09:08.391876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:09:08.391876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:09:08.391876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:09:08.391876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:09:08.391876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:08.391876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:09:08.391876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:08.391876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:09:08.391876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:08.392937 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:09:08.392937 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:08.392937 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:09:08.392937 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:08.392937 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:09:08.392937 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:09:08.392937 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:08.392937 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:09:08.392937 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:08.392937 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:09:08.392937 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:09:08.392937 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:08.392937 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:09:08.393635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:08.393635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:09:08.394218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:08.394218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:09:08.394218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:09:08.394218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:09:08.394218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:09:08.394218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:09:08.394218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:09:08.394218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:09:08.394218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:09:08.394218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:09:08.394218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:09:08.394218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:09:08.394218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:09:08.394218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:09:08.394218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:09:08.394218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:09:08.395144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:09:08.395144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:09:08.395144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:08.395144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:09:08.395144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:08.395144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:09:08.395144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:08.395144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:09:08.395144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:08.395144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:09:08.395144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:08.395144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:09:08.395144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:09:08.395144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:08.395144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:09:08.395144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:08.395144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:09:08.395144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:09:08.395144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:08.396106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:09:08.396106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:08.396106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:09:08.396106 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:09:08.396106 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:09:08.396106 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:09:08.396106 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:09:08.396106 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:09:08.396523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:08.396946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:09:08.396946 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:08.396946 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:09:08.396946 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:09:08.396946 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:08.396946 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:09:08.396946 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:08.396946 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:09:08.397725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:08.397725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:09:08.397725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:09:08.397725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:09:08.397725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:08.397725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:09:08.397725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:08.397725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:09:08.397725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:08.397725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:09:08.397725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:08.397725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:09:08.397725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:08.397725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:09:08.397725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:09:08.398464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:08.398464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:09:08.398464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:08.398464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:09:08.398464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:09:08.398464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:08.398464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:09:08.398464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:08.398464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:09:08.398935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:08.398935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:09:08.398935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:09:08.398935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:09:08.398935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:09:08.398935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:09:08.398935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:09:08.398935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:09:08.398935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:09:08.398935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:09:08.398935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:08.398935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:09:08.399496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:08.399496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:09:08.399496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:08.399496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:09:08.399496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:08.399496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:09:08.399496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:08.399496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:09:08.399496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:09:08.399496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:08.399496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:09:08.399496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:08.399496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:09:08.399496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:09:08.400159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:08.400159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:09:08.400159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:08.400159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:09:08.400159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:08.400159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:09:08.400159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:09:08.400159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:09:08.400159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:09:08.400159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:09:08.400159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:09:08.400159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:09:08.400159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:09:08.400159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:09:08.400810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:09:08.400810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:09:08.400810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:09:08.400810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:09:08.400810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:09:08.400810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:09:08.400810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:09:08.400810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:09:08.400810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:09:08.400810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:09:08.400810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:09:08.400810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:08.400810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:09:08.400810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:08.400810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:09:08.400810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:08.400810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:09:08.400810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:08.400810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:09:08.401741 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:08.401741 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:09:08.401741 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:09:08.401741 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:08.401741 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:09:08.401741 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:08.401741 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:09:08.401741 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:09:08.401741 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:08.401741 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:09:08.401741 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:08.401741 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:09:08.414966 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:09:08.415358 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:09:08.415571 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:09:08.415773 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:09:08.416329 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:09:08.416680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:08.417539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:09:08.417539 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:08.417539 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:09:08.417539 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:09:08.417539 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:08.417539 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:09:08.417539 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:08.417539 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:09:08.418813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:08.418813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:09:08.418813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:09:08.418813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:09:08.418813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:08.418813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:09:08.418813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:08.418813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:09:08.418813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:08.418813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:09:08.418813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:08.418813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:09:08.418813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:08.418813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:09:08.418813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:09:08.419256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:08.419256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:09:08.419256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:08.419256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:09:08.419256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:09:08.419256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:08.419256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:09:08.419256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:08.419256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:09:08.420393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:08.420393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:09:08.420393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:09:08.420393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:09:08.420393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:09:08.420393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:09:08.420393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:09:08.420393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:09:08.420393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:09:08.420393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:08.420393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:09:08.420726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:08.420726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:09:08.420726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:08.420726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:09:08.420726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:08.420726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:09:08.420726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:08.420726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:09:08.420726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:09:08.420726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:08.420726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:09:08.420726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:08.420726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:09:08.420726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:09:08.421130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:08.421130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:09:08.421130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:08.421130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:09:08.422367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:08.422367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:09:08.422367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:09:08.422367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:09:08.422367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:09:08.422367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:09:08.422367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:09:08.422367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:09:08.422367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:09:08.422367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:09:08.422367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:09:08.422367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:09:08.422367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:09:08.422367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:09:08.422367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:09:08.422367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:09:08.422367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:09:08.422879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:09:08.422879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:09:08.422879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:09:08.422879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:08.422879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:09:08.422879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:08.422879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:09:08.422879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:08.422879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:09:08.422879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:08.422879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:09:08.422879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:08.422879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:09:08.422879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:09:08.422879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:08.422879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:09:08.422879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:08.422879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:09:08.422879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:09:08.423396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:08.423396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:09:08.423396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:08.423396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:09:08.423677 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:09:08.424033 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:09:08.424209 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:09:08.424428 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:09:08.425026 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:09:08.425441 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:09:08.425649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:08.426766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:09:08.426766 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:08.426766 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:09:08.426766 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:09:08.426766 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:08.426766 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:09:08.426766 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:08.426766 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:09:08.428103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:08.428103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:09:08.428103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:09:08.428103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:09:08.428103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:09:08.428103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:09:08.428103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:09:08.428103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:08.428103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:09:08.428103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:08.428103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:09:08.428103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:08.428464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:09:08.428464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:08.428464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:09:08.428464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:08.428464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:09:08.428464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:09:08.428464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:08.428464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:09:08.428464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:08.428464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:09:08.428464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:09:08.428464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:08.428464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:09:08.428822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:08.428822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:09:08.429087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:08.429087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:09:08.429087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:09:08.429087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:09:08.429087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:09:08.429087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:09:08.429087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:09:08.429087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:09:08.429087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:09:08.429087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:09:08.429087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:09:08.429087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:09:08.429087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:09:08.429087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:09:08.429087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:09:08.429087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:09:08.429531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:09:08.429531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:09:08.429531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:08.429531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:09:08.429531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:08.429531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:09:08.429531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:08.429531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:09:08.429531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:08.429531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:09:08.429531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:08.429531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:09:08.429531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:09:08.429531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:08.429531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:09:08.429531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:08.429531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:09:08.429531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:09:08.429531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:08.430125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:09:08.430125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:08.430125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:09:12.874266 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:09:12.875364 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:09:12.875364 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:09:12.875821 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:09:12.909692 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:09:12.910157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:12.911154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:09:12.911154 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:12.911154 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:09:12.911154 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:09:12.911154 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:12.911154 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:09:12.911154 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:12.911154 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:09:12.912230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:12.912230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:09:12.912230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:09:12.912230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:09:12.912230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:09:12.912230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:09:12.912230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:09:12.912230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:12.912230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:09:12.912230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:12.912230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:09:12.912230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:12.912803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:09:12.912803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:12.912803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:09:12.912803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:12.912803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:09:12.912803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:09:12.912803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:12.912803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:09:12.912803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:12.912803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:09:12.912803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:09:12.912803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:12.912803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:09:12.913590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:12.913590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:09:12.913590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:12.913590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:09:12.913590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:09:12.913590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:09:12.913590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:09:12.913590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:09:12.913590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:09:12.913590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:09:12.913590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:09:12.913590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:09:12.913590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:09:12.913590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:09:12.913590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:09:12.914354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:09:12.914354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:09:12.914354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:09:12.914354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:09:12.914354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:09:12.914354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:12.914354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:09:12.914354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:12.914354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:09:12.914354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:12.914354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:09:12.914354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:12.914354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:09:12.914354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:12.914354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:09:12.914354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:09:12.914354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:12.914354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:09:12.914354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:12.915506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:09:12.915506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:09:12.915506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:12.915506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:09:12.915506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:12.915506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:09:13.386544 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:09:13.387266 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:09:13.387466 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:09:13.388641 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:09:13.389184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:13.390252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:09:13.390252 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:13.390252 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:09:13.390252 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:09:13.390252 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:13.390252 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:09:13.390252 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:13.390252 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:09:13.391438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:13.391438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:09:13.391438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:09:13.391438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:09:13.391438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:09:13.391438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:09:13.391438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:09:13.391438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:13.391438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:09:13.391438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:13.391438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:09:13.391438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:13.392243 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:09:13.392243 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:13.392243 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:09:13.392243 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:13.392243 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:09:13.392243 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:09:13.392243 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:13.392243 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:09:13.392243 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:13.392243 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:09:13.392243 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:09:13.392243 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:13.392243 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:09:13.392999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:13.392999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:09:13.392999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:13.392999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:09:13.392999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:09:13.392999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:09:13.392999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:09:13.392999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:09:13.392999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:09:13.392999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:09:13.392999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:09:13.392999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:09:13.392999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:09:13.392999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:09:13.392999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:09:13.393805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:09:13.393805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:09:13.393805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:09:13.393805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:09:13.393805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:09:13.393805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:13.393805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:09:13.393805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:13.393805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:09:13.393805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:13.393805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:09:13.393805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:13.393805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:09:13.393805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:13.393805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:09:13.393805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:09:13.393805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:13.393805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:09:13.393805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:13.394974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:09:13.394974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:09:13.394974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:13.394974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:09:13.394974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:13.394974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:09:13.395290 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:09:13.395444 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:09:13.395528 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:09:13.395666 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:09:13.396167 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:09:13.396481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:13.397277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:09:13.397277 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:13.397277 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:09:13.397277 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:09:13.397277 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:13.397277 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:09:13.397277 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:13.397277 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:09:13.398023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:13.398023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:09:13.398023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:09:13.398023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:09:13.398023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:13.398023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:09:13.398023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:13.398023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:09:13.398023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:13.398023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:09:13.398023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:13.398023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:09:13.398023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:13.398023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:09:13.398023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:09:13.398472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:13.398472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:09:13.398472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:13.398472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:09:13.398472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:09:13.398472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:13.398472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:09:13.398472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:13.398472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:09:13.398842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:13.398842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:09:13.398842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:09:13.398842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:09:13.398842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:09:13.398842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:09:13.398842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:09:13.398842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:09:13.398842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:09:13.398842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:09:13.398842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:13.398842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:09:13.399193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:13.399193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:09:13.399193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:13.399193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:09:13.399193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:13.399193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:09:13.399193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:13.399193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:09:13.399193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:09:13.399193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:13.399193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:09:13.399193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:13.399193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:09:13.399193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:09:13.399548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:13.399548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:09:13.399548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:13.399548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:09:13.399850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:13.399850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:09:13.399850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:09:13.399850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:09:13.399850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:09:13.399850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:09:13.399850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:09:13.399850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:09:13.399850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:09:13.399850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:09:13.399850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:09:13.399850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:09:13.399850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:09:13.399850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:09:13.399850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:09:13.399850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:09:13.400291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:09:13.400291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:09:13.400291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:09:13.400291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:09:13.400291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:09:13.400291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:13.400291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:09:13.400291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:13.400291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:09:13.400291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:13.400291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:09:13.400291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:13.400291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:09:13.400291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:13.400291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:09:13.400291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:09:13.400291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:13.400291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:09:13.400867 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:13.400867 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:09:13.400867 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:09:13.400867 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:13.400867 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:09:13.400867 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:13.400867 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:09:13.415360 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:09:13.415732 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:09:13.416029 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:09:13.416247 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:09:13.416950 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:09:13.417386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:13.418227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:09:13.418227 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:13.418227 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:09:13.418227 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:09:13.418227 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:13.418227 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:09:13.418227 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:13.418227 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:09:13.419401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:13.419401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:09:13.419401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:09:13.419401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:09:13.419401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:13.419401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:09:13.419401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:13.419401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:09:13.419401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:13.419401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:09:13.419401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:13.419401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:09:13.419401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:13.419401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:09:13.419401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:09:13.420094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:13.420094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:09:13.420094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:13.420094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:09:13.420094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:09:13.420094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:13.420094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:09:13.420094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:13.420094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:09:13.420861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:13.420861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:09:13.420861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:09:13.420861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:09:13.420861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:09:13.420861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:09:13.420861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:09:13.420861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:09:13.420861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:09:13.420861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:13.420861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:09:13.421481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:13.421481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:09:13.421481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:13.421481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:09:13.421481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:13.421481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:09:13.421481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:13.421481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:09:13.421481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:09:13.421481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:13.421481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:09:13.421481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:13.421481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:09:13.421481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:09:13.422073 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:13.422073 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:09:13.422073 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:13.422073 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:09:13.422571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:13.422571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:09:13.422571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:09:13.422571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:09:13.422571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:09:13.422571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:09:13.422571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:09:13.422571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:09:13.422571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:09:13.422571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:09:13.422571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:09:13.422571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:09:13.422571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:09:13.422571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:09:13.422571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:09:13.422571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:09:13.422571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:09:13.423321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:09:13.423321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:09:13.423321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:09:13.423321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:13.423321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:09:13.423321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:13.423321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:09:13.423321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:13.423321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:09:13.423321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:13.423321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:09:13.423321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:13.423321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:09:13.423321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:09:13.423321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:13.423321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:09:13.423321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:13.423321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:09:13.423321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:09:13.423799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:13.423799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:09:13.423799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:13.423799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:09:13.424136 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:09:13.424508 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:09:13.424685 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:09:13.425128 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:09:13.425741 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:09:13.426281 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:09:13.426495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:13.427328 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:09:13.427328 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:13.427328 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:09:13.427328 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:09:13.427328 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:13.427328 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:09:13.427328 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:13.427328 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:09:13.428220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:13.428220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:09:13.428220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:09:13.428220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:09:13.428220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:09:13.428220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:09:13.428220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:09:13.428220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:13.428220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:09:13.428220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:13.428220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:09:13.428220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:13.428584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:09:13.428584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:13.428584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:09:13.428584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:13.428584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:09:13.428584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:09:13.428584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:13.428584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:09:13.428584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:13.428584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:09:13.428584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:09:13.428584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:13.428584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:09:13.428966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:13.428966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:09:13.429416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:13.429416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:09:13.429416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:09:13.429416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:09:13.429416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:09:13.429416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:09:13.429416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:09:13.429416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:09:13.429416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:09:13.429416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:09:13.429416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:09:13.429416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:09:13.429416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:09:13.429416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:09:13.429416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:09:13.429416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:09:13.429977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:09:13.429977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:09:13.429977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:13.429977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:09:13.429977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:13.429977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:09:13.429977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:13.429977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:09:13.429977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:13.429977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:09:13.429977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:13.429977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:09:13.429977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:09:13.429977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:13.429977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:09:13.429977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:13.429977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:09:13.429977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:09:13.429977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:13.430597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:09:13.430597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:13.430597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:09:17.875444 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:09:17.876211 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:09:17.876338 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:09:17.877270 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:09:17.908995 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:09:17.908995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:17.912047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:09:17.912047 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:17.912047 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:09:17.912047 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:09:17.912047 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:17.912047 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:09:17.912047 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:17.912047 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:09:17.912047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:17.912047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:09:17.912047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:09:17.912047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:09:17.912047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:09:17.912047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:09:17.912047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:09:17.912529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:17.912529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:09:17.912529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:17.912529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:09:17.912529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:17.912529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:09:17.912529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:17.912529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:09:17.912529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:17.912529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:09:17.912529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:09:17.912529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:17.912529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:09:17.912529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:17.912921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:09:17.912921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:09:17.912921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:17.912921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:09:17.912921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:17.912921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:09:17.912921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:17.912921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:09:17.912921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:09:17.912921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:09:17.912921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:09:17.912921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:09:17.912921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:09:17.912921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:09:17.913306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:09:17.913306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:09:17.913306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:09:17.913306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:09:17.913306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:09:17.913306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:09:17.913306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:09:17.913306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:09:17.913306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:09:17.913306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:09:17.913306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:17.913306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:09:17.913306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:17.913306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:09:17.913306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:17.913306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:09:17.913306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:17.913306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:09:17.913306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:17.913306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:09:17.913306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:09:17.913882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:17.913882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:09:17.913882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:17.913882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:09:17.913882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:09:17.913882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:17.913882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:09:17.913882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:17.913882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:09:18.387476 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:09:18.388884 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:09:18.388884 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:09:18.389905 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:09:18.390340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:18.391414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:09:18.391414 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:18.391414 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:09:18.391414 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:09:18.391414 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:18.391414 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:09:18.391414 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:18.391414 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:09:18.393182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:18.393182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:09:18.393182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:09:18.393182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:09:18.393182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:09:18.393182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:09:18.393182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:09:18.393182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:18.393182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:09:18.393182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:18.393182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:09:18.393182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:18.393718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:09:18.393718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:18.393718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:09:18.393718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:18.393718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:09:18.393718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:09:18.393718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:18.393718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:09:18.393718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:18.393718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:09:18.393718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:09:18.393718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:18.393718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:09:18.394344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:18.394344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:09:18.395046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:18.395046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:09:18.395046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:09:18.395046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:09:18.395046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:09:18.395046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:09:18.395046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:09:18.395046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:09:18.395046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:09:18.395046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:09:18.395046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:09:18.395046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:09:18.395046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:09:18.395046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:09:18.395046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:09:18.395046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:09:18.395781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:09:18.395781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:09:18.395781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:18.395781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:09:18.395781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:18.395781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:09:18.395781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:18.395781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:09:18.395781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:18.395781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:09:18.395781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:18.395781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:09:18.395781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:09:18.395781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:18.395781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:09:18.395781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:18.395781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:09:18.395781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:09:18.395781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:18.396695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:09:18.396695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:18.396695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:09:18.396695 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:09:18.396695 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:09:18.396695 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:09:18.396695 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:09:18.397041 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:09:18.397386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:18.398160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:09:18.398160 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:18.398160 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:09:18.398160 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:09:18.398160 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:18.398160 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:09:18.398160 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:18.398160 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:09:18.398995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:18.398995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:09:18.398995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:09:18.398995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:09:18.398995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:18.398995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:09:18.398995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:18.398995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:09:18.398995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:18.398995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:09:18.398995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:18.398995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:09:18.398995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:18.398995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:09:18.398995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:09:18.399435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:18.399435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:09:18.399435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:18.399435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:09:18.399435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:09:18.399435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:18.399435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:09:18.399435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:18.399435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:09:18.399881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:18.399881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:09:18.399881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:09:18.399881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:09:18.399881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:09:18.399881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:09:18.399881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:09:18.399881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:09:18.399881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:09:18.399881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:09:18.399881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:18.399881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:09:18.400271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:18.400271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:09:18.400271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:18.400271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:09:18.400271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:18.400271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:09:18.400271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:18.400271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:09:18.400271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:09:18.400271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:18.400271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:09:18.400271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:18.400271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:09:18.400271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:09:18.400698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:18.400698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:09:18.400698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:18.400698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:09:18.400906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:18.400906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:09:18.400906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:09:18.400906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:09:18.400906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:09:18.400906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:09:18.400906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:09:18.400906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:09:18.400906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:09:18.400906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:09:18.400906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:09:18.400906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:09:18.400906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:09:18.400906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:09:18.400906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:09:18.400906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:09:18.401579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:09:18.401579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:09:18.401579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:09:18.401579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:09:18.401579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:09:18.401579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:18.401579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:09:18.401579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:18.401579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:09:18.401579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:18.401579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:09:18.401579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:18.401579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:09:18.401579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:18.401579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:09:18.401579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:09:18.401579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:18.401579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:09:18.402097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:18.402097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:09:18.402097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:09:18.402097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:18.402097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:09:18.402097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:18.402097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:09:18.416039 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:09:18.416489 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:09:18.416669 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:09:18.416910 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:09:18.417691 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:09:18.418223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:18.419243 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:09:18.419243 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:18.419243 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:09:18.419243 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:09:18.419243 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:18.419243 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:09:18.419243 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:18.419243 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:09:18.420655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:18.420655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:09:18.420655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:09:18.420655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:09:18.420655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:18.420655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:09:18.420655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:18.420655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:09:18.420655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:18.420655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:09:18.420655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:18.420655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:09:18.420655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:18.420655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:09:18.420655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:09:18.421713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:18.421713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:09:18.421713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:18.421713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:09:18.421713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:09:18.421713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:18.421713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:09:18.421713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:18.421713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:09:18.422487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:18.422487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:09:18.422487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:09:18.422487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:09:18.422487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:09:18.422487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:09:18.422487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:09:18.422487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:09:18.422487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:09:18.422487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:18.422487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:09:18.423166 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:18.423166 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:09:18.423166 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:18.423166 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:09:18.423166 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:18.423166 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:09:18.423166 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:18.423166 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:09:18.423166 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:09:18.423166 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:18.423166 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:09:18.423166 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:18.423166 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:09:18.423166 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:09:18.423833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:18.423833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:09:18.423833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:18.423833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:09:18.424084 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:18.424084 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:09:18.424084 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:09:18.424084 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:09:18.424084 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:09:18.424084 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:09:18.424084 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:09:18.424084 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:09:18.424084 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:09:18.424084 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:09:18.424084 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:09:18.424084 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:09:18.424084 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:09:18.424084 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:09:18.424084 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:09:18.424084 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:09:18.424084 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:09:18.424848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:09:18.424848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:09:18.424848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:09:18.424848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:18.424848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:09:18.424848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:18.424848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:09:18.424848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:18.424848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:09:18.424848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:18.424848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:09:18.424848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:18.424848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:09:18.424848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:09:18.424848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:18.424848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:09:18.424848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:18.424848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:09:18.424848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:09:18.425423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:18.425423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:09:18.425423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:18.425423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:09:18.425423 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:09:18.425423 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:09:18.425423 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:09:18.425423 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:09:18.425879 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:09:18.426117 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:09:18.426250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:18.426892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:09:18.426892 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:18.426892 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:09:18.426892 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:09:18.426892 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:18.426892 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:09:18.426892 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:18.426892 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:09:18.429002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:18.429002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:09:18.429002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:09:18.429002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:09:18.429002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:09:18.429002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:09:18.429002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:09:18.429002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:18.429002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:09:18.429002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:18.429002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:09:18.429002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:18.429376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:09:18.429376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:18.429376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:09:18.429376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:18.429376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:09:18.429376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:09:18.429376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:18.429376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:09:18.429376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:18.429376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:09:18.429376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:09:18.429376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:18.429376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:09:18.429740 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:18.429740 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:09:18.431015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:18.431015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:09:18.431015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:09:18.431015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:09:18.431015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:09:18.431015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:09:18.431015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:09:18.431015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:09:18.431015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:09:18.431015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:09:18.431015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:09:18.431015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:09:18.431015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:09:18.431015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:09:18.431015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:09:18.431015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:09:18.431497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:09:18.431497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:09:18.431497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:18.431497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:09:18.431497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:18.431497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:09:18.431497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:18.431497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:09:18.431497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:18.431497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:09:18.431497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:18.431497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:09:18.431497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:09:18.431497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:18.431497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:09:18.431497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:18.431497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:09:18.431497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:09:18.431497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:18.432169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:09:18.432169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:18.432169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:09:22.876291 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:09:22.877038 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:09:22.877307 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:09:22.878622 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:09:22.910223 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:09:22.910600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:22.911441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:09:22.911441 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:22.911441 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:09:22.911441 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:09:22.911441 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:22.911441 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:09:22.911441 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:22.911441 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:09:22.912296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:22.912296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:09:22.912296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:09:22.912296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:09:22.912296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:09:22.912296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:09:22.912296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:09:22.912296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:22.912296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:09:22.912296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:22.912296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:09:22.912296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:22.912900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:09:22.912900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:22.912900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:09:22.912900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:22.912900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:09:22.912900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:09:22.912900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:22.912900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:09:22.912900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:22.912900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:09:22.912900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:09:22.912900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:22.912900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:09:22.913484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:22.913484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:09:22.913484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:22.913484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:09:22.913484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:09:22.913484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:09:22.913484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:09:22.913484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:09:22.913484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:09:22.913484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:09:22.913484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:09:22.913484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:09:22.913484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:09:22.913484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:09:22.913484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:09:22.914406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:09:22.914406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:09:22.914406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:09:22.914406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:09:22.914406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:09:22.914406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:22.914406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:09:22.914406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:22.914406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:09:22.914406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:22.914406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:09:22.914406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:22.914406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:09:22.914406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:22.914406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:09:22.914406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:09:22.914406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:22.914406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:09:22.914406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:22.915389 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:09:22.915389 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:09:22.915389 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:22.915389 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:09:22.915389 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:22.915389 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:09:23.388531 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:09:23.389412 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:09:23.389666 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:09:23.390670 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:09:23.391009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:23.391909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:09:23.391909 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:23.391909 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:09:23.391909 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:09:23.391909 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:23.391909 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:09:23.391909 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:23.391909 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:09:23.393941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:23.393941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:09:23.393941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:09:23.393941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:09:23.393941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:09:23.393941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:09:23.393941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:09:23.393941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:23.393941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:09:23.393941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:23.393941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:09:23.393941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:23.394357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:09:23.394357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:23.394357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:09:23.394357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:23.394357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:09:23.394357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:09:23.394357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:23.394357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:09:23.394357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:23.394357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:09:23.394357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:09:23.394357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:23.394357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:09:23.394806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:23.394806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:09:23.396358 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:23.396358 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:09:23.396358 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:09:23.396358 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:09:23.396358 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:09:23.396358 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:09:23.396358 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:09:23.396358 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:09:23.396358 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:09:23.396358 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:09:23.396358 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:09:23.396358 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:09:23.396358 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:09:23.396358 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:09:23.396358 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:09:23.396358 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:09:23.396882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:09:23.396882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:09:23.396882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:23.396882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:09:23.396882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:23.396882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:09:23.396882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:23.396882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:09:23.396882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:23.396882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:09:23.396882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:23.396882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:09:23.396882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:09:23.396882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:23.396882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:09:23.396882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:23.396882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:09:23.396882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:09:23.396882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:23.397446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:09:23.397446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:23.397446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:09:23.397446 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:09:23.397446 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:09:23.397446 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:09:23.397640 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:09:23.398016 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:09:23.398303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:23.399021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:09:23.399021 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:23.399021 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:09:23.399021 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:09:23.399021 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:23.399021 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:09:23.399021 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:23.399021 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:09:23.399833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:23.399833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:09:23.399833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:09:23.399833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:09:23.399833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:23.399833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:09:23.399833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:23.399833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:09:23.399833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:23.399833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:09:23.399833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:23.399833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:09:23.399833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:23.399833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:09:23.399833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:09:23.400419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:23.400419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:09:23.400419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:23.400419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:09:23.400419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:09:23.400419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:23.400419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:09:23.400419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:23.400419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:09:23.400817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:23.400817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:09:23.400817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:09:23.400817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:09:23.400817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:09:23.400817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:09:23.400817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:09:23.400817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:09:23.400817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:09:23.400817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:09:23.400817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:23.400817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:09:23.401523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:23.401523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:09:23.401523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:23.401523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:09:23.401523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:23.401523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:09:23.401523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:23.401523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:09:23.401523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:09:23.401523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:23.401523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:09:23.401523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:23.401523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:09:23.401523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:09:23.402019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:23.402019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:09:23.402019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:23.402019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:09:23.402019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:23.402019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:09:23.402019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:09:23.402019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:09:23.402019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:09:23.402019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:09:23.402019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:09:23.402019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:09:23.402019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:09:23.402019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:09:23.402401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:09:23.402401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:09:23.402401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:09:23.402401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:09:23.402401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:09:23.402401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:09:23.402401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:09:23.402401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:09:23.402401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:09:23.402401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:09:23.402401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:09:23.402401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:23.402401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:09:23.402401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:23.402401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:09:23.402401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:23.402401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:09:23.402401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:23.402401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:09:23.403035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:23.403035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:09:23.403035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:09:23.403035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:23.403035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:09:23.403035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:23.403035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:09:23.403035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:09:23.403035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:23.403035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:09:23.403035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:23.403035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:09:23.416581 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:09:23.417358 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:09:23.417540 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:09:23.417780 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:09:23.418455 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:09:23.418780 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:23.419534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:09:23.419534 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:23.419534 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:09:23.419534 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:09:23.419534 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:23.419534 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:09:23.419534 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:23.419534 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:09:23.421028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:23.421028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:09:23.421028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:09:23.421028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:09:23.421028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:23.421028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:09:23.421028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:23.421028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:09:23.421028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:23.421028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:09:23.421028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:23.421028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:09:23.421028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:23.421028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:09:23.421028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:09:23.421720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:23.421720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:09:23.421720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:23.421720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:09:23.421720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:09:23.421720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:23.421720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:09:23.421720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:23.421720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:09:23.422144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:23.422144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:09:23.422144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:09:23.422144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:09:23.422144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:09:23.422144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:09:23.422144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:09:23.422144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:09:23.422144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:09:23.422144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:23.422144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:09:23.422882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:23.422882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:09:23.422882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:23.422882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:09:23.422882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:23.422882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:09:23.422882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:23.422882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:09:23.422882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:09:23.422882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:23.422882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:09:23.422882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:23.422882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:09:23.422882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:09:23.423715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:23.423715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:09:23.423715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:23.423715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:09:23.423715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:23.423715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:09:23.423715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:09:23.423715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:09:23.423715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:09:23.423715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:09:23.423715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:09:23.423715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:09:23.423715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:09:23.423715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:09:23.424405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:09:23.424405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:09:23.424405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:09:23.424405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:09:23.424405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:09:23.424405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:09:23.424405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:09:23.424405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:09:23.424405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:09:23.424405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:09:23.424405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:23.424405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:09:23.424405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:23.424405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:09:23.424405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:23.424405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:09:23.424405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:23.424405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:09:23.424405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:23.425444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:09:23.425444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:09:23.425444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:23.425444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:09:23.425444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:23.425444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:09:23.425444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:09:23.425444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:23.425444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:09:23.425444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:23.425444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:09:23.425444 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:09:23.425444 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:09:23.426029 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:09:23.426029 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:09:23.426460 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:09:23.426869 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:09:23.427015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:23.427765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:09:23.427765 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:23.427765 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:09:23.427765 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:09:23.427765 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:23.427765 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:09:23.427765 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:23.427765 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:09:23.429139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:23.429139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:09:23.429139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:09:23.429139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:09:23.429139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:09:23.429139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:09:23.429139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:09:23.429139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:23.429139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:09:23.429139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:23.429139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:09:23.429139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:23.429521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:09:23.429521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:23.429521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:09:23.429521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:23.429521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:09:23.429521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:09:23.429521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:23.429521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:09:23.429521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:23.429521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:09:23.429521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:09:23.429521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:23.429521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:09:23.429904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:23.429904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:09:23.430536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:23.430536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:09:23.430536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:09:23.430536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:09:23.430536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:09:23.430536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:09:23.430536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:09:23.430536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:09:23.430536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:09:23.430536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:09:23.430536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:09:23.430536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:09:23.430536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:09:23.430536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:09:23.430536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:09:23.430536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:09:23.431054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:09:23.431054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:09:23.431054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:23.431054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:09:23.431054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:23.431054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:09:23.431054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:23.431054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:09:23.431054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:23.431054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:09:23.431054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:23.431054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:09:23.431054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:09:23.431054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:23.431054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:09:23.431054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:23.431054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:09:23.431054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:09:23.431054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:23.431603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:09:23.431603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:23.431603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:09:27.877087 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:09:27.877984 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:09:27.877984 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:09:27.879297 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:09:27.914213 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:09:27.914659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:27.915537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:09:27.915537 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:27.915537 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:09:27.915537 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:09:27.915537 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:27.915537 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:09:27.915537 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:27.915537 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:09:27.916376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:27.916376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:09:27.916376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:09:27.916376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:09:27.916376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:09:27.916376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:09:27.916376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:09:27.916376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:27.916376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:09:27.916376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:27.916376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:09:27.916376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:27.917024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:09:27.917024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:27.917024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:09:27.917024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:27.917024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:09:27.917024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:09:27.917024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:27.917024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:09:27.917024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:27.917024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:09:27.917024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:09:27.917024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:27.917024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:09:27.917644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:27.917644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:09:27.917644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:27.917644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:09:27.917644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:09:27.917644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:09:27.917644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:09:27.917644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:09:27.917644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:09:27.917644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:09:27.917644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:09:27.917644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:09:27.917644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:09:27.917644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:09:27.917644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:09:27.918394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:09:27.918394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:09:27.918394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:09:27.918394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:09:27.918394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:09:27.918394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:27.918394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:09:27.918394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:27.918394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:09:27.918394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:27.918394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:09:27.918394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:27.918394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:09:27.918394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:27.918394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:09:27.918394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:09:27.918394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:27.918394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:09:27.918394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:27.919324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:09:27.919324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:09:27.919324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:27.919324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:09:27.919324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:27.919324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:09:28.389376 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:09:28.390061 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:09:28.390250 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:09:28.391375 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:09:28.391609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:28.392233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:09:28.392233 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:28.392233 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:09:28.392233 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:09:28.392233 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:28.392233 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:09:28.392233 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:28.392233 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:09:28.393270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:28.393270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:09:28.393270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:09:28.393270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:09:28.393270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:09:28.393270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:09:28.393270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:09:28.393270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:28.393270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:09:28.393270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:28.393270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:09:28.393270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:28.393985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:09:28.393985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:28.393985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:09:28.393985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:28.393985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:09:28.393985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:09:28.393985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:28.393985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:09:28.393985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:28.393985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:09:28.393985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:09:28.393985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:28.393985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:09:28.394620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:28.394620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:09:28.394620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:28.394620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:09:28.394620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:09:28.394620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:09:28.394620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:09:28.394620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:09:28.394620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:09:28.394620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:09:28.394620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:09:28.394620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:09:28.394620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:09:28.394620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:09:28.394620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:09:28.395981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:09:28.395981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:09:28.395981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:09:28.395981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:09:28.395981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:09:28.395981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:28.395981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:09:28.395981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:28.395981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:09:28.395981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:28.395981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:09:28.395981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:28.395981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:09:28.395981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:28.395981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:09:28.395981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:09:28.395981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:28.395981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:09:28.395981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:28.397075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:09:28.397075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:09:28.397075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:28.397075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:09:28.397075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:28.397075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:09:28.397392 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:09:28.397541 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:09:28.397636 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:09:28.397767 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:09:28.398139 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:09:28.398342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:28.399079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:09:28.399079 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:28.399079 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:09:28.399079 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:09:28.399079 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:28.399079 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:09:28.399079 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:28.399079 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:09:28.399880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:28.399880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:09:28.399880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:09:28.399880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:09:28.399880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:28.399880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:09:28.399880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:28.399880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:09:28.399880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:28.399880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:09:28.399880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:28.399880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:09:28.399880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:28.399880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:09:28.399880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:09:28.400624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:28.400624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:09:28.400624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:28.400624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:09:28.400624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:09:28.400624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:28.400624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:09:28.400624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:28.400624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:09:28.400935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:28.400935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:09:28.400935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:09:28.400935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:09:28.400935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:09:28.400935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:09:28.400935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:09:28.400935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:09:28.400935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:09:28.400935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:09:28.400935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:28.400935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:09:28.401300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:28.401300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:09:28.401300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:28.401300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:09:28.401300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:28.401300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:09:28.401300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:28.401300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:09:28.401300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:09:28.401300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:28.401300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:09:28.401300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:28.401300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:09:28.401300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:09:28.401684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:28.401684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:09:28.401684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:28.401684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:09:28.401684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:28.401684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:09:28.401684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:09:28.401684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:09:28.401684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:09:28.401684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:09:28.401684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:09:28.401684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:09:28.401684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:09:28.401684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:09:28.402151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:09:28.402151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:09:28.402151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:09:28.402151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:09:28.402151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:09:28.402151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:09:28.402151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:09:28.402151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:09:28.402151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:09:28.402151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:09:28.402151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:09:28.402151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:28.402151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:09:28.402151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:28.402151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:09:28.402151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:28.402151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:09:28.402151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:28.402151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:09:28.402781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:28.402781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:09:28.402781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:09:28.402781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:28.402781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:09:28.402781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:28.402781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:09:28.402781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:09:28.402781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:28.402781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:09:28.402781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:28.402781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:09:28.417256 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:09:28.417658 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:09:28.417874 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:09:28.418107 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:09:28.418726 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:09:28.419295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:28.420130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:09:28.420130 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:28.420130 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:09:28.420130 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:09:28.420130 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:28.420130 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:09:28.420130 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:28.420130 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:09:28.421314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:28.421314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:09:28.421314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:09:28.421314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:09:28.421314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:28.421314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:09:28.421314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:28.421314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:09:28.421314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:28.421314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:09:28.421314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:28.421314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:09:28.421314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:28.421314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:09:28.421314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:09:28.422225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:28.422225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:09:28.422225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:28.422225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:09:28.422225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:09:28.422225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:28.422225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:09:28.422225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:28.422225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:09:28.423473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:28.423473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:09:28.423473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:09:28.423473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:09:28.423473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:09:28.423473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:09:28.423473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:09:28.423473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:09:28.423473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:09:28.423473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:28.423473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:09:28.424284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:28.424284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:09:28.424284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:28.424284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:09:28.424284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:28.424284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:09:28.424284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:28.424284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:09:28.424284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:09:28.424284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:28.424284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:09:28.424284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:28.424284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:09:28.424284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:09:28.425073 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:28.425073 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:09:28.425073 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:28.425073 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:09:28.425733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:28.425733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:09:28.425733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:09:28.425733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:09:28.425733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:09:28.425733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:09:28.425733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:09:28.425733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:09:28.425733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:09:28.425733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:09:28.425733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:09:28.425733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:09:28.425733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:09:28.425733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:09:28.425733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:09:28.425733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:09:28.425733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:09:28.426799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:09:28.426799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:09:28.426799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:09:28.426799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:28.426799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:09:28.426799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:28.426799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:09:28.426799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:28.426799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:09:28.426799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:28.426799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:09:28.426799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:28.426799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:09:28.426799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:09:28.426799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:28.426799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:09:28.426799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:28.426799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:09:28.426799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:09:28.427807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:28.427807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:09:28.427807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:28.427807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:09:28.427807 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:09:28.427807 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:09:28.427807 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:09:28.427807 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:09:28.428149 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:09:28.428149 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:09:28.428149 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:28.428149 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:09:28.428149 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:28.428149 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:09:28.428149 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:09:28.428149 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:28.428149 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:09:28.428149 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:28.428149 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:09:28.428591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:28.428591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:09:28.428591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:09:28.428591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:09:28.428591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:09:28.428591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:09:28.428591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:09:28.428591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:28.428591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:09:28.428591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:28.428591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:09:28.428591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:28.429096 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:09:28.429096 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:28.429096 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:09:28.429096 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:28.429096 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:09:28.429096 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:09:28.429096 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:28.429096 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:09:28.429096 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:28.429096 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:09:28.429096 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:09:28.429096 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:28.429096 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:09:28.429555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:28.429555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:09:28.429555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:28.429555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:09:28.429555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:09:28.429555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:09:28.429555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:09:28.429555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:09:28.429555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:09:28.429555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:09:28.429555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:09:28.429555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:09:28.429555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:09:28.429555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:09:28.429555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:09:28.430140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:09:28.430140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:09:28.430140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:09:28.430140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:09:28.430140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:09:28.430140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:28.430140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:09:28.430140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:28.430140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:09:28.430140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:28.430140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:09:28.430140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:28.430140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:09:28.430140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:28.430140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:09:28.430140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:09:28.430140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:28.430140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:09:28.430140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:28.430841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:09:28.430841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:09:28.430841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:28.430841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:09:28.430841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:28.430841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:09:32.877902 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:09:32.879220 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:09:32.879220 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:09:32.880323 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:09:32.919086 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:09:32.919489 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:32.921025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:09:32.921025 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:32.921025 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:09:32.921025 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:09:32.921025 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:32.921025 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:09:32.921025 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:32.921025 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:09:32.921815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:32.921815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:09:32.921815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:09:32.921815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:09:32.921815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:09:32.921815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:09:32.921815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:09:32.921815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:32.921815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:09:32.921815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:32.921815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:09:32.921815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:32.922354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:09:32.922354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:32.922354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:09:32.922354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:32.922354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:09:32.922354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:09:32.922354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:32.922354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:09:32.922354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:32.922354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:09:32.922354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:09:32.922354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:32.922354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:09:32.923432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:32.923432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:09:32.923432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:32.923432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:09:32.923432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:09:32.923432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:09:32.923432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:09:32.923432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:09:32.923432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:09:32.923432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:09:32.923432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:09:32.923432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:09:32.923432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:09:32.923432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:09:32.923432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:09:32.924784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:09:32.924784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:09:32.924784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:09:32.924784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:09:32.924784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:09:32.924784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:32.924784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:09:32.924784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:32.924784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:09:32.924784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:32.924784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:09:32.924784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:32.924784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:09:32.924784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:32.924784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:09:32.924784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:09:32.924784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:32.924784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:09:32.924784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:32.926687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:09:32.926687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:09:32.926687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:32.926687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:09:32.926687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:32.926687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:09:33.390162 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:09:33.390686 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:09:33.390847 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:09:33.391731 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:09:33.392008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:33.392847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:09:33.392847 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:33.392847 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:09:33.392847 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:09:33.392847 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:33.392847 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:09:33.392847 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:33.392847 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:09:33.394554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:33.394554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:09:33.394554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:09:33.394554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:09:33.394554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:09:33.394554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:09:33.394554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:09:33.394554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:33.394554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:09:33.394554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:33.394554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:09:33.394554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:33.395107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:09:33.395107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:33.395107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:09:33.395107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:33.395107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:09:33.395107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:09:33.395107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:33.395107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:09:33.395107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:33.395107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:09:33.395107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:09:33.395107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:33.395107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:09:33.395686 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:33.395686 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:09:33.396144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:33.396144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:09:33.396144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:09:33.396144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:09:33.396144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:09:33.396144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:09:33.396144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:09:33.396144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:09:33.396144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:09:33.396144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:09:33.396144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:09:33.396144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:09:33.396144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:09:33.396144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:09:33.396144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:09:33.396144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:09:33.396959 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:09:33.396959 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:09:33.396959 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:33.396959 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:09:33.396959 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:33.396959 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:09:33.396959 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:33.396959 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:09:33.396959 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:33.396959 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:09:33.396959 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:33.396959 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:09:33.396959 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:09:33.396959 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:33.396959 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:09:33.396959 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:33.396959 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:09:33.396959 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:09:33.396959 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:33.397847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:09:33.397847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:33.397847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:09:33.397847 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:09:33.398092 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:09:33.398245 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:09:33.398356 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:09:33.398682 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:09:33.398909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:33.399467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:09:33.399467 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:33.399467 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:09:33.399467 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:09:33.399467 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:33.399467 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:09:33.399467 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:33.399467 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:09:33.400257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:33.400257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:09:33.400257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:09:33.400257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:09:33.400257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:33.400257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:09:33.400257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:33.400257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:09:33.400257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:33.400257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:09:33.400257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:33.400257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:09:33.400257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:33.400257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:09:33.400257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:09:33.401003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:33.401003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:09:33.401003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:33.401003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:09:33.401003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:09:33.401003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:33.401003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:09:33.401003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:33.401003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:09:33.401418 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:33.401418 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:09:33.401418 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:09:33.401418 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:09:33.401418 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:09:33.401418 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:09:33.401418 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:09:33.401418 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:09:33.401418 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:09:33.401418 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:09:33.401418 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:33.401418 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:09:33.401990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:33.401990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:09:33.401990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:33.401990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:09:33.401990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:33.401990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:09:33.401990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:33.401990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:09:33.401990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:09:33.401990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:33.401990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:09:33.401990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:33.401990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:09:33.401990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:09:33.402630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:33.402630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:09:33.402630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:33.402630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:09:33.402630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:33.402630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:09:33.402630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:09:33.402630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:09:33.402630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:09:33.402630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:09:33.402630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:09:33.402630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:09:33.402630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:09:33.402630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:09:33.403298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:09:33.403298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:09:33.403298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:09:33.403298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:09:33.403298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:09:33.403298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:09:33.403298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:09:33.403298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:09:33.403298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:09:33.403298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:09:33.403298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:09:33.403298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:33.403298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:09:33.403298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:33.403298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:09:33.403298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:33.403298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:09:33.403298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:33.403298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:09:33.404186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:33.404186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:09:33.404186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:09:33.404186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:33.404186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:09:33.404186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:33.404186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:09:33.404186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:09:33.404186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:33.404186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:09:33.404186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:33.404186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:09:33.417861 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:09:33.418291 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:09:33.418497 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:09:33.418714 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:09:33.419264 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:09:33.419685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:33.420541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:09:33.420541 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:33.420541 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:09:33.420541 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:09:33.420541 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:33.420541 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:09:33.420541 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:33.420541 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:09:33.422120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:33.422120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:09:33.422120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:09:33.422120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:09:33.422120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:33.422120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:09:33.422120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:33.422120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:09:33.422120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:33.422120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:09:33.422120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:33.422120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:09:33.422120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:33.422120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:09:33.422120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:09:33.422801 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:33.422801 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:09:33.422801 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:33.422801 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:09:33.422801 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:09:33.422801 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:33.422801 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:09:33.422801 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:33.422801 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:09:33.424091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:33.424091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:09:33.424091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:09:33.424091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:09:33.424091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:09:33.424091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:09:33.424091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:09:33.424091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:09:33.424091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:09:33.424091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:33.424091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:09:33.424599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:33.424599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:09:33.424599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:33.424599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:09:33.424599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:33.424599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:09:33.424599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:33.424599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:09:33.424599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:09:33.424599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:33.424599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:09:33.424599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:33.424599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:09:33.424599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:09:33.425374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:33.425374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:09:33.425374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:33.425374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:09:33.426333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:33.426333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:09:33.426333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:09:33.426333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:09:33.426333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:09:33.426333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:09:33.426333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:09:33.426333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:09:33.426333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:09:33.426333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:09:33.426333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:09:33.426333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:09:33.426333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:09:33.426333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:09:33.426333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:09:33.426333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:09:33.426333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:09:33.427120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:09:33.427120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:09:33.427120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:09:33.427120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:33.427120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:09:33.427120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:33.427120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:09:33.427120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:33.427120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:09:33.427120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:33.427120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:09:33.427120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:33.427120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:09:33.427120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:09:33.427120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:33.427120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:09:33.427120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:33.427120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:09:33.427120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:09:33.427918 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:33.427918 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:09:33.427918 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:33.427918 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:09:33.427918 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:09:33.427918 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:09:33.427918 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:09:33.427918 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:09:33.428188 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:09:33.428309 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:09:33.428478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:33.429253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:09:33.429253 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:33.429253 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:09:33.429253 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:09:33.429253 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:33.429253 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:09:33.429253 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:33.429253 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:09:33.430327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:33.430327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:09:33.430327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:09:33.430327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:09:33.430327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:09:33.430327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:09:33.430327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:09:33.430327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:33.430327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:09:33.430327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:33.430327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:09:33.430327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:33.430703 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:09:33.430703 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:33.430703 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:09:33.430703 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:33.430703 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:09:33.430703 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:09:33.430703 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:33.430703 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:09:33.430703 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:33.430703 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:09:33.430703 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:09:33.430703 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:33.430703 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:09:33.431099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:33.431099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:09:33.431346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:33.431346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:09:33.431346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:09:33.431346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:09:33.431346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:09:33.431346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:09:33.431346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:09:33.431346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:09:33.431346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:09:33.431346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:09:33.431346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:09:33.431346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:09:33.431346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:09:33.431346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:09:33.431346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:09:33.431346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:09:33.431853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:09:33.431853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:09:33.431853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:33.431853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:09:33.431853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:33.431853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:09:33.431853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:33.431853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:09:33.431853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:33.431853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:09:33.431853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:33.431853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:09:33.431853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:09:33.431853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:33.431853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:09:33.431853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:33.431853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:09:33.431853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:09:33.431853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:33.432389 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:09:33.432389 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:33.432389 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:09:37.878469 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:09:37.880109 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:09:37.880284 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:09:37.881522 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:09:37.897073 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG dbcounter [-] [106314] Writing DB stats neutron:SELECT=238 {{(pid=106314) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 09 09:09:37.919776 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:09:37.920425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:37.921544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:09:37.921544 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:37.921544 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:09:37.921544 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:09:37.921544 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:37.921544 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:09:37.921544 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:37.921544 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:09:37.922768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:37.922768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:09:37.922768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:09:37.922768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:09:37.922768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:09:37.922768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:09:37.922768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:09:37.922768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:37.922768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:09:37.922768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:37.922768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:09:37.922768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:37.923514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:09:37.923514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:37.923514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:09:37.923514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:37.923514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:09:37.923514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:09:37.923514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:37.923514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:09:37.923514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:37.923514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:09:37.923514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:09:37.923514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:37.923514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:09:37.924261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:37.924261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:09:37.924261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:37.924261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:09:37.924261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:09:37.924261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:09:37.924261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:09:37.924261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:09:37.924261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:09:37.924261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:09:37.924261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:09:37.924261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:09:37.924261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:09:37.924261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:09:37.924261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:09:37.925098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:09:37.925098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:09:37.925098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:09:37.925098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:09:37.925098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:09:37.925098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:37.925098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:09:37.925098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:37.925098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:09:37.925098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:37.925098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:09:37.925098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:37.925098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:09:37.925098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:37.925098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:09:37.925098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:09:37.925098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:37.925098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:09:37.925098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:37.926124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:09:37.926124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:09:37.926124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:37.926124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:09:37.926124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:37.926124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:09:38.390771 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:09:38.391054 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:09:38.391245 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:09:38.392158 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:09:38.392473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:38.393368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:09:38.393368 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:38.393368 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:09:38.393368 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:09:38.393368 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:38.393368 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:09:38.393368 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:38.393368 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:09:38.394883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:38.394883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:09:38.394883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:09:38.394883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:09:38.394883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:09:38.394883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:09:38.394883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:09:38.394883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:38.394883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:09:38.394883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:38.394883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:09:38.394883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:38.395708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:09:38.395708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:38.395708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:09:38.395708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:38.395708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:09:38.395708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:09:38.395708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:38.395708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:09:38.395708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:38.395708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:09:38.395708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:09:38.395708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:38.395708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:09:38.396566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:38.396566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:09:38.397256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:38.397256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:09:38.397256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:09:38.397256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:09:38.397256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:09:38.397256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:09:38.397256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:09:38.397256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:09:38.397256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:09:38.397256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:09:38.397256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:09:38.397256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:09:38.397256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:09:38.397256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:09:38.397256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:09:38.397256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:09:38.398137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:09:38.398137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:09:38.398137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:38.398137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:09:38.398137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:38.398137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:09:38.398137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:38.398137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:09:38.398137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:38.398137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:09:38.398137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:38.398137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:09:38.398137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:09:38.398137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:38.398137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:09:38.398137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:38.398137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:09:38.398137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:09:38.398137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:38.399501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:09:38.399501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:38.399501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:09:38.399501 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:09:38.399501 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:09:38.399501 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:09:38.399501 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:09:38.400001 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:09:38.400079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:38.401076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:09:38.401076 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:38.401076 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:09:38.401076 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:09:38.401076 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:38.401076 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:09:38.401076 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:38.401076 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:09:38.401838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:38.401838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:09:38.401838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:09:38.401838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:09:38.401838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:38.401838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:09:38.401838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:38.401838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:09:38.401838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:38.401838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:09:38.401838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:38.401838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:09:38.401838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:38.401838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:09:38.401838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:09:38.402248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:38.402248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:09:38.402248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:38.402248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:09:38.402248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:09:38.402248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:38.402248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:09:38.402248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:38.402248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:09:38.402796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:38.402796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:09:38.402796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:09:38.402796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:09:38.402796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:09:38.402796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:09:38.402796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:09:38.402796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:09:38.402796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:09:38.402796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:09:38.402796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:38.402796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:09:38.403208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:38.403208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:09:38.403208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:38.403208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:09:38.403208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:38.403208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:09:38.403208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:38.403208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:09:38.403208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:09:38.403208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:38.403208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:09:38.403208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:38.403208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:09:38.403208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:09:38.403595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:38.403595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:09:38.403595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:38.403595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:09:38.403948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:38.403948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:09:38.403948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:09:38.403948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:09:38.403948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:09:38.403948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:09:38.403948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:09:38.403948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:09:38.403948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:09:38.403948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:09:38.403948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:09:38.403948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:09:38.403948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:09:38.403948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:09:38.403948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:09:38.403948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:09:38.404401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:09:38.404401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:09:38.404401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:09:38.404401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:09:38.404401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:09:38.404401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:38.404401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:09:38.404401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:38.404401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:09:38.404401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:38.404401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:09:38.404401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:38.404401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:09:38.404401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:38.404401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:09:38.404401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:09:38.404401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:38.404401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:09:38.404976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:38.404976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:09:38.404976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:09:38.404976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:38.404976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:09:38.404976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:38.404976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:09:38.418286 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:09:38.418598 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:09:38.418758 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:09:38.418974 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:09:38.419590 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:09:38.420053 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:38.420927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:09:38.420927 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:38.420927 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:09:38.420927 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:09:38.420927 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:38.420927 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:09:38.420927 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:38.420927 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:09:38.422357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:38.422357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:09:38.422357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:09:38.422357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:09:38.422357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:38.422357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:09:38.422357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:38.422357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:09:38.422357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:38.422357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:09:38.422357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:38.422357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:09:38.422357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:38.422357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:09:38.422357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:09:38.423394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:38.423394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:09:38.423394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:38.423394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:09:38.423394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:09:38.423394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:38.423394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:09:38.423394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:38.423394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:09:38.424373 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:38.424373 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:09:38.424373 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:09:38.424373 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:09:38.424373 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:09:38.424373 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:09:38.424373 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:09:38.424373 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:09:38.424373 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:09:38.424373 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:38.424373 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:09:38.425151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:38.425151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:09:38.425151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:38.425151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:09:38.425151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:38.425151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:09:38.425151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:38.425151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:09:38.425151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:09:38.425151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:38.425151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:09:38.425151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:38.425151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:09:38.425151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:09:38.425940 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:38.425940 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:09:38.425940 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:38.425940 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:09:38.426738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:38.426738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:09:38.426738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:09:38.426738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:09:38.426738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:09:38.426738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:09:38.426738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:09:38.426738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:09:38.426738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:09:38.426738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:09:38.426738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:09:38.426738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:09:38.426738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:09:38.426738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:09:38.426738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:09:38.426738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:09:38.426738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:09:38.427957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:09:38.427957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:09:38.427957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:09:38.427957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:38.427957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:09:38.427957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:38.427957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:09:38.427957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:38.427957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:09:38.427957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:38.427957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:09:38.427957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:38.427957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:09:38.427957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:09:38.427957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:38.427957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:09:38.427957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:38.427957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:09:38.427957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:09:38.428733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:38.428733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:09:38.428733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:38.428733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:09:38.428733 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:09:38.428733 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:09:38.428733 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:09:38.428733 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:09:38.429554 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:09:38.429554 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:09:38.429554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:38.430087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:09:38.430087 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:38.430087 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:09:38.430087 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:09:38.430087 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:38.430087 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:09:38.430087 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:38.430087 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:09:38.430919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:38.430919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:09:38.430919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:09:38.430919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:09:38.430919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:09:38.430919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:09:38.430919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:09:38.430919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:38.430919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:09:38.430919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:38.430919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:09:38.430919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:38.431494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:09:38.431494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:38.431494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:09:38.431494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:38.431494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:09:38.431494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:09:38.431494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:38.431494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:09:38.431494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:38.431494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:09:38.431494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:09:38.431494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:38.431494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:09:38.432088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:38.432088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:09:38.432360 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:38.432360 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:09:38.432360 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:09:38.432360 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:09:38.432360 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:09:38.432360 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:09:38.432360 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:09:38.432360 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:09:38.432360 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:09:38.432360 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:09:38.432360 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:09:38.432360 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:09:38.432360 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:09:38.432360 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:09:38.432360 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:09:38.432360 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:09:38.432888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:09:38.432888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:09:38.432888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:38.432888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:09:38.432888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:38.432888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:09:38.432888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:38.432888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:09:38.432888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:38.432888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:09:38.432888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:38.432888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:09:38.432888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:09:38.432888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:38.432888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:09:38.432888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:38.432888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:09:38.432888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:09:38.432888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:38.433425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:09:38.433425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:38.433425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:09:42.880400 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:09:42.881267 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:09:42.881267 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:09:42.882193 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:09:42.926605 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:09:42.927182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:42.928512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:09:42.928512 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:42.928512 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:09:42.928512 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:09:42.928512 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:42.928512 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:09:42.928512 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:42.928512 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:09:42.930344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.048 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:42.930344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:09:42.930344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:09:42.930344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:09:42.930344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:09:42.930344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:09:42.930344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:09:42.930344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:42.930344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:09:42.930344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:42.930344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:09:42.930344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:42.930731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:09:42.930731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:42.930731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:09:42.930731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:42.930731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:09:42.930731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:09:42.930731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:42.930731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:09:42.930731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:42.930731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:09:42.930731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:09:42.930731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:42.930731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:09:42.931141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:42.931141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:09:42.931989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:42.931989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:09:42.931989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:09:42.931989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:09:42.931989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:09:42.931989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:09:42.931989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:09:42.931989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:09:42.931989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:09:42.931989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:09:42.931989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:09:42.931989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:09:42.931989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:09:42.931989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:09:42.931989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:09:42.931989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:09:42.932460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:09:42.932460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:09:42.932460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:42.932460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:09:42.932460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:42.932460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:09:42.932460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:42.932460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:09:42.932460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:42.932460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:09:42.932460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:42.932460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:09:42.932460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:09:42.932460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:42.932460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:09:42.932460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:42.932460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:09:42.932460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:09:42.932460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:42.932999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:09:42.932999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:42.932999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:09:43.391442 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:09:43.393040 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:09:43.393400 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:09:43.394540 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:09:43.394937 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:43.395764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:09:43.395764 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:43.395764 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:09:43.395764 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:09:43.395764 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:43.395764 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:09:43.395764 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:43.395764 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:09:43.397734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:43.397734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:09:43.397734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:09:43.397734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:09:43.397734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:09:43.397734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:09:43.397734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:09:43.397734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:43.397734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:09:43.397734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:43.397734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:09:43.397734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:43.398150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:09:43.398150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:43.398150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:09:43.398150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:43.398150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:09:43.398150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:09:43.398150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:43.398150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:09:43.398150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:43.398150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:09:43.398150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:09:43.398150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:43.398150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:09:43.398478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:43.398478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:09:43.399191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:43.399191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:09:43.399191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:09:43.399191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:09:43.399191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:09:43.399191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:09:43.399191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:09:43.399191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:09:43.399191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:09:43.399191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:09:43.399191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:09:43.399191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:09:43.399191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:09:43.399191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:09:43.399191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:09:43.399191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:09:43.399634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:09:43.399634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:09:43.399634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:43.399634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:09:43.399634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:43.399634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:09:43.399634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:43.399634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:09:43.399634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:43.399634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:09:43.399634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:43.399634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:09:43.399634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:09:43.399634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:43.399634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:09:43.399634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:43.399634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:09:43.399634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:09:43.399634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:43.400118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:09:43.400118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:43.400118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:09:43.400118 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:09:43.400118 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:09:43.400118 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:09:43.400118 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:09:43.400758 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:09:43.400758 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:43.401190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:09:43.401190 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:43.401190 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:09:43.401190 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:09:43.401190 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:43.401190 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:09:43.401190 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:43.401190 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:09:43.401786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:43.401786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:09:43.401786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:09:43.401786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:09:43.401786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:43.401786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:09:43.401786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:43.401786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:09:43.401786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:43.401786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:09:43.401786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:43.401786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:09:43.401786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:43.401786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:09:43.401786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:09:43.402695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:43.402695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:09:43.402695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:43.402695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:09:43.402695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:09:43.402695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:43.402695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:09:43.402695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:43.402695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:09:43.402695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:43.402695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:09:43.402695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:09:43.402695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:09:43.402695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:09:43.403462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:09:43.403462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:09:43.403462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:09:43.403462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:09:43.403462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:09:43.403462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:43.403462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:09:43.403462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:43.403462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:09:43.403462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:43.403462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:09:43.403462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:43.403462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:09:43.403836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:43.403836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:09:43.403836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:09:43.403836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:43.403836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:09:43.403836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:43.403836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:09:43.403836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:09:43.403836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:43.403836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:09:43.403836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:43.403836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:09:43.404132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:43.404132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:09:43.404132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:09:43.404132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:09:43.404132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:09:43.404132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:09:43.404132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:09:43.404132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:09:43.404132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:09:43.404132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:09:43.404132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:09:43.404132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:09:43.404132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:09:43.404132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:09:43.404132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:09:43.404132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:09:43.404507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:09:43.404507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:09:43.404507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:09:43.404507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:09:43.404507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:09:43.404507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:43.404507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:09:43.404507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:43.404507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:09:43.404507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:43.404507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:09:43.404507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:43.404507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:09:43.404507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:43.404507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:09:43.404507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:09:43.404507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:43.404507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:09:43.404995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:43.404995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:09:43.404995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:09:43.404995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:43.404995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:09:43.404995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:43.404995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:09:43.418867 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:09:43.419317 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:09:43.419565 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:09:43.419899 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:09:43.420521 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:09:43.421224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:43.422200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:09:43.422200 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:43.422200 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:09:43.422200 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:09:43.422200 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:43.422200 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:09:43.422200 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:43.422200 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:09:43.423332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:43.423332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:09:43.423332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:09:43.423332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:09:43.423332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:43.423332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:09:43.423332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:43.423332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:09:43.423332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:43.423332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:09:43.423332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:43.423332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:09:43.423332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:43.423332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:09:43.423332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:09:43.423947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:43.423947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:09:43.423947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:43.423947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:09:43.423947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:09:43.423947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:43.423947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:09:43.423947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:43.423947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:09:43.424331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:43.424331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:09:43.424331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:09:43.424331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:09:43.424331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:09:43.424331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:09:43.424331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:09:43.424331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:09:43.424331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:09:43.424331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:43.424331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:09:43.424867 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:43.424867 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:09:43.424867 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:43.424867 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:09:43.424867 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:43.424867 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:09:43.424867 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:43.424867 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:09:43.424867 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:09:43.424867 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:43.424867 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:09:43.424867 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:43.424867 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:09:43.424867 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:09:43.425345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:43.425345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:09:43.425345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:43.425345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:09:43.425460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:43.425460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:09:43.425460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:09:43.425460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:09:43.425460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:09:43.425460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:09:43.425460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:09:43.425460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:09:43.425460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:09:43.425460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:09:43.425460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:09:43.425460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:09:43.425460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:09:43.425460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:09:43.425460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:09:43.425460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:09:43.425460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:09:43.425896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:09:43.425896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:09:43.425896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:09:43.425896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:43.425896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:09:43.425896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:43.425896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:09:43.425896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:43.425896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:09:43.425896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:43.425896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:09:43.425896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:43.425896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:09:43.425896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:09:43.425896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:43.425896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:09:43.425896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:43.425896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:09:43.425896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:09:43.426364 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:43.426364 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:09:43.426364 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:43.426364 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:09:43.427765 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:09:43.428037 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:09:43.428156 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:09:43.428321 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:09:43.428914 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:09:43.429309 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:09:43.429492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:43.430301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:09:43.430301 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:43.430301 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:09:43.430301 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:09:43.430301 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:43.430301 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:09:43.430301 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:43.430301 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:09:43.431177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:43.431177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:09:43.431177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:09:43.431177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:09:43.431177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:09:43.431177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:09:43.431177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:09:43.431177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:43.431177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:09:43.431177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:43.431177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:09:43.431177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:43.431513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:09:43.431513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:43.431513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:09:43.431513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:43.431513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:09:43.431513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:09:43.431513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:43.431513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:09:43.431513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:43.431513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:09:43.431513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:09:43.431513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:43.431513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:09:43.431847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:43.431847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:09:43.432157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:43.432157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:09:43.432157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:09:43.432157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:09:43.432157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:09:43.432157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:09:43.432157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:09:43.432157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:09:43.432157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:09:43.432157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:09:43.432157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:09:43.432157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:09:43.432157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:09:43.432157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:09:43.432157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:09:43.432157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:09:43.432550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:09:43.432550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:09:43.432550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:43.432550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:09:43.432550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:43.432550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:09:43.432550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:43.432550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:09:43.432550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:43.432550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:09:43.432550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:43.432550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:09:43.432550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:09:43.432550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:43.432550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:09:43.432550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:43.432550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:09:43.432550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:09:43.432550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:43.433646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:09:43.433646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:43.433646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:09:47.881142 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:09:47.882077 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:09:47.882077 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:09:47.883436 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:09:47.912897 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:09:47.913314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:47.914152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:09:47.914152 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:47.914152 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:09:47.914152 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:09:47.914152 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:47.914152 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:09:47.914152 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:47.914152 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:09:47.915274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.032 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:47.915274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:09:47.915274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:09:47.915274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:09:47.915274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:09:47.915274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:09:47.915274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:09:47.915274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:47.915274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:09:47.915274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:47.915274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:09:47.915274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:47.915650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:09:47.915650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:47.915650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:09:47.915650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:47.915650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:09:47.915650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:09:47.915650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:47.915650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:09:47.915650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:47.915650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:09:47.915650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:09:47.915650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:47.915650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:09:47.916058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:47.916058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:09:47.916726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:47.916726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:09:47.916726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:09:47.916726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:09:47.916726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:09:47.916726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:09:47.916726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:09:47.916726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:09:47.916726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:09:47.916726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:09:47.916726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:09:47.916726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:09:47.916726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:09:47.916726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:09:47.916726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:09:47.916726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:09:47.917699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:09:47.917699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:09:47.917699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:47.917699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:09:47.917699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:47.917699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:09:47.917699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:47.917699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:09:47.917699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:47.917699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:09:47.917699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:47.917699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:09:47.917699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:09:47.917699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:47.917699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:09:47.917699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:47.917699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:09:47.917699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:09:47.917699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:47.918609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:09:47.918609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:47.918609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:09:47.942466 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_for_inconsistencies' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:09:47.942897 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_for_inconsistencies {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:09:47.943059 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Checking Neutron and OVN revision consistency. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:09:47.953611 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Maintenance task: Synchronizing Neutron and OVN databases started {{(pid=106314) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:440}}
May 09 09:09:47.954186 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Maintenance task: Number of inconsistencies found at create/update: networks=2, security_groups=1 {{(pid=106314) _log /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:419}}
May 09 09:09:47.956385 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Maintenance task: Fixing resource f8a38fcb-e8a2-421a-a340-bf7f2f5cc72b (type: networks) at create/update {{(pid=106314) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 09 09:09:47.983977 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-f8a38fcb-e8a2-421a-a340-bf7f2f5cc72b, resource={'id': 'f8a38fcb-e8a2-421a-a340-bf7f2f5cc72b', 'name': 'private', 'tenant_id': 'f4d54dfe001d43fbb344093a1ccabd6a', 'project_id': 'f4d54dfe001d43fbb344093a1ccabd6a', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['96e72a03-97b8-412a-b2b9-1bf048ce1066', 'bbd6957a-953f-4fc1-8761-5d93128564d9'], 'standard_attr_id': 12, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-05-09T09:04:34Z', 'updated_at': '2026-05-09T09:04:37Z', 'revision_number': 3, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 3730}, resource_type=networks, if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:09:47.984585 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-f8a38fcb-e8a2-421a-a340-bf7f2f5cc72b, col_values=(('external_ids', {'neutron:network_name': 'private', 'neutron:mtu': '1372', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:09:47.985402 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-78a32c7c-7637-4b83-b0ab-62aee8bb3038, external_ids_update=None, columns={'tag': 3730}, if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:09:47.985713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:47.987851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:09:47.987851 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:47.987851 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:09:47.987851 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:09:47.987851 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:47.987851 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:09:47.987851 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:47.987851 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:09:47.987851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Maintenance task: Failed to fix resource f8a38fcb-e8a2-421a-a340-bf7f2f5cc72b (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:47.987851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:09:47.987851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 09 09:09:47.987851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 09 09:09:47.987851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 09 09:09:47.987851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 09 09:09:47.988577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2332, in update_network
May 09 09:09:47.988577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:09:47.988577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:47.988577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:09:47.988577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:47.988577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:09:47.988577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:47.988577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:09:47.988577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:47.988577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:09:47.988577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:47.988577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:09:47.988577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:09:47.989361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:47.989361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:09:47.989361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:47.989361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:09:47.989361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:09:47.989361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:47.989361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:09:47.989361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:47.989361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:09:47.989361 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Maintenance task: Fixing resource d18a9fac-2ebf-4d1e-a86c-0f33fba8eeda (type: networks) at create/update {{(pid=106314) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 09 09:09:48.015929 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-d18a9fac-2ebf-4d1e-a86c-0f33fba8eeda, resource={'id': 'd18a9fac-2ebf-4d1e-a86c-0f33fba8eeda', 'name': 'public', 'tenant_id': 'f4d54dfe001d43fbb344093a1ccabd6a', 'project_id': 'f4d54dfe001d43fbb344093a1ccabd6a', 'admin_state_up': True, 'mtu': 1430, 'status': 'ACTIVE', 'subnets': ['088bbd0f-7e81-4a76-8399-24d15c984009', 'bbd6a8cc-80e6-4336-a70b-1c962c561590'], 'standard_attr_id': 18, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': True, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'is_default': True, 'tags': [], 'created_at': '2026-05-09T09:04:40Z', 'updated_at': '2026-05-09T09:04:50Z', 'revision_number': 3, 'provider:network_type': 'flat', 'provider:physical_network': 'public', 'provider:segmentation_id': None}, resource_type=networks, if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:09:48.016461 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-d18a9fac-2ebf-4d1e-a86c-0f33fba8eeda, col_values=(('external_ids', {'neutron:network_name': 'public', 'neutron:mtu': '1430', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'flat', 'neutron:provnet-physical-network': 'public'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false', 'fdb_age_threshold': '300', 'broadcast-arps-to-all-routers': 'true'})), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:09:48.017139 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-1163f525-70cd-4b84-ae9b-7d56378305e9, external_ids_update=None, columns={'tag': []}, if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:09:48.017621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:48.018679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:09:48.018679 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:48.018679 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:09:48.018679 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:09:48.018679 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:48.018679 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:09:48.018679 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:48.018679 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:09:48.019594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Maintenance task: Failed to fix resource d18a9fac-2ebf-4d1e-a86c-0f33fba8eeda (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:48.019594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:09:48.019594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 09 09:09:48.019594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 09 09:09:48.019594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 09 09:09:48.019594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 09 09:09:48.019594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2332, in update_network
May 09 09:09:48.019594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:09:48.019594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:48.019594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:09:48.019594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:48.020226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:09:48.020226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:48.020226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:09:48.020226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:48.020226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:09:48.020226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:48.020226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:09:48.020226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:09:48.020226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:48.020226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:09:48.020226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:48.020226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:09:48.020226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:09:48.020226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:48.020960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:09:48.020960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:48.020960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:09:48.020960 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Maintenance task: Fixing resource 559aa7f0-02df-4065-8e59-cd14b6329cbd (type: security_groups) at create/update {{(pid=106314) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 09 09:09:48.075171 np0000173033 neutron-ovn-maintenance-worker[106314]: INFO neutron.db.ovn_revision_numbers_db [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Successfully bumped revision number for resource 559aa7f0-02df-4065-8e59-cd14b6329cbd (type: security_groups) to 9
May 09 09:09:48.075377 np0000173033 neutron-ovn-maintenance-worker[106314]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_for_inconsistencies finished in 0.132 seconds
May 09 09:09:48.392623 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:09:48.393995 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:09:48.393995 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:09:48.394655 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:09:48.395007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:48.396080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:09:48.396080 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:48.396080 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:09:48.396080 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:09:48.396080 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:48.396080 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:09:48.396080 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:48.396080 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:09:48.399569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:48.399569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:09:48.399569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:09:48.399569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:09:48.399569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:09:48.399569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:09:48.399569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:09:48.399569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:48.399569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:09:48.399569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:48.399569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:09:48.399569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:48.400438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:09:48.400438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:48.400438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:09:48.400438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:48.400438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:09:48.400438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:09:48.400438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:48.400438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:09:48.400438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:48.400438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:09:48.400438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:09:48.400438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:48.400438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:09:48.400808 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:48.400808 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:09:48.402112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:48.402112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:09:48.402112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:09:48.402112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:09:48.402112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:09:48.402112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:09:48.402112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:09:48.402112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:09:48.402112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:09:48.402112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:09:48.402112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:09:48.402112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:09:48.402112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:09:48.402112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:09:48.402112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:09:48.402112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:09:48.402621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:09:48.402621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:09:48.402621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:48.402621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:09:48.402621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:48.402621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:09:48.402621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:48.402621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:09:48.402621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:48.402621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:09:48.402621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:48.402621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:09:48.402621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:09:48.402621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:48.402621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:09:48.402621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:48.402621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:09:48.402621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:09:48.402621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:48.406125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:09:48.406125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:48.406125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:09:48.406125 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:09:48.406125 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:09:48.406125 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:09:48.406125 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:09:48.406125 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:09:48.406488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:48.406488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:09:48.406488 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:48.406488 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:09:48.406488 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:09:48.406488 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:48.406488 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:09:48.406488 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:48.406488 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:09:48.406488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:48.406488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:09:48.406488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:09:48.406488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:09:48.406488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:48.406488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:09:48.406488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:48.406488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:09:48.407000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:48.407000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:09:48.407000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:48.407000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:09:48.407000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:48.407000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:09:48.407000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:09:48.407000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:48.407000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:09:48.407000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:48.407000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:09:48.407000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:09:48.407000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:48.407000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:09:48.407000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:48.407000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:09:48.407483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:48.407483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:09:48.407483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:09:48.407483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:09:48.407483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:09:48.407483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:09:48.407483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:09:48.407483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:09:48.407483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:09:48.407483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:09:48.407483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:48.407483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:09:48.407873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:48.407873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:09:48.407873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:48.407873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:09:48.407873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:48.407873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:09:48.407873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:48.407873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:09:48.407873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:09:48.407873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:48.407873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:09:48.407873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:48.407873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:09:48.407873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:09:48.408250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:48.408250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:09:48.408250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:48.408250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:09:48.408250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:48.408250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:09:48.408250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:09:48.408250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:09:48.408250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:09:48.408250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:09:48.408250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:09:48.408250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:09:48.408250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:09:48.408250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:09:48.408645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:09:48.408645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:09:48.408645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:09:48.408645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:09:48.408645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:09:48.408645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:09:48.408645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:09:48.408645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:09:48.408645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:09:48.408645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:09:48.408645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:09:48.408645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:48.408645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:09:48.408645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:48.408645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:09:48.408645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:48.408645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:09:48.408645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:48.408645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:09:48.409451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:48.409451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:09:48.409451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:09:48.409451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:48.409451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:09:48.409451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:48.409451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:09:48.409451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:09:48.409451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:48.409451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:09:48.409451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:48.409451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:09:48.419828 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:09:48.419828 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:09:48.419828 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:09:48.419828 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:09:48.420179 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:09:48.420382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:48.422139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:09:48.422139 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:48.422139 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:09:48.422139 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:09:48.422139 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:48.422139 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:09:48.422139 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:48.422139 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:09:48.422139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:48.422139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:09:48.422139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:09:48.422139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:09:48.422139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:48.422139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:09:48.422139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:48.422139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:09:48.422139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:48.422139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:09:48.423206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:48.423206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:09:48.423206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:48.423206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:09:48.423206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:09:48.423206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:48.423206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:09:48.423206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:48.423206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:09:48.423206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:09:48.423206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:48.423206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:09:48.423206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:48.423206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:09:48.423852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:48.423852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:09:48.423852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:09:48.423852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:09:48.423852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:09:48.423852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:09:48.423852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:09:48.423852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:09:48.423852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:09:48.423852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:48.423852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:09:48.424327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:48.424327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:09:48.424327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:48.424327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:09:48.424327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:48.424327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:09:48.424327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:48.424327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:09:48.424327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:09:48.424327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:48.424327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:09:48.424327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:48.424327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:09:48.424327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:09:48.424912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:48.424912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:09:48.424912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:48.424912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:09:48.428054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:48.428054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:09:48.428054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:09:48.428054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:09:48.428054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:09:48.428054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:09:48.428054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:09:48.428054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:09:48.428054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:09:48.428054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:09:48.428054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:09:48.428054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:09:48.428054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:09:48.428054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:09:48.428054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:09:48.428054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:09:48.428054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:09:48.429164 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:09:48.429164 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:09:48.429164 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:09:48.429164 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:48.429164 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:09:48.429164 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:48.429164 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:09:48.429164 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:48.429164 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:09:48.429164 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:48.429164 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:09:48.429164 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:48.429164 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:09:48.429164 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:09:48.429164 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:48.429164 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:09:48.429164 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:48.429164 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:09:48.429164 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:09:48.430251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:48.430251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:09:48.430251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:48.430251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:09:48.430251 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:09:48.430251 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:09:48.430251 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:09:48.430251 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:09:48.430773 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:09:48.430773 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:09:48.430773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:48.430773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:09:48.430773 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:48.430773 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:09:48.430773 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:09:48.430773 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:48.430773 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:09:48.430773 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:48.430773 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:09:48.431472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:48.431472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:09:48.431472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:09:48.431472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:09:48.431472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:09:48.431472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:09:48.431472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:09:48.431472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:48.431472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:09:48.431472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:48.431472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:09:48.431472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:48.432214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:09:48.432214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:48.432214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:09:48.432214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:48.432214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:09:48.432214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:09:48.432214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:48.432214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:09:48.432214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:48.432214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:09:48.432214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:09:48.432214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:48.432214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:09:48.432949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:48.432949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:09:48.432949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:48.432949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:09:48.432949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:09:48.432949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:09:48.432949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:09:48.432949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:09:48.432949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:09:48.432949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:09:48.432949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:09:48.432949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:09:48.432949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:09:48.432949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:09:48.432949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:09:48.433823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:09:48.433823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:09:48.433823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:09:48.433823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:09:48.433823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:09:48.433823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:48.433823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:09:48.433823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:48.433823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:09:48.433823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:48.433823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:09:48.433823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:48.433823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:09:48.433823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:48.433823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:09:48.433823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:09:48.433823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:48.433823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:09:48.433823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:48.434315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:09:48.434315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:09:48.434315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:48.434315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:09:48.434315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:48.434315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:09:52.881806 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:09:52.887322 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:09:52.887322 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:09:52.887322 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:09:52.918787 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:09:52.919189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:52.920109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:09:52.920109 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:52.920109 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:09:52.920109 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:09:52.920109 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:52.920109 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:09:52.920109 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:52.920109 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:09:52.920885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:52.920885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:09:52.920885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:09:52.920885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:09:52.920885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:09:52.920885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:09:52.920885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:09:52.920885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:52.920885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:09:52.920885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:52.920885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:09:52.920885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:52.921267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:09:52.921267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:52.921267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:09:52.921267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:52.921267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:09:52.921267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:09:52.921267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:52.921267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:09:52.921267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:52.921267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:09:52.921267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:09:52.921267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:52.921267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:09:52.921626 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:52.921626 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:09:52.921626 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:52.921626 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:09:52.921626 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:09:52.921626 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:09:52.921626 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:09:52.921626 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:09:52.921626 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:09:52.921626 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:09:52.921626 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:09:52.921626 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:09:52.921626 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:09:52.921626 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:09:52.921626 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:09:52.922109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:09:52.922109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:09:52.922109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:09:52.922109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:09:52.922109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:09:52.922109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:52.922109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:09:52.922109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:52.922109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:09:52.922109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:52.922109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:09:52.922109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:52.922109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:09:52.922109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:52.922109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:09:52.922109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:09:52.922109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:52.922109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:09:52.922109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:52.922601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:09:52.922601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:09:52.922601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:52.922601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:09:52.922601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:52.922601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:09:53.393624 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:09:53.394071 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:09:53.394272 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:09:53.395253 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:09:53.395788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:53.396331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:09:53.396331 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:53.396331 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:09:53.396331 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:09:53.396331 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:53.396331 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:09:53.396331 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:53.396331 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:09:53.398077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:53.398077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:09:53.398077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:09:53.398077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:09:53.398077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:09:53.398077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:09:53.398077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:09:53.398077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:53.398077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:09:53.398077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:53.398077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:09:53.398077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:53.398707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:09:53.398707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:53.398707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:09:53.398707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:53.398707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:09:53.398707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:09:53.398707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:53.398707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:09:53.398707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:53.398707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:09:53.398707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:09:53.398707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:53.398707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:09:53.399119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:53.399119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:09:53.400367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:53.400367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:09:53.400367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:09:53.400367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:09:53.400367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:09:53.400367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:09:53.400367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:09:53.400367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:09:53.400367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:09:53.400367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:09:53.400367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:09:53.400367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:09:53.400367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:09:53.400367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:09:53.400367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:09:53.400367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:09:53.400943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:09:53.400943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:09:53.400943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:53.400943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:09:53.400943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:53.400943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:09:53.400943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:53.400943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:09:53.400943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:53.400943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:09:53.400943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:53.400943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:09:53.400943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:09:53.400943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:53.400943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:09:53.400943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:53.400943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:09:53.400943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:09:53.400943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:53.401521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:09:53.401521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:53.401521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:09:53.402180 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:09:53.402180 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:09:53.402345 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:09:53.402397 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:09:53.403510 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:09:53.403670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:53.404085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:09:53.404085 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:53.404085 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:09:53.404085 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:09:53.404085 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:53.404085 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:09:53.404085 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:53.404085 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:09:53.404930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:53.404930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:09:53.404930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:09:53.404930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:09:53.404930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:53.404930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:09:53.404930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:53.404930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:09:53.404930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:53.404930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:09:53.404930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:53.404930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:09:53.404930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:53.404930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:09:53.404930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:09:53.405805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:53.405805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:09:53.405805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:53.405805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:09:53.405805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:09:53.405805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:53.405805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:09:53.405805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:53.405805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:09:53.406246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:53.406246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:09:53.406246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:09:53.406246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:09:53.406246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:09:53.406246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:09:53.406246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:09:53.406246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:09:53.406246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:09:53.406246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:09:53.406246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:53.406246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:09:53.406840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:53.406840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:09:53.406840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:53.406840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:09:53.406840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:53.406840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:09:53.406840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:53.406840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:09:53.406840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:09:53.406840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:53.406840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:09:53.406840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:53.406840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:09:53.406840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:09:53.407419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:53.407419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:09:53.407419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:53.407419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:09:53.407419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:53.407419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:09:53.407419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:09:53.407419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:09:53.407419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:09:53.407419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:09:53.407419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:09:53.407419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:09:53.407419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:09:53.407419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:09:53.407664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:09:53.407664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:09:53.407664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:09:53.407664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:09:53.407664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:09:53.407664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:09:53.407664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:09:53.407664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:09:53.407664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:09:53.407664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:09:53.407664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:09:53.407664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:53.407664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:09:53.407664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:53.407664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:09:53.407664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:53.407664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:09:53.407664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:53.407664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:09:53.408046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:53.408046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:09:53.408046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:09:53.408046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:53.408046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:09:53.408046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:53.408046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:09:53.408046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:09:53.408046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:53.408046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:09:53.408046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:53.408046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:09:53.419638 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:09:53.420049 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:09:53.420202 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:09:53.420544 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:09:53.421500 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:09:53.421761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:53.422387 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:09:53.422387 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:53.422387 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:09:53.422387 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:09:53.422387 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:53.422387 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:09:53.422387 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:53.422387 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:09:53.423659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:53.423659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:09:53.423659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:09:53.423659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:09:53.423659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:53.423659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:09:53.423659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:53.423659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:09:53.423659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:53.423659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:09:53.423659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:53.423659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:09:53.423659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:53.423659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:09:53.423659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:09:53.424061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:53.424061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:09:53.424061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:53.424061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:09:53.424061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:09:53.424061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:53.424061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:09:53.424061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:53.424061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:09:53.425100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:53.425100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:09:53.425100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:09:53.425100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:09:53.425100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:09:53.425100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:09:53.425100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:09:53.425100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:09:53.425100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:09:53.425100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:53.425100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:09:53.425441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:53.425441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:09:53.425441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:53.425441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:09:53.425441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:53.425441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:09:53.425441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:53.425441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:09:53.425441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:09:53.425441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:53.425441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:09:53.425441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:53.425441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:09:53.425441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:09:53.425820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:53.425820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:09:53.425820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:53.425820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:09:53.426905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:53.426905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:09:53.426905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:09:53.426905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:09:53.426905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:09:53.426905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:09:53.426905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:09:53.426905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:09:53.426905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:09:53.426905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:09:53.426905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:09:53.426905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:09:53.426905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:09:53.426905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:09:53.426905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:09:53.426905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:09:53.426905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:09:53.427499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:09:53.427499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:09:53.427499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:09:53.427499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:53.427499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:09:53.427499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:53.427499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:09:53.427499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:53.427499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:09:53.427499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:53.427499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:09:53.427499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:53.427499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:09:53.427499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:09:53.427499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:53.427499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:09:53.427499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:53.427499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:09:53.427499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:09:53.428041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:53.428041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:09:53.428041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:53.428041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:09:53.428935 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:09:53.428935 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:09:53.428935 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:09:53.428935 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:09:53.429208 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:09:53.429443 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:09:53.429621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:53.430102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:09:53.430102 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:53.430102 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:09:53.430102 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:09:53.430102 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:53.430102 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:09:53.430102 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:53.430102 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:09:53.431010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:53.431010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:09:53.431010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:09:53.431010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:09:53.431010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:09:53.431010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:09:53.431010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:09:53.431010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:53.431010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:09:53.431010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:53.431010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:09:53.431010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:53.431733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:09:53.431733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:53.431733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:09:53.431733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:53.431733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:09:53.431733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:09:53.431733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:53.431733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:09:53.431733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:53.431733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:09:53.431733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:09:53.431733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:53.431733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:09:53.432274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:53.432274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:09:53.432274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:53.432274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:09:53.432274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:09:53.432274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:09:53.432274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:09:53.432274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:09:53.432274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:09:53.432274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:09:53.432274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:09:53.432274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:09:53.432274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:09:53.432274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:09:53.432274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:09:53.432633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:09:53.432633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:09:53.432633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:09:53.432633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:09:53.432633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:09:53.432633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:53.432633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:09:53.432633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:53.432633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:09:53.432633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:53.432633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:09:53.432633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:53.432633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:09:53.432633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:53.432633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:09:53.432633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:09:53.432633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:53.432633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:09:53.432633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:53.433531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:09:53.433531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:09:53.433531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:53.433531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:09:53.433531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:53.433531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:09:57.882523 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:09:57.883274 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:09:57.883274 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:09:57.884447 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:09:57.918486 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:09:57.918910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:57.919883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:09:57.919883 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:57.919883 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:09:57.919883 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:09:57.919883 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:57.919883 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:09:57.919883 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:57.919883 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:09:57.920651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:57.920651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:09:57.920651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:09:57.920651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:09:57.920651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:09:57.920651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:09:57.920651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:09:57.920651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:57.920651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:09:57.920651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:57.920651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:09:57.920651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:57.921289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:09:57.921289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:57.921289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:09:57.921289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:57.921289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:09:57.921289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:09:57.921289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:57.921289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:09:57.921289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:57.921289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:09:57.921289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:09:57.921289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:57.921289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:09:57.921896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:57.921896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:09:57.921896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:57.921896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:09:57.921896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:09:57.921896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:09:57.921896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:09:57.921896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:09:57.921896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:09:57.921896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:09:57.921896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:09:57.921896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:09:57.921896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:09:57.921896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:09:57.921896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:09:57.922608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:09:57.922608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:09:57.922608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:09:57.922608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:09:57.922608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:09:57.922608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:57.922608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:09:57.922608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:57.922608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:09:57.922608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:57.922608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:09:57.922608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:57.922608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:09:57.922608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:57.922608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:09:57.922608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:09:57.922608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:57.922608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:09:57.922608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:57.923510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:09:57.923510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:09:57.923510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:57.923510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:09:57.923510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:57.923510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:09:58.394319 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:09:58.394778 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:09:58.395027 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:09:58.396196 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:09:58.396587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:58.397537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:09:58.397537 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:58.397537 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:09:58.397537 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:09:58.397537 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:58.397537 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:09:58.397537 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:58.397537 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:09:58.399648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:58.399648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:09:58.399648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:09:58.399648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:09:58.399648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:09:58.399648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:09:58.399648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:09:58.399648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:58.399648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:09:58.399648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:58.399648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:09:58.399648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:58.400322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:09:58.400322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:58.400322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:09:58.400322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:58.400322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:09:58.400322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:09:58.400322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:58.400322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:09:58.400322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:58.400322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:09:58.400322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:09:58.400322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:58.400322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:09:58.400897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:58.400897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:09:58.401732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:58.401732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:09:58.401732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:09:58.401732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:09:58.401732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:09:58.401732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:09:58.401732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:09:58.401732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:09:58.401732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:09:58.401732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:09:58.401732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:09:58.401732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:09:58.401732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:09:58.401732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:09:58.401732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:09:58.401732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:09:58.402501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:09:58.402501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:09:58.402501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:58.402501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:09:58.402501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:58.402501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:09:58.402501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:58.402501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:09:58.402501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:58.402501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:09:58.402501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:58.402501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:09:58.402501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:09:58.402501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:58.402501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:09:58.402501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:58.402501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:09:58.402501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:09:58.402501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:58.403339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:09:58.403339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:58.403339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:09:58.403339 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:09:58.403339 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:09:58.403339 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:09:58.403339 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:09:58.403898 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:09:58.404133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:58.404890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:09:58.404890 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:58.404890 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:09:58.404890 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:09:58.404890 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:58.404890 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:09:58.404890 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:58.404890 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:09:58.405699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:58.405699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:09:58.405699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:09:58.405699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:09:58.405699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:58.405699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:09:58.405699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:58.405699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:09:58.405699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:58.405699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:09:58.405699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:58.405699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:09:58.405699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:58.405699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:09:58.405699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:09:58.406148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:58.406148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:09:58.406148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:58.406148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:09:58.406148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:09:58.406148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:58.406148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:09:58.406148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:58.406148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:09:58.406733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:58.406733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:09:58.406733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:09:58.406733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:09:58.406733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:09:58.406733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:09:58.406733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:09:58.406733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:09:58.406733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:09:58.406733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:09:58.406733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:58.406733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:09:58.407391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:58.407391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:09:58.407391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:58.407391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:09:58.407391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:58.407391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:09:58.407391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:58.407391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:09:58.407391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:09:58.407391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:58.407391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:09:58.407391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:58.407391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:09:58.407391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:09:58.407837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:58.407837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:09:58.407837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:58.407837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:09:58.408131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:58.408131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:09:58.408131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:09:58.408131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:09:58.408131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:09:58.408131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:09:58.408131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:09:58.408131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:09:58.408131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:09:58.408131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:09:58.408131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:09:58.408131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:09:58.408131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:09:58.408131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:09:58.408131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:09:58.408131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:09:58.408588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:09:58.408588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:09:58.408588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:09:58.408588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:09:58.408588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:09:58.408588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:58.408588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:09:58.408588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:58.408588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:09:58.408588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:58.408588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:09:58.408588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:58.408588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:09:58.408588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:58.408588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:09:58.408588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:09:58.408588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:58.408588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:09:58.409098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:58.409098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:09:58.409098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:09:58.409098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:58.409098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:09:58.409098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:58.409098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:09:58.419962 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:09:58.420179 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:09:58.420320 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:09:58.420480 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:09:58.421094 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:09:58.421479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:58.422444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:09:58.422444 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:58.422444 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:09:58.422444 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:09:58.422444 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:58.422444 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:09:58.422444 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:58.422444 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:09:58.423366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:58.423366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:09:58.423366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:09:58.423366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:09:58.423366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:58.423366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:09:58.423366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:58.423366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:09:58.423366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:58.423366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:09:58.423366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:58.423366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:09:58.423366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:58.423366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:09:58.423366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:09:58.423792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:58.423792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:09:58.423792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:58.423792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:09:58.423792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:09:58.423792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:58.423792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:09:58.423792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:58.423792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:09:58.425043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:58.425043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:09:58.425043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:09:58.425043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:09:58.425043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:09:58.425043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:09:58.425043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:09:58.425043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:09:58.425043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:09:58.425043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:58.425043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:09:58.425541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:58.425541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:09:58.425541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:58.425541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:09:58.425541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:58.425541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:09:58.425541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:58.425541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:09:58.425541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:09:58.425541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:58.425541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:09:58.425541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:58.425541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:09:58.425541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:09:58.426189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:58.426189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:09:58.426189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:58.426189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:09:58.426925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:58.426925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:09:58.426925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:09:58.426925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:09:58.426925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:09:58.426925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:09:58.426925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:09:58.426925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:09:58.426925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:09:58.426925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:09:58.426925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:09:58.426925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:09:58.426925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:09:58.426925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:09:58.426925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:09:58.426925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:09:58.426925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:09:58.427795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:09:58.427795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:09:58.427795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:09:58.427795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:58.427795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:09:58.427795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:58.427795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:09:58.427795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:58.427795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:09:58.427795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:58.427795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:09:58.427795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:58.427795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:09:58.427795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:09:58.427795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:58.427795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:09:58.427795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:58.427795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:09:58.427795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:09:58.428638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:58.428638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:09:58.428638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:58.428638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:09:58.428638 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:09:58.428936 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:09:58.429125 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:09:58.429221 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:09:58.429591 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:09:58.429897 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:09:58.430025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:58.431853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:09:58.431853 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:58.431853 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:09:58.431853 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:09:58.431853 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:58.431853 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:09:58.431853 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:58.431853 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:09:58.432775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:58.432775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:09:58.432775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:09:58.432775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:09:58.432775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:09:58.432775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:09:58.432775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:09:58.432775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:58.432775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:09:58.432775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:58.432775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:09:58.432775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:58.433135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:09:58.433135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:58.433135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:09:58.433135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:58.433135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:09:58.433135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:09:58.433135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:58.433135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:09:58.433135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:58.433135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:09:58.433135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:09:58.433135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:58.433135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:09:58.433513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:58.433513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:09:58.433971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:58.433971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:09:58.433971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:09:58.433971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:09:58.433971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:09:58.433971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:09:58.433971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:09:58.433971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:09:58.433971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:09:58.433971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:09:58.433971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:09:58.433971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:09:58.433971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:09:58.433971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:09:58.433971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:09:58.433971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:09:58.434419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:09:58.434419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:09:58.434419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:58.434419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:09:58.434419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:09:58.434419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:09:58.434419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:09:58.434419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:09:58.434419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:09:58.434419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:09:58.434419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:09:58.434419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:09:58.434419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:09:58.434419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:09:58.434419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:09:58.434419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:09:58.434419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:09:58.434419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:09:58.434419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:09:58.434964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:09:58.434964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:09:58.434964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:10:02.883154 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:10:02.884179 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:10:02.884179 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:10:02.884179 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:10:02.916429 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:10:02.916887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:02.917918 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:10:02.917918 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:02.917918 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:10:02.917918 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:10:02.917918 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:02.917918 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:10:02.917918 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:02.917918 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:10:02.918875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:02.918875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:10:02.918875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:10:02.918875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:10:02.918875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:10:02.918875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:10:02.918875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:10:02.918875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:02.918875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:10:02.918875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:02.918875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:10:02.918875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:02.919536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:10:02.919536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:02.919536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:10:02.919536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:02.919536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:10:02.919536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:10:02.919536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:02.919536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:10:02.919536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:02.919536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:10:02.919536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:10:02.919536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:02.919536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:10:02.931897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:02.931897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:10:02.931897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:02.931897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:10:02.931897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:10:02.931897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:10:02.931897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:10:02.931897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:10:02.931897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:10:02.931897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:10:02.931897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:10:02.931897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:10:02.931897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:10:02.931897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:10:02.931897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:10:02.933091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:10:02.933091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:10:02.933091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:10:02.933091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:10:02.933091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:10:02.933091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:02.933091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:10:02.933091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:02.933091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:10:02.933091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:02.933091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:10:02.933091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:02.933091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:10:02.933091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:02.933091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:10:02.933091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:10:02.933091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:02.933091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:10:02.933091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:02.933697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:10:02.933697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:10:02.933697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:02.933697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:10:02.933697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:02.933697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:10:03.396391 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:10:03.396391 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:10:03.396391 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:10:03.396391 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:10:03.399020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:03.399020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:10:03.399020 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:03.399020 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:10:03.399020 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:10:03.399020 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:03.399020 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:10:03.399020 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:03.399020 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:10:03.400765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:03.400765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:10:03.400765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:10:03.400765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:10:03.400765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:10:03.400765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:10:03.400765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:10:03.400765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:03.400765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:10:03.400765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:03.400765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:10:03.400765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:03.401660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:10:03.401660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:03.401660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:10:03.401660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:03.401660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:10:03.401660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:10:03.401660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:03.401660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:10:03.401660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:03.401660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:10:03.401660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:10:03.401660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:03.401660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:10:03.402342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:03.402342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:10:03.402342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:03.402342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:10:03.402342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:10:03.402342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:10:03.402342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:10:03.402342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:10:03.402342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:10:03.402342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:10:03.402342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:10:03.402342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:10:03.402342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:10:03.402342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:10:03.402342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:10:03.403079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:10:03.403079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:10:03.403079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:10:03.403079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:10:03.403079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:10:03.403079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:03.403079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:10:03.403079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:03.403079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:10:03.403079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:03.403079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:10:03.403079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:03.403079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:10:03.403079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:03.403079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:10:03.403079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:10:03.403079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:03.403079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:10:03.403079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:03.404353 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:10:03.404353 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:10:03.404353 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:03.404353 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:10:03.404353 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:03.404353 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:10:03.404353 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:10:03.404353 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:10:03.404353 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:10:03.404353 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:10:03.405779 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:10:03.405779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:03.405779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:10:03.405779 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:03.405779 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:10:03.405779 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:10:03.405779 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:03.405779 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:10:03.405779 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:03.405779 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:10:03.405779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:03.405779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:10:03.405779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:10:03.405779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:10:03.407330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:03.407330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:10:03.407330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:03.407330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:10:03.407330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:03.407330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:10:03.407330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:03.407330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:10:03.407330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:03.407330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:10:03.407330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:10:03.407330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:03.407330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:10:03.407330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:03.407330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:10:03.407330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:10:03.407330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:03.407330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:10:03.409283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:03.409283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:10:03.409283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:03.409283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:10:03.409283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:10:03.409283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:10:03.409283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:10:03.409283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:10:03.409283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:10:03.409283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:10:03.409283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:10:03.409283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:10:03.410197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:03.410197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:10:03.410197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:03.410197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:10:03.410197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:03.410197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:10:03.410197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:03.410197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:10:03.410197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:03.410197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:10:03.410197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:10:03.410197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:03.410197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:10:03.410197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:03.411114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:10:03.411114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:10:03.411114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:03.411114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:10:03.411114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:03.411114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:10:03.411114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:03.411114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:10:03.411114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:10:03.411114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:10:03.411114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:10:03.411114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:10:03.411114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:10:03.411114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:10:03.411875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:10:03.411875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:10:03.411875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:10:03.411875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:10:03.411875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:10:03.411875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:10:03.411875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:10:03.411875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:10:03.411875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:10:03.411875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:10:03.411875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:10:03.411875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:10:03.411875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:10:03.411875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:03.411875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:10:03.411875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:03.411875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:10:03.411875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:03.411875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:10:03.412604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:03.412604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:10:03.412604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:03.412604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:10:03.412604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:10:03.412604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:03.412604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:10:03.412604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:03.412604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:10:03.412604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:10:03.412604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:03.412604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:10:03.412604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:03.412604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:10:03.420410 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:10:03.420882 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:10:03.421075 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:10:03.421296 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:10:03.421878 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:10:03.422284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:03.423251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:10:03.423251 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:03.423251 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:10:03.423251 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:10:03.423251 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:03.423251 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:10:03.423251 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:03.423251 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:10:03.424633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:03.424633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:10:03.424633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:10:03.424633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:10:03.424633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:03.424633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:10:03.424633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:03.424633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:10:03.424633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:03.424633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:10:03.424633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:03.424633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:10:03.424633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:03.424633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:10:03.424633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:10:03.425570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:03.425570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:10:03.425570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:03.425570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:10:03.425570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:10:03.425570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:03.425570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:10:03.425570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:03.425570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:10:03.426350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:03.426350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:10:03.426350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:10:03.426350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:10:03.426350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:10:03.426350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:10:03.426350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:10:03.426350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:10:03.426350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:10:03.426350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:03.426350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:10:03.427129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:03.427129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:10:03.427129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:03.427129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:10:03.427129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:03.427129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:10:03.427129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:03.427129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:10:03.427129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:10:03.427129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:03.427129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:10:03.427129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:03.427129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:10:03.427129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:10:03.428449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:03.428449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:10:03.428449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:03.428449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:10:03.428449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:03.428449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:10:03.428449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:10:03.428449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:10:03.428449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:10:03.428449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:10:03.428449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:10:03.428449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:10:03.428449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:10:03.428449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:10:03.429364 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:10:03.429364 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:10:03.429364 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:10:03.429364 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:10:03.429364 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:10:03.429364 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:10:03.429364 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:10:03.429364 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:10:03.429364 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:10:03.429364 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:10:03.429364 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:03.429364 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:10:03.429364 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:03.429364 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:10:03.429364 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:03.429364 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:10:03.429364 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:03.429364 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:10:03.429364 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:03.430440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:10:03.430440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:10:03.430440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:03.430440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:10:03.430440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:03.430440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:10:03.430440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:10:03.430440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:03.430440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:10:03.430440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:03.430440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:10:03.430440 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:10:03.430440 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:10:03.431676 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:10:03.431676 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:10:03.431676 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:10:03.431676 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:10:03.431676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:03.431676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:10:03.431676 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:03.431676 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:10:03.431676 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:10:03.432072 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:03.432072 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:10:03.432072 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:03.432072 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:10:03.432072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:03.432072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:10:03.432072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:10:03.432072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:10:03.432072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:10:03.432072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:10:03.432072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:10:03.432072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:03.432072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:10:03.432072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:03.432590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:10:03.432590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:03.432590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:10:03.432590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:03.432590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:10:03.432590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:03.432590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:10:03.432590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:10:03.432590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:03.432590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:10:03.432590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:03.432590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:10:03.432590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:10:03.432590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:03.433072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:10:03.433072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:03.433072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:10:03.433072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:03.433072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:10:03.433072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:10:03.433072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:10:03.433072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:10:03.433072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:10:03.433072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:10:03.433072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:10:03.433072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:10:03.433072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:10:03.433072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:10:03.433072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:10:03.433072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:10:03.433632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:10:03.433632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:10:03.433632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:10:03.433632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:10:03.433632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:10:03.433632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:03.433632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:10:03.433632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:03.433632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:10:03.433632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:03.433632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:10:03.433632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:03.433632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:10:03.433632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:03.433632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:10:03.433632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:10:03.433632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:03.433632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:10:03.433632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:03.434328 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:10:03.434328 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:10:03.434328 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:03.434328 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:10:03.434328 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:03.434328 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:10:07.884095 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:10:07.884954 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:10:07.884954 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:10:07.886416 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:10:07.924460 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:10:07.924661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:07.925250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:10:07.925250 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:07.925250 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:10:07.925250 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:10:07.925250 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:07.925250 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:10:07.925250 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:07.925250 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:10:07.926336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:07.926336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:10:07.926336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:10:07.926336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:10:07.926336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:10:07.926336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:10:07.926336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:10:07.926336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:07.926336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:10:07.926336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:07.926336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:10:07.926336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:07.927057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:10:07.927057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:07.927057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:10:07.927057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:07.927057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:10:07.927057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:10:07.927057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:07.927057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:10:07.927057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:07.927057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:10:07.927057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:10:07.927057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:07.927057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:10:07.928131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:07.928131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:10:07.928131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:07.928131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:10:07.928131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:10:07.928131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:10:07.928131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:10:07.928131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:10:07.928131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:10:07.928131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:10:07.928131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:10:07.928131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:10:07.928131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:10:07.928131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:10:07.928131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:10:07.928976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:10:07.928976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:10:07.928976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:10:07.928976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:10:07.928976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:10:07.928976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:07.928976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:10:07.928976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:07.928976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:10:07.928976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:07.928976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:10:07.928976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:07.928976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:10:07.928976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:07.928976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:10:07.928976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:10:07.928976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:07.928976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:10:07.928976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:07.929974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:10:07.929974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:10:07.929974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:07.929974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:10:07.929974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:07.929974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:10:08.395777 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:10:08.396427 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:10:08.396615 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:10:08.398303 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:10:08.398515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:08.399164 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:10:08.399164 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:08.399164 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:10:08.399164 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:10:08.399164 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:08.399164 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:10:08.399164 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:08.399164 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:10:08.400663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:08.400663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:10:08.400663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:10:08.400663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:10:08.400663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:10:08.400663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:10:08.400663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:10:08.400663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:08.400663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:10:08.400663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:08.400663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:10:08.400663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:08.401281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:10:08.401281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:08.401281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:10:08.401281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:08.401281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:10:08.401281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:10:08.401281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:08.401281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:10:08.401281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:08.401281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:10:08.401281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:10:08.401281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:08.401281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:10:08.401831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:08.401831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:10:08.402400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:08.402400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:10:08.402400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:10:08.402400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:10:08.402400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:10:08.402400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:10:08.402400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:10:08.402400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:10:08.402400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:10:08.402400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:10:08.402400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:10:08.402400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:10:08.402400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:10:08.402400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:10:08.402400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:10:08.402400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:10:08.403153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:10:08.403153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:10:08.403153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:08.403153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:10:08.403153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:08.403153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:10:08.403153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:08.403153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:10:08.403153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:08.403153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:10:08.403153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:08.403153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:10:08.403153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:10:08.403153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:08.403153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:10:08.403153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:08.403153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:10:08.403153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:10:08.403153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:08.404002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:10:08.404002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:08.404002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:10:08.404002 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:10:08.404002 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:10:08.404002 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:10:08.404002 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:10:08.404566 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:10:08.405270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:08.405270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:10:08.405270 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:08.405270 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:10:08.405270 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:10:08.405270 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:08.405270 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:10:08.405270 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:08.405270 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:10:08.406254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:08.406254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:10:08.406254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:10:08.406254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:10:08.406254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:08.406254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:10:08.406254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:08.406254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:10:08.406254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:08.406254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:10:08.406254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:08.406254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:10:08.406254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:08.406254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:10:08.406254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:10:08.407864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:08.407864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:10:08.407864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:08.407864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:10:08.407864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:10:08.407864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:08.407864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:10:08.407864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:08.407864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:10:08.407864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:08.407864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:10:08.407864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:10:08.407864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:10:08.407864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:10:08.409281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:10:08.409281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:10:08.409281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:10:08.409281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:10:08.409281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:10:08.409281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:08.409281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:10:08.409281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:08.409281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:10:08.409281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:08.409281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:10:08.409281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:08.409281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:10:08.410156 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:08.410156 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:10:08.410156 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:10:08.410156 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:08.410156 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:10:08.410156 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:08.410156 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:10:08.410156 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:10:08.410156 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:08.410156 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:10:08.410156 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:08.410156 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:10:08.411329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:08.411329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:10:08.411329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:10:08.411329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:10:08.411329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:10:08.411329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:10:08.411329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:10:08.411329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:10:08.411329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:10:08.411329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:10:08.411329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:10:08.411329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:10:08.411329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:10:08.411329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:10:08.411329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:10:08.411329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:10:08.411788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:10:08.411788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:10:08.411788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:10:08.411788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:10:08.411788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:10:08.411788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:08.411788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:10:08.411788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:08.411788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:10:08.411788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:08.411788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:10:08.411788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:08.411788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:10:08.411788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:08.411788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:10:08.411788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:10:08.411788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:08.411788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:10:08.412101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:08.412101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:10:08.412101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:10:08.412101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:08.412101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:10:08.412101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:08.412101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:10:08.420945 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:10:08.421360 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:10:08.421557 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:10:08.421812 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:10:08.422133 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:10:08.422301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:08.422990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:10:08.422990 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:08.422990 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:10:08.422990 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:10:08.422990 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:08.422990 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:10:08.422990 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:08.422990 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:10:08.424900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:08.424900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:10:08.424900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:10:08.424900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:10:08.424900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:08.424900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:10:08.424900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:08.424900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:10:08.424900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:08.424900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:10:08.424900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:08.424900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:10:08.424900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:08.424900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:10:08.424900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:10:08.425547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:08.425547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:10:08.425547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:08.425547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:10:08.425547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:10:08.425547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:08.425547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:10:08.425547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:08.425547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:10:08.426857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:08.426857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:10:08.426857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:10:08.426857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:10:08.426857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:10:08.426857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:10:08.426857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:10:08.426857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:10:08.426857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:10:08.426857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:08.426857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:10:08.428273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:08.428273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:10:08.428273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:08.428273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:10:08.428273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:08.428273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:10:08.428273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:08.428273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:10:08.428273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:10:08.428273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:08.428273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:10:08.428273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:08.428273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:10:08.428273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:10:08.428889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:08.428889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:10:08.428889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:08.428889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:10:08.428889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:08.428889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:10:08.428889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:10:08.428889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:10:08.428889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:10:08.428889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:10:08.428889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:10:08.428889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:10:08.428889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:10:08.428889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:10:08.429644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:10:08.429644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:10:08.429644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:10:08.429644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:10:08.429644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:10:08.429644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:10:08.429644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:10:08.429644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:10:08.429644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:10:08.429644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:10:08.429644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:08.429644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:10:08.429644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:08.429644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:10:08.429644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:08.429644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:10:08.429644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:08.429644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:10:08.429644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:08.431197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:10:08.431197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:10:08.431197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:08.431197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:10:08.431197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:08.431197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:10:08.431197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:10:08.431197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:08.431197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:10:08.431197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:08.431197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:10:08.431197 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:10:08.431197 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:10:08.432304 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:10:08.432304 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:10:08.432304 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:10:08.432304 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:10:08.432304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:08.432304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:10:08.432304 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:08.432304 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:10:08.432304 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:10:08.432840 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:08.432840 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:10:08.432840 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:08.432840 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:10:08.432840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:08.432840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:10:08.432840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:10:08.432840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:10:08.432840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:10:08.432840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:10:08.432840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:10:08.432840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:08.432840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:10:08.432840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:08.433406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:10:08.433406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:08.433406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:10:08.433406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:08.433406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:10:08.433406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:08.433406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:10:08.433406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:10:08.433406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:08.433406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:10:08.433406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:08.433406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:10:08.433406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:10:08.433406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:08.433732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:10:08.433732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:08.433732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:10:08.433732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:08.433732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:10:08.433732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:10:08.433732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:10:08.433732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:10:08.433732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:10:08.433732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:10:08.433732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:10:08.433732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:10:08.433732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:10:08.433732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:10:08.433732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:10:08.433732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:10:08.434171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:10:08.434171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:10:08.434171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:10:08.434171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:10:08.434171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:10:08.434171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:08.434171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:10:08.434171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:08.434171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:10:08.434171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:08.434171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:10:08.434171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:08.434171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:10:08.434171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:08.434171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:10:08.434171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:10:08.434171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:08.434171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:10:08.434171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:08.435204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:10:08.435204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:10:08.435204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:08.435204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:10:08.435204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:08.435204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:10:12.884612 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:10:12.885563 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:10:12.885654 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:10:12.887089 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:10:12.916306 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:10:12.916706 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:12.917581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:10:12.917581 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:12.917581 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:10:12.917581 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:10:12.917581 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:12.917581 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:10:12.917581 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:12.917581 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:10:12.918427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.032 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:12.918427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:10:12.918427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:10:12.918427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:10:12.918427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:10:12.918427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:10:12.918427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:10:12.918427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:12.918427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:10:12.918427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:12.918427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:10:12.918427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:12.919021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:10:12.919021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:12.919021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:10:12.919021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:12.919021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:10:12.919021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:10:12.919021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:12.919021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:10:12.919021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:12.919021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:10:12.919021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:10:12.919021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:12.919021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:10:12.919638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:12.919638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:10:12.919638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:12.919638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:10:12.919638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:10:12.919638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:10:12.919638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:10:12.919638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:10:12.919638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:10:12.919638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:10:12.919638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:10:12.919638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:10:12.919638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:10:12.919638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:10:12.919638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:10:12.920302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:10:12.920302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:10:12.920302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:10:12.920302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:10:12.920302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:10:12.920302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:12.920302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:10:12.920302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:12.920302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:10:12.920302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:12.920302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:10:12.920302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:12.920302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:10:12.920302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:12.920302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:10:12.920302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:10:12.920302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:12.920302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:10:12.920302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:12.921105 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:10:12.921105 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:10:12.921105 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:12.921105 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:10:12.921105 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:12.921105 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:10:13.397575 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:10:13.398199 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:10:13.398394 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:10:13.399683 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:10:13.399970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:13.400553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:10:13.400553 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:13.400553 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:10:13.400553 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:10:13.400553 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:13.400553 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:10:13.400553 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:13.400553 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:10:13.401369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:13.401369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:10:13.401369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:10:13.401369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:10:13.401369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:10:13.401369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:10:13.401369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:10:13.401369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:13.401369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:10:13.401369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:13.401369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:10:13.401369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:13.401771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:10:13.401771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:13.401771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:10:13.401771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:13.401771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:10:13.401771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:10:13.401771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:13.401771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:10:13.401771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:13.401771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:10:13.401771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:10:13.401771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:13.401771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:10:13.402153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:13.402153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:10:13.402220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:13.402220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:10:13.402220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:10:13.402220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:10:13.402220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:10:13.402220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:10:13.402220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:10:13.402220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:10:13.402220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:10:13.402220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:10:13.402220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:10:13.402220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:10:13.402220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:10:13.402220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:10:13.402220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:10:13.402220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:10:13.402816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:10:13.402816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:10:13.402816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:13.402816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:10:13.402816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:13.402816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:10:13.402816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:13.402816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:10:13.402816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:13.402816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:10:13.402816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:13.402816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:10:13.402816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:10:13.402816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:13.402816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:10:13.402816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:13.402816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:10:13.402816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:10:13.402816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:13.403594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:10:13.403594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:13.403594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:10:13.403594 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:10:13.403719 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:10:13.403719 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:10:13.403827 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:10:13.404205 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:10:13.404419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:13.405013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:10:13.405013 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:13.405013 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:10:13.405013 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:10:13.405013 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:13.405013 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:10:13.405013 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:13.405013 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:10:13.405539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:13.405539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:10:13.405539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:10:13.405539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:10:13.405539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:13.405539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:10:13.405539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:13.405539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:10:13.405539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:13.405539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:10:13.405539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:13.405539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:10:13.405539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:13.405539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:10:13.405539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:10:13.406039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:13.406039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:10:13.406039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:13.406039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:10:13.406039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:10:13.406039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:13.406039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:10:13.406039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:13.406039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:10:13.406303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:13.406303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:10:13.406303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:10:13.406303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:10:13.406303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:10:13.406303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:10:13.406303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:10:13.406303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:10:13.406303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:10:13.406303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:10:13.406303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:13.406303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:10:13.406659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:13.406659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:10:13.406659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:13.406659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:10:13.406659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:13.406659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:10:13.406659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:13.406659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:10:13.406659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:10:13.406659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:13.406659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:10:13.406659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:13.406659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:10:13.406659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:10:13.407141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:13.407141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:10:13.407141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:13.407141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:10:13.407141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:13.407141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:10:13.407141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:10:13.407141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:10:13.407141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:10:13.407141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:10:13.407141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:10:13.407141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:10:13.407141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:10:13.407141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:10:13.407546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:10:13.407546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:10:13.407546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:10:13.407546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:10:13.407546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:10:13.407546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:10:13.407546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:10:13.407546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:10:13.407546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:10:13.407546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:10:13.407546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:10:13.407546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:13.407546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:10:13.407546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:13.407546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:10:13.407546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:13.407546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:10:13.407546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:13.407546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:10:13.408241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:13.408241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:10:13.408241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:10:13.408241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:13.408241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:10:13.408241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:13.408241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:10:13.408241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:10:13.408241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:13.408241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:10:13.408241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:13.408241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:10:13.421590 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:10:13.421940 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:10:13.422119 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:10:13.422437 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:10:13.423090 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:10:13.423483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:13.424381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:10:13.424381 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:13.424381 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:10:13.424381 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:10:13.424381 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:13.424381 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:10:13.424381 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:13.424381 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:10:13.425707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:13.425707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:10:13.425707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:10:13.425707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:10:13.425707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:13.425707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:10:13.425707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:13.425707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:10:13.425707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:13.425707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:10:13.425707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:13.425707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:10:13.425707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:13.425707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:10:13.425707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:10:13.426414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:13.426414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:10:13.426414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:13.426414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:10:13.426414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:10:13.426414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:13.426414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:10:13.426414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:13.426414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:10:13.427289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:13.427289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:10:13.427289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:10:13.427289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:10:13.427289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:10:13.427289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:10:13.427289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:10:13.427289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:10:13.427289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:10:13.427289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:13.427289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:10:13.427850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:13.427850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:10:13.427850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:13.427850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:10:13.427850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:13.427850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:10:13.427850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:13.427850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:10:13.427850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:10:13.427850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:13.427850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:10:13.427850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:13.427850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:10:13.427850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:10:13.428439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:13.428439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:10:13.428439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:13.428439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:10:13.429296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:13.429296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:10:13.429296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:10:13.429296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:10:13.429296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:10:13.429296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:10:13.429296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:10:13.429296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:10:13.429296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:10:13.429296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:10:13.429296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:10:13.429296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:10:13.429296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:10:13.429296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:10:13.429296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:10:13.429296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:10:13.429296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:10:13.429990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:10:13.429990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:10:13.429990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:10:13.429990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:13.429990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:10:13.429990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:13.429990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:10:13.429990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:13.429990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:10:13.429990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:13.429990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:10:13.429990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:13.429990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:10:13.429990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:10:13.429990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:13.429990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:10:13.429990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:13.429990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:10:13.429990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:10:13.430585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:13.430585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:10:13.430585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:13.430585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:10:13.430585 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:10:13.430585 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:10:13.430585 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:10:13.430585 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:10:13.430974 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:10:13.431325 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:10:13.431517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:13.432255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:10:13.432255 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:13.432255 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:10:13.432255 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:10:13.432255 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:13.432255 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:10:13.432255 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:13.432255 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:10:13.433184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:13.433184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:10:13.433184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:10:13.433184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:10:13.433184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:10:13.433184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:10:13.433184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:10:13.433184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:13.433184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:10:13.433184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:13.433184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:10:13.433184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:13.433554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:10:13.433554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:13.433554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:10:13.433554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:13.433554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:10:13.433554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:10:13.433554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:13.433554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:10:13.433554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:13.433554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:10:13.433554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:10:13.433554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:13.433554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:10:13.433924 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:13.433924 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:10:13.434218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:13.434218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:10:13.434218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:10:13.434218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:10:13.434218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:10:13.434218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:10:13.434218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:10:13.434218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:10:13.434218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:10:13.434218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:10:13.434218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:10:13.434218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:10:13.434218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:10:13.434218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:10:13.434218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:10:13.434218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:10:13.434654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:10:13.434654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:10:13.434654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:13.434654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:10:13.434654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:13.434654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:10:13.434654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:13.434654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:10:13.434654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:13.434654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:10:13.434654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:13.434654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:10:13.434654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:10:13.434654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:13.434654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:10:13.434654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:13.434654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:10:13.434654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:10:13.434654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:13.435172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:10:13.435172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:13.435172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:10:17.886933 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:10:17.886933 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:10:17.886933 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:10:17.886933 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:10:17.922342 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:10:17.922723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:17.923647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:10:17.923647 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:17.923647 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:10:17.923647 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:10:17.923647 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:17.923647 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:10:17.923647 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:17.923647 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:10:17.924836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:17.924836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:10:17.924836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:10:17.924836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:10:17.924836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:10:17.924836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:10:17.924836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:10:17.924836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:17.924836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:10:17.924836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:17.924836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:10:17.924836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:17.925517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:10:17.925517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:17.925517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:10:17.925517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:17.925517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:10:17.925517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:10:17.925517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:17.925517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:10:17.925517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:17.925517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:10:17.925517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:10:17.925517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:17.925517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:10:17.927143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:17.927143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:10:17.927143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:17.927143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:10:17.927143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:10:17.927143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:10:17.927143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:10:17.927143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:10:17.927143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:10:17.927143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:10:17.927143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:10:17.927143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:10:17.927143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:10:17.927143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:10:17.927143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:10:17.930425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:10:17.930425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:10:17.930425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:10:17.930425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:10:17.930425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:10:17.930425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:17.930425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:10:17.930425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:17.930425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:10:17.930425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:17.930425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:10:17.930425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:17.930425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:10:17.930425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:17.930425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:10:17.930425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:10:17.930425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:17.930425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:10:17.930425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:17.931855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:10:17.931855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:10:17.931855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:17.931855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:10:17.931855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:17.931855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:10:18.398308 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:10:18.398639 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:10:18.400345 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:10:18.400345 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:10:18.400345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:18.400844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:10:18.400844 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:18.400844 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:10:18.400844 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:10:18.400844 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:18.400844 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:10:18.400844 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:18.400844 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:10:18.402402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:18.402402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:10:18.402402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:10:18.402402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:10:18.402402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:10:18.402402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:10:18.402402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:10:18.402402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:18.402402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:10:18.402402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:18.402402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:10:18.402402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:18.403421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:10:18.403421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:18.403421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:10:18.403421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:18.403421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:10:18.403421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:10:18.403421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:18.403421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:10:18.403421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:18.403421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:10:18.403421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:10:18.403421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:18.403421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:10:18.404655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:18.404655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:10:18.404655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:18.404655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:10:18.404655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:10:18.404655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:10:18.404655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:10:18.404655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:10:18.404655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:10:18.404655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:10:18.404655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:10:18.404655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:10:18.404655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:10:18.404655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:10:18.404655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:10:18.406485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:10:18.406485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:10:18.406485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:10:18.406485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:10:18.406485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:10:18.406485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:18.406485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:10:18.406485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:18.406485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:10:18.406485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:18.406485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:10:18.406485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:18.406485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:10:18.406485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:18.406485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:10:18.406485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:10:18.406485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:18.406485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:10:18.406485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:18.407674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:10:18.407674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:10:18.407674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:18.407674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:10:18.407674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:18.407674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:10:18.407674 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:10:18.407674 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:10:18.407674 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:10:18.407674 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:10:18.408040 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:10:18.408040 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:18.408040 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:10:18.408040 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:18.408040 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:10:18.408040 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:10:18.408040 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:18.408040 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:10:18.408040 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:18.408040 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:10:18.408469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:18.408469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:10:18.408469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:10:18.408469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:10:18.408469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:18.408469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:10:18.408469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:18.408469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:10:18.408469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:18.408469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:10:18.408469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:18.408469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:10:18.408469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:18.408469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:10:18.408469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:10:18.410297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:18.410297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:10:18.410297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:18.410297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:10:18.410297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:10:18.410297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:18.410297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:10:18.410297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:18.410297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:10:18.410297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:18.410297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:10:18.410297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:10:18.410297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:10:18.410297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:10:18.411007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:10:18.411007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:10:18.411007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:10:18.411007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:10:18.411007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:10:18.411007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:18.411007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:10:18.411007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:18.411007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:10:18.411007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:18.411007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:10:18.411007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:18.411007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:10:18.411477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:18.411477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:10:18.411477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:10:18.411477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:18.411477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:10:18.411477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:18.411477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:10:18.411477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:10:18.411477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:18.411477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:10:18.411477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:18.411477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:10:18.412409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:18.412409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:10:18.412409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:10:18.412409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:10:18.412409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:10:18.412409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:10:18.412409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:10:18.412409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:10:18.412409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:10:18.412409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:10:18.412409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:10:18.412409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:10:18.412409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:10:18.412409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:10:18.412409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:10:18.412409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:10:18.414525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:10:18.414525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:10:18.414525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:10:18.414525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:10:18.414525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:10:18.414525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:18.414525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:10:18.414525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:18.414525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:10:18.414525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:18.414525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:10:18.414525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:18.414525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:10:18.414525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:18.414525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:10:18.414525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:10:18.414525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:18.414525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:10:18.415084 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:18.415084 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:10:18.415084 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:10:18.415084 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:18.415084 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:10:18.415084 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:18.415084 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:10:18.421887 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:10:18.422197 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:10:18.422269 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:10:18.422590 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:10:18.429566 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:10:18.429983 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:18.430424 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:10:18.433410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:10:18.433410 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:18.433410 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:10:18.433410 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:10:18.433410 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:18.433410 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:10:18.433410 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:18.433410 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:10:18.434759 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:18.434759 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:10:18.434759 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:10:18.434759 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:10:18.434759 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:18.434759 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:10:18.434759 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:18.434759 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:10:18.434759 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:18.434759 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:10:18.434759 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:18.434759 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:10:18.434759 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:18.434759 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:10:18.434759 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:10:18.435195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:18.435195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:10:18.435195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:18.435195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:10:18.435195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:10:18.435195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:18.435195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:10:18.435195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:18.435195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:10:18.436779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.012 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:18.436779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:10:18.436779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:10:18.436779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:10:18.436779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:10:18.436779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:10:18.436779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:10:18.436779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:10:18.436779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:10:18.436779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:18.436779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:10:18.437098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:18.437098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:10:18.437098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:18.437098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:10:18.437098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:18.437098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:10:18.437098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:18.437098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:10:18.437098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:10:18.437098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:18.437098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:10:18.437098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:18.437098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:10:18.437098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:10:18.437447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:18.437447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:10:18.437447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:18.437447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:10:18.438444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:18.438444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:10:18.438444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:10:18.438444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:10:18.438444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:10:18.438444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:10:18.438444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:10:18.438444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:10:18.438444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:10:18.438444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:10:18.438444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:10:18.438444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:10:18.438444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:10:18.438444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:10:18.438444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:10:18.438444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:10:18.438444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:10:18.438954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:10:18.438954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:10:18.438954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:10:18.438954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:18.438954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:10:18.438954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:18.438954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:10:18.438954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:18.438954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:10:18.438954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:18.438954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:10:18.438954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:18.438954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:10:18.438954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:10:18.438954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:18.438954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:10:18.438954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:18.438954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:10:18.438954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:10:18.439440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:18.439440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:10:18.439440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:18.439440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:10:18.439440 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:10:18.439440 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:10:18.439440 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:10:18.440805 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:10:18.441104 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:10:18.441235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:18.441899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:10:18.441899 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:18.441899 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:10:18.441899 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:10:18.441899 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:18.441899 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:10:18.441899 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:18.441899 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:10:18.443077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:18.443077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:10:18.443077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:10:18.443077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:10:18.443077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:10:18.443077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:10:18.443077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:10:18.443077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:18.443077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:10:18.443077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:18.443077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:10:18.443077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:18.443565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:10:18.443565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:18.443565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:10:18.443565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:18.443565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:10:18.443565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:10:18.443565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:18.443565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:10:18.443565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:18.443565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:10:18.443565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:10:18.443565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:18.443565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:10:18.444304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:18.444304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:10:18.444524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:18.444524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:10:18.444524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:10:18.444524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:10:18.444524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:10:18.444524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:10:18.444524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:10:18.444524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:10:18.444524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:10:18.444524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:10:18.444524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:10:18.444524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:10:18.444524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:10:18.444524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:10:18.444524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:10:18.444524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:10:18.445286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:10:18.445286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:10:18.445286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:18.445286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:10:18.445286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:18.445286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:10:18.445286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:18.445286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:10:18.445286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:18.445286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:10:18.445286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:18.445286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:10:18.445286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:10:18.445286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:18.445286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:10:18.445286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:18.445286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:10:18.445286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:10:18.445286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:18.445785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:10:18.445785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:18.445785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:10:22.886728 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:10:22.887346 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:10:22.887489 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:10:22.888499 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:10:22.922175 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:10:22.922673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:22.923599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:10:22.923599 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:22.923599 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:10:22.923599 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:10:22.923599 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:22.923599 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:10:22.923599 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:22.923599 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:10:22.924400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:22.924400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:10:22.924400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:10:22.924400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:10:22.924400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:10:22.924400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:10:22.924400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:10:22.924400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:22.924400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:10:22.924400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:22.924400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:10:22.924400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:22.924774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:10:22.924774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:22.924774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:10:22.924774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:22.924774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:10:22.924774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:10:22.924774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:22.924774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:10:22.924774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:22.924774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:10:22.924774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:10:22.924774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:22.924774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:10:22.925117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:22.925117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:10:22.925205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:22.925205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:10:22.925205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:10:22.925205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:10:22.925205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:10:22.925205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:10:22.925205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:10:22.925205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:10:22.925205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:10:22.925205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:10:22.925205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:10:22.925205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:10:22.925205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:10:22.925205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:10:22.925205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:10:22.925205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:10:22.925611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:10:22.925611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:10:22.925611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:22.925611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:10:22.925611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:22.925611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:10:22.925611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:22.925611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:10:22.925611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:22.925611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:10:22.925611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:22.925611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:10:22.925611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:10:22.925611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:22.925611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:10:22.925611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:22.925611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:10:22.925611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:10:22.925611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:22.926126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:10:22.926126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:22.926126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:10:23.401161 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:10:23.402198 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:10:23.402319 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:10:23.403150 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:10:23.403626 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:23.404402 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:10:23.406670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:10:23.406670 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:23.406670 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:10:23.406670 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:10:23.406670 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:23.406670 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:10:23.406670 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:23.406670 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:10:23.407100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:23.407100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:10:23.407100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:10:23.407100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:10:23.407100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:10:23.407100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:10:23.407100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:10:23.407100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:23.407100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:10:23.407100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:23.407100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:10:23.407100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:23.407651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:10:23.407651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:23.407651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:10:23.407651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:23.407651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:10:23.407651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:10:23.407651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:23.407651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:10:23.407651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:23.407651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:10:23.407651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:10:23.407651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:23.407651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:10:23.408408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:23.408408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:10:23.408408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:23.408408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:10:23.408408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:10:23.408408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:10:23.408408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:10:23.408408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:10:23.408408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:10:23.408408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:10:23.408408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:10:23.408408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:10:23.408408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:10:23.408408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:10:23.408408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:10:23.409661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:10:23.409661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:10:23.409661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:10:23.409661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:10:23.409661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:10:23.409661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:23.409661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:10:23.409661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:23.409661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:10:23.409661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:23.409661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:10:23.409661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:23.409661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:10:23.409661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:23.409661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:10:23.409661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:10:23.409661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:23.409661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:10:23.409661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:23.410957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:10:23.410957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:10:23.410957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:23.410957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:10:23.410957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:23.410957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:10:23.410957 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:10:23.410957 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:10:23.410957 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:10:23.410957 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:10:23.413896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:23.413896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:10:23.413896 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:23.413896 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:10:23.413896 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:10:23.413896 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:23.413896 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:10:23.413896 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:23.413896 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:10:23.413896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:23.413896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:10:23.413896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:10:23.413896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:10:23.413896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:23.413896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:10:23.413896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:23.413896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:10:23.414557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:23.414557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:10:23.414557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:23.414557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:10:23.414557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:23.414557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:10:23.414557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:10:23.414557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:23.414557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:10:23.414557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:23.414557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:10:23.414557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:10:23.414557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:23.414557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:10:23.414557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:23.414557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:10:23.415050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:23.415050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:10:23.415050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:10:23.415050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:10:23.415050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:10:23.415050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:10:23.415050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:10:23.415050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:10:23.415050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:10:23.415050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:10:23.415050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:23.415050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:10:23.415409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:23.415409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:10:23.415409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:23.415409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:10:23.415409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:23.415409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:10:23.415409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:23.415409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:10:23.415409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:10:23.415409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:23.415409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:10:23.415409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:23.415409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:10:23.415409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:10:23.415809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:23.415809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:10:23.415809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:23.415809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:10:23.415809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:23.415809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:10:23.415809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:10:23.415809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:10:23.415809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:10:23.415809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:10:23.415809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:10:23.415809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:10:23.415809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:10:23.415809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:10:23.416209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:10:23.416209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:10:23.416209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:10:23.416209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:10:23.416209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:10:23.416209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:10:23.416209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:10:23.416209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:10:23.416209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:10:23.416209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:10:23.416209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:10:23.416209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:23.416209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:10:23.416209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:23.416209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:10:23.416209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:23.416209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:10:23.416209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:23.416209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:10:23.416731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:23.416731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:10:23.416731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:10:23.416731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:23.416731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:10:23.416731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:23.416731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:10:23.416731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:10:23.416731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:23.416731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:10:23.416731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:23.416731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:10:23.422396 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:10:23.423505 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:10:23.423652 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:10:23.423842 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:10:23.424588 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:10:23.425079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:23.425942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:10:23.425942 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:23.425942 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:10:23.425942 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:10:23.425942 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:23.425942 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:10:23.425942 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:23.425942 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:10:23.427001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:23.427001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:10:23.427001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:10:23.427001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:10:23.427001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:23.427001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:10:23.427001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:23.427001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:10:23.427001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:23.427001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:10:23.427001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:23.427001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:10:23.427001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:23.427001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:10:23.427001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:10:23.427401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:23.427401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:10:23.427401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:23.427401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:10:23.427401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:10:23.427401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:23.427401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:10:23.427401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:23.427401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:10:23.427959 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:23.427959 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:10:23.427959 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:10:23.427959 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:10:23.427959 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:10:23.427959 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:10:23.427959 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:10:23.427959 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:10:23.427959 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:10:23.427959 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:23.427959 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:10:23.428300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:23.428300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:10:23.428300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:23.428300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:10:23.428300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:23.428300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:10:23.428300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:23.428300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:10:23.428300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:10:23.428300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:23.428300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:10:23.428300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:23.428300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:10:23.428300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:10:23.428771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:23.428771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:10:23.428771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:23.428771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:10:23.429410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:23.429410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:10:23.429410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:10:23.429410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:10:23.429410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:10:23.429410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:10:23.429410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:10:23.429410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:10:23.429410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:10:23.429410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:10:23.429410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:10:23.429410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:10:23.429410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:10:23.429410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:10:23.429410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:10:23.429410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:10:23.429410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:10:23.429995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:10:23.429995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:10:23.429995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:10:23.429995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:23.429995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:10:23.429995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:23.429995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:10:23.429995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:23.429995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:10:23.429995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:23.429995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:10:23.429995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:23.429995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:10:23.429995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:10:23.429995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:23.429995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:10:23.429995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:23.429995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:10:23.429995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:10:23.430462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:23.430462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:10:23.430462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:23.430462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:10:23.439020 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:10:23.439400 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:10:23.439606 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:10:23.439944 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:10:23.440737 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:10:23.441368 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:10:23.441629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:23.442607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:10:23.442607 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:23.442607 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:10:23.442607 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:10:23.442607 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:23.442607 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:10:23.442607 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:23.442607 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:10:23.444023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:23.444023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:10:23.444023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:10:23.444023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:10:23.444023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:10:23.444023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:10:23.444023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:10:23.444023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:23.444023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:10:23.444023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:23.444023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:10:23.444023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:23.444725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:10:23.444725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:23.444725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:10:23.444725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:23.444725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:10:23.444725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:10:23.444725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:23.444725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:10:23.444725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:23.444725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:10:23.444725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:10:23.444725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:23.444725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:10:23.445409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:23.445409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:10:23.446388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:23.446388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:10:23.446388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:10:23.446388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:10:23.446388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:10:23.446388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:10:23.446388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:10:23.446388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:10:23.446388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:10:23.446388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:10:23.446388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:10:23.446388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:10:23.446388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:10:23.446388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:10:23.446388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:10:23.446388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:10:23.447191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:10:23.447191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:10:23.447191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:23.447191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:10:23.447191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:23.447191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:10:23.447191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:23.447191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:10:23.447191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:23.447191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:10:23.447191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:23.447191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:10:23.447191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:10:23.447191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:23.447191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:10:23.447191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:23.447191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:10:23.447191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:10:23.447191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:23.448029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:10:23.448029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:23.448029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:10:27.887676 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:10:27.888693 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:10:27.888827 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:10:27.890436 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:10:27.931360 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:10:27.931934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:27.932962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:10:27.932962 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:27.932962 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:10:27.932962 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:10:27.932962 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:27.932962 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:10:27.932962 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:27.932962 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:10:27.933915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.044 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:27.933915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:10:27.933915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:10:27.933915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:10:27.933915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:10:27.933915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:10:27.933915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:10:27.933915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:27.933915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:10:27.933915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:27.933915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:10:27.933915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:27.934716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:10:27.934716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:27.934716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:10:27.934716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:27.934716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:10:27.934716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:10:27.934716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:27.934716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:10:27.934716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:27.934716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:10:27.934716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:10:27.934716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:27.934716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:10:27.935498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:27.935498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:10:27.935498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:27.935498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:10:27.935498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:10:27.935498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:10:27.935498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:10:27.935498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:10:27.935498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:10:27.935498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:10:27.935498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:10:27.935498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:10:27.935498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:10:27.935498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:10:27.935498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:10:27.936363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:10:27.936363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:10:27.936363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:10:27.936363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:10:27.936363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:10:27.936363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:27.936363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:10:27.936363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:27.936363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:10:27.936363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:27.936363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:10:27.936363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:27.936363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:10:27.936363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:27.936363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:10:27.936363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:10:27.936363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:27.936363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:10:27.936363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:27.937045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:10:27.937045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:10:27.937045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:27.937045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:10:27.937045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:27.937045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:10:28.402498 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:10:28.403159 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:10:28.403355 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:10:28.405104 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:10:28.405574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:28.406712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:10:28.406712 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:28.406712 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:10:28.406712 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:10:28.406712 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:28.406712 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:10:28.406712 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:28.406712 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:10:28.408640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:28.408640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:10:28.408640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:10:28.408640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:10:28.408640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:10:28.408640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:10:28.408640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:10:28.408640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:28.408640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:10:28.408640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:28.408640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:10:28.408640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:28.409294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:10:28.409294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:28.409294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:10:28.409294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:28.409294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:10:28.409294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:10:28.409294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:28.409294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:10:28.409294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:28.409294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:10:28.409294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:10:28.409294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:28.409294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:10:28.409848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:28.409848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:10:28.410704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:28.410704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:10:28.410704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:10:28.410704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:10:28.410704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:10:28.410704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:10:28.410704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:10:28.410704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:10:28.410704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:10:28.410704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:10:28.410704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:10:28.410704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:10:28.410704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:10:28.410704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:10:28.410704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:10:28.410704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:10:28.411503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:10:28.411503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:10:28.411503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:28.411503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:10:28.411503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:28.411503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:10:28.411503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:28.411503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:10:28.411503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:28.411503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:10:28.411503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:28.411503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:10:28.411503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:10:28.411503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:28.411503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:10:28.411503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:28.411503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:10:28.411503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:10:28.411503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:28.412050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:10:28.412050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:28.412050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:10:28.412050 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:10:28.412050 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:10:28.412050 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:10:28.412050 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:10:28.412050 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:10:28.412284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:28.412916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:10:28.412916 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:28.412916 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:10:28.412916 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:10:28.412916 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:28.412916 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:10:28.412916 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:28.412916 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:10:28.413716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:28.413716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:10:28.413716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:10:28.413716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:10:28.413716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:28.413716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:10:28.413716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:28.413716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:10:28.413716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:28.413716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:10:28.413716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:28.413716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:10:28.413716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:28.413716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:10:28.413716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:10:28.414140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:28.414140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:10:28.414140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:28.414140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:10:28.414140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:10:28.414140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:28.414140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:10:28.414140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:28.414140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:10:28.414680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:28.414680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:10:28.414680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:10:28.414680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:10:28.414680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:10:28.414680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:10:28.414680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:10:28.414680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:10:28.414680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:10:28.414680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:10:28.414680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:28.414680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:10:28.415026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:28.415026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:10:28.415026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:28.415026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:10:28.415026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:28.415026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:10:28.415026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:28.415026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:10:28.415026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:10:28.415026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:28.415026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:10:28.415026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:28.415026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:10:28.415026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:10:28.415367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:28.415367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:10:28.415367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:28.415367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:10:28.415853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:28.415853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:10:28.415853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:10:28.415853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:10:28.415853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:10:28.415853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:10:28.415853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:10:28.415853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:10:28.415853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:10:28.415853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:10:28.415853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:10:28.415853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:10:28.415853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:10:28.415853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:10:28.415853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:10:28.415853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:10:28.416372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:10:28.416372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:10:28.416372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:10:28.416372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:10:28.416372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:10:28.416372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:28.416372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:10:28.416372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:28.416372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:10:28.416372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:28.416372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:10:28.416372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:28.416372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:10:28.416372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:28.416372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:10:28.416372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:10:28.416372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:28.416372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:10:28.416848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:28.416848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:10:28.416848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:10:28.416848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:28.416848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:10:28.416848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:28.416848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:10:28.423602 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:10:28.424013 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:10:28.424182 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:10:28.424378 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:10:28.425004 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:10:28.425453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:28.426648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:10:28.426648 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:28.426648 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:10:28.426648 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:10:28.426648 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:28.426648 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:10:28.426648 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:28.426648 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:10:28.427931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:28.427931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:10:28.427931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:10:28.427931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:10:28.427931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:28.427931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:10:28.427931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:28.427931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:10:28.427931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:28.427931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:10:28.427931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:28.427931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:10:28.427931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:28.427931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:10:28.427931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:10:28.428371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:28.428371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:10:28.428371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:28.428371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:10:28.428371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:10:28.428371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:28.428371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:10:28.428371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:28.428371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:10:28.429285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:28.429285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:10:28.429285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:10:28.429285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:10:28.429285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:10:28.429285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:10:28.429285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:10:28.429285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:10:28.429285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:10:28.429285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:28.429285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:10:28.429660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:28.429660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:10:28.429660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:28.429660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:10:28.429660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:28.429660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:10:28.429660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:28.429660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:10:28.429660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:10:28.429660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:28.429660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:10:28.429660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:28.429660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:10:28.429660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:10:28.430173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:28.430173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:10:28.430173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:28.430173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:10:28.430766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:28.430766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:10:28.430766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:10:28.430766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:10:28.430766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:10:28.430766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:10:28.430766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:10:28.430766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:10:28.430766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:10:28.430766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:10:28.430766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:10:28.430766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:10:28.430766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:10:28.430766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:10:28.430766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:10:28.430766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:10:28.430766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:10:28.431200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:10:28.431200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:10:28.431200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:10:28.431200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:28.431200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:10:28.431200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:28.431200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:10:28.431200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:28.431200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:10:28.431200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:28.431200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:10:28.431200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:28.431200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:10:28.431200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:10:28.431200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:28.431200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:10:28.431200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:28.431200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:10:28.431200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:10:28.431983 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:28.431983 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:10:28.431983 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:28.431983 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:10:28.439551 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:10:28.439938 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:10:28.440062 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:10:28.440254 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:10:28.441266 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:10:28.441759 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:10:28.441995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:28.442897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:10:28.442897 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:28.442897 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:10:28.442897 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:10:28.442897 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:28.442897 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:10:28.442897 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:28.442897 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:10:28.443948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:28.443948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:10:28.443948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:10:28.443948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:10:28.443948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:10:28.443948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:10:28.443948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:10:28.443948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:28.443948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:10:28.443948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:28.443948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:10:28.443948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:28.444494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:10:28.444494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:28.444494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:10:28.444494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:28.444494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:10:28.444494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:10:28.444494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:28.444494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:10:28.444494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:28.444494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:10:28.444494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:10:28.444494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:28.444494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:10:28.445157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:28.445157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:10:28.445157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:28.445157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:10:28.445157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:10:28.445157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:10:28.445157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:10:28.445157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:10:28.445157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:10:28.445157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:10:28.445157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:10:28.445157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:10:28.445157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:10:28.445157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:10:28.445157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:10:28.445856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:10:28.445856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:10:28.445856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:10:28.445856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:10:28.445856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:10:28.445856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:28.445856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:10:28.445856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:28.445856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:10:28.445856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:28.445856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:10:28.445856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:28.445856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:10:28.445856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:28.445856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:10:28.445856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:10:28.445856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:28.445856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:10:28.445856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:28.446715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:10:28.446715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:10:28.446715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:28.446715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:10:28.446715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:28.446715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:10:32.888553 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:10:32.889858 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:10:32.889858 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:10:32.891153 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:10:32.924535 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:10:32.924837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:32.925518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:10:32.925518 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:32.925518 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:10:32.925518 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:10:32.925518 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:32.925518 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:10:32.925518 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:32.925518 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:10:32.926389 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:32.926389 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:10:32.926389 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:10:32.926389 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:10:32.926389 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:10:32.926389 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:10:32.926389 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:10:32.926389 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:32.926389 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:10:32.926389 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:32.926389 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:10:32.926389 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:32.926812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:10:32.926812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:32.926812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:10:32.926812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:32.926812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:10:32.926812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:10:32.926812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:32.926812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:10:32.926812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:32.926812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:10:32.926812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:10:32.926812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:32.926812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:10:32.927205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:32.927205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:10:32.927463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:32.927463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:10:32.927463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:10:32.927463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:10:32.927463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:10:32.927463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:10:32.927463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:10:32.927463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:10:32.927463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:10:32.927463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:10:32.927463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:10:32.927463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:10:32.927463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:10:32.927463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:10:32.927463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:10:32.927463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:10:32.927836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:10:32.927836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:10:32.927836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:32.927836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:10:32.927836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:32.927836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:10:32.927836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:32.927836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:10:32.927836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:32.927836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:10:32.927836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:32.927836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:10:32.927836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:10:32.927836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:32.927836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:10:32.927836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:32.927836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:10:32.927836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:10:32.927836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:32.928196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:10:32.928196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:32.928196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:10:33.403135 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:10:33.403537 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:10:33.403780 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:10:33.404864 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:10:33.405206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:33.406151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:10:33.406151 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:33.406151 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:10:33.406151 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:10:33.406151 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:33.406151 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:10:33.406151 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:33.406151 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:10:33.407979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:33.407979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:10:33.407979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:10:33.407979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:10:33.407979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:10:33.407979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:10:33.407979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:10:33.407979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:33.407979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:10:33.407979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:33.407979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:10:33.407979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:33.408547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:10:33.408547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:33.408547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:10:33.408547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:33.408547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:10:33.408547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:10:33.408547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:33.408547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:10:33.408547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:33.408547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:10:33.408547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:10:33.408547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:33.408547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:10:33.409147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:33.409147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:10:33.409760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:33.409760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:10:33.409760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:10:33.409760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:10:33.409760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:10:33.409760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:10:33.409760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:10:33.409760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:10:33.409760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:10:33.409760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:10:33.409760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:10:33.409760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:10:33.409760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:10:33.409760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:10:33.409760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:10:33.409760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:10:33.410517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:10:33.410517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:10:33.410517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:33.410517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:10:33.410517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:33.410517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:10:33.410517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:33.410517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:10:33.410517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:33.410517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:10:33.410517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:33.410517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:10:33.410517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:10:33.410517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:33.410517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:10:33.410517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:33.410517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:10:33.410517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:10:33.410517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:33.411459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:10:33.411459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:33.411459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:10:33.411459 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:10:33.411870 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:10:33.412045 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:10:33.412227 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:10:33.412835 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:10:33.413149 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:33.413967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:10:33.413967 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:33.413967 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:10:33.413967 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:10:33.413967 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:33.413967 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:10:33.413967 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:33.413967 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:10:33.415225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:33.415225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:10:33.415225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:10:33.415225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:10:33.415225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:33.415225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:10:33.415225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:33.415225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:10:33.415225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:33.415225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:10:33.415225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:33.415225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:10:33.415225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:33.415225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:10:33.415225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:10:33.415661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:33.415661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:10:33.415661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:33.415661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:10:33.415661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:10:33.415661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:33.415661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:10:33.415661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:33.415661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:10:33.416810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:33.416810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:10:33.416810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:10:33.416810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:10:33.416810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:10:33.416810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:10:33.416810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:10:33.416810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:10:33.416810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:10:33.416810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:10:33.416810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:33.416810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:10:33.417242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:33.417242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:10:33.417242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:33.417242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:10:33.417242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:33.417242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:10:33.417242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:33.417242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:10:33.417242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:10:33.417242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:33.417242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:10:33.417242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:33.417242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:10:33.417242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:10:33.417760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:33.417760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:10:33.417760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:33.417760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:10:33.417943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:33.417943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:10:33.417943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:10:33.417943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:10:33.417943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:10:33.417943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:10:33.417943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:10:33.417943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:10:33.417943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:10:33.417943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:10:33.417943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:10:33.417943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:10:33.417943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:10:33.417943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:10:33.417943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:10:33.417943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:10:33.418473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:10:33.418473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:10:33.418473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:10:33.418473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:10:33.418473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:10:33.418473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:33.418473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:10:33.418473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:33.418473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:10:33.418473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:33.418473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:10:33.418473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:33.418473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:10:33.418473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:33.418473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:10:33.418473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:10:33.418473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:33.418473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:10:33.418969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:33.418969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:10:33.418969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:10:33.418969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:33.418969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:10:33.418969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:33.418969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:10:33.423922 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:10:33.424134 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:10:33.424237 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:10:33.424363 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:10:33.424759 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:10:33.425003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:33.425580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:10:33.425580 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:33.425580 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:10:33.425580 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:10:33.425580 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:33.425580 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:10:33.425580 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:33.425580 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:10:33.426862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:33.426862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:10:33.426862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:10:33.426862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:10:33.426862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:33.426862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:10:33.426862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:33.426862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:10:33.426862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:33.426862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:10:33.426862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:33.426862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:10:33.426862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:33.426862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:10:33.426862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:10:33.427300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:33.427300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:10:33.427300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:33.427300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:10:33.427300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:10:33.427300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:33.427300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:10:33.427300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:33.427300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:10:33.428456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:33.428456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:10:33.428456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:10:33.428456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:10:33.428456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:10:33.428456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:10:33.428456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:10:33.428456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:10:33.428456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:10:33.428456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:33.428456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:10:33.428863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:33.428863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:10:33.428863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:33.428863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:10:33.428863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:33.428863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:10:33.428863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:33.428863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:10:33.428863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:10:33.428863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:33.428863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:10:33.428863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:33.428863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:10:33.428863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:10:33.429231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:33.429231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:10:33.429231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:33.429231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:10:33.430381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:33.430381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:10:33.430381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:10:33.430381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:10:33.430381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:10:33.430381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:10:33.430381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:10:33.430381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:10:33.430381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:10:33.430381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:10:33.430381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:10:33.430381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:10:33.430381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:10:33.430381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:10:33.430381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:10:33.430381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:10:33.430381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:10:33.430919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:10:33.430919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:10:33.430919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:10:33.430919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:33.430919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:10:33.430919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:33.430919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:10:33.430919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:33.430919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:10:33.430919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:33.430919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:10:33.430919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:33.430919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:10:33.430919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:10:33.430919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:33.430919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:10:33.430919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:33.430919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:10:33.430919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:10:33.431434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:33.431434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:10:33.431434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:33.431434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:10:33.440027 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:10:33.440279 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:10:33.440525 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:10:33.440770 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:10:33.441270 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:10:33.441648 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:10:33.441867 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:33.442582 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:10:33.442582 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:33.442582 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:10:33.442582 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:10:33.442582 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:33.442582 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:10:33.442582 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:33.442582 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:10:33.443958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:33.443958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:10:33.443958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:10:33.443958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:10:33.443958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:10:33.443958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:10:33.443958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:10:33.443958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:33.443958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:10:33.443958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:33.443958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:10:33.443958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:33.444533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:10:33.444533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:33.444533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:10:33.444533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:33.444533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:10:33.444533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:10:33.444533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:33.444533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:10:33.444533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:33.444533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:10:33.444533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:10:33.444533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:33.444533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:10:33.445098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:33.445098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:10:33.445524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:33.445524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:10:33.445524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:10:33.445524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:10:33.445524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:10:33.445524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:10:33.445524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:10:33.445524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:10:33.445524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:10:33.445524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:10:33.445524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:10:33.445524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:10:33.445524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:10:33.445524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:10:33.445524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:10:33.445524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:10:33.446313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:10:33.446313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:10:33.446313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:33.446313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:10:33.446313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:33.446313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:10:33.446313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:33.446313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:10:33.446313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:33.446313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:10:33.446313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:33.446313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:10:33.446313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:10:33.446313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:33.446313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:10:33.446313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:33.446313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:10:33.446313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:10:33.446313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:33.447082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:10:33.447082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:33.447082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:10:37.889194 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:10:37.890004 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:10:37.890004 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:10:37.891277 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:10:37.909599 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG dbcounter [-] [106314] Writing DB stats neutron:SELECT=303,neutron:UPDATE=1 {{(pid=106314) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 09 09:10:37.931951 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:10:37.932351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:37.933307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:10:37.933307 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:37.933307 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:10:37.933307 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:10:37.933307 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:37.933307 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:10:37.933307 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:37.933307 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:10:37.934344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.043 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:37.934344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:10:37.934344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:10:37.934344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:10:37.934344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:10:37.934344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:10:37.934344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:10:37.934344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:37.934344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:10:37.934344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:37.934344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:10:37.934344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:37.934730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:10:37.934730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:37.934730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:10:37.934730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:37.934730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:10:37.934730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:10:37.934730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:37.934730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:10:37.934730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:37.934730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:10:37.934730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:10:37.934730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:37.934730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:10:37.935148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:37.935148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:10:37.935472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:37.935472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:10:37.935472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:10:37.935472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:10:37.935472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:10:37.935472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:10:37.935472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:10:37.935472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:10:37.935472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:10:37.935472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:10:37.935472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:10:37.935472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:10:37.935472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:10:37.935472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:10:37.935472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:10:37.935472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:10:37.935953 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:10:37.935953 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:10:37.935953 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:37.935953 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:10:37.935953 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:37.935953 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:10:37.935953 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:37.935953 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:10:37.935953 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:37.935953 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:10:37.935953 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:37.935953 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:10:37.935953 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:10:37.935953 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:37.935953 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:10:37.935953 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:37.935953 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:10:37.935953 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:10:37.935953 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:37.936493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:10:37.936493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:37.936493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:10:38.403779 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:10:38.404658 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:10:38.404967 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:10:38.407483 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:10:38.408113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:38.409188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:10:38.409188 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:38.409188 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:10:38.409188 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:10:38.409188 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:38.409188 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:10:38.409188 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:38.409188 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:10:38.410701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:38.410701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:10:38.410701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:10:38.410701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:10:38.410701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:10:38.410701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:10:38.410701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:10:38.410701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:38.410701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:10:38.410701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:38.410701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:10:38.410701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:38.411572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:10:38.411572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:38.411572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:10:38.411572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:38.411572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:10:38.411572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:10:38.411572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:38.411572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:10:38.411572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:38.411572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:10:38.411572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:10:38.411572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:38.411572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:10:38.412419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:38.412419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:10:38.412562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:38.412562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:10:38.412562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:10:38.412562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:10:38.412562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:10:38.412562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:10:38.412562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:10:38.412562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:10:38.412562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:10:38.412562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:10:38.412562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:10:38.412562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:10:38.412562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:10:38.412562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:10:38.412562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:10:38.412562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:10:38.413471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:10:38.413471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:10:38.413471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:38.413471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:10:38.413471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:38.413471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:10:38.413471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:38.413471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:10:38.413471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:38.413471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:10:38.413471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:38.413471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:10:38.413471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:10:38.413471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:38.413471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:10:38.413471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:38.413471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:10:38.413471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:10:38.413471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:38.414792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:10:38.414792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:38.414792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:10:38.414792 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:10:38.414792 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:10:38.414792 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:10:38.414792 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:10:38.415318 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:10:38.415604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:38.416436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:10:38.416436 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:38.416436 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:10:38.416436 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:10:38.416436 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:38.416436 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:10:38.416436 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:38.416436 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:10:38.417157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:38.417157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:10:38.417157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:10:38.417157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:10:38.417157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:38.417157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:10:38.417157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:38.417157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:10:38.417157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:38.417157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:10:38.417157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:38.417157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:10:38.417157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:38.417157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:10:38.417157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:10:38.417780 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:38.417780 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:10:38.417780 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:38.417780 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:10:38.417780 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:10:38.417780 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:38.417780 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:10:38.417780 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:38.417780 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:10:38.418249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:38.418249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:10:38.418249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:10:38.418249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:10:38.418249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:10:38.418249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:10:38.418249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:10:38.418249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:10:38.418249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:10:38.418249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:10:38.418249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:38.418249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:10:38.418767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:38.418767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:10:38.418767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:38.418767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:10:38.418767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:38.418767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:10:38.418767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:38.418767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:10:38.418767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:10:38.418767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:38.418767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:10:38.418767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:38.418767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:10:38.418767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:10:38.419310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:38.419310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:10:38.419310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:38.419310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:10:38.419310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:38.419310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:10:38.419310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:10:38.419310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:10:38.419310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:10:38.419310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:10:38.419310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:10:38.419310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:10:38.419310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:10:38.419310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:10:38.419877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:10:38.419877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:10:38.419877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:10:38.419877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:10:38.419877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:10:38.419877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:10:38.419877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:10:38.419877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:10:38.419877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:10:38.419877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:10:38.419877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:10:38.419877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:38.419877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:10:38.419877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:38.419877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:10:38.419877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:38.419877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:10:38.419877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:38.419877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:10:38.420626 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:38.420626 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:10:38.420626 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:10:38.420626 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:38.420626 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:10:38.420626 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:38.420626 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:10:38.420626 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:10:38.420626 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:38.420626 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:10:38.420626 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:38.420626 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:10:38.424246 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:10:38.424490 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:10:38.424606 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:10:38.424730 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:10:38.426273 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:10:38.426490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:38.427061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:10:38.427061 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:38.427061 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:10:38.427061 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:10:38.427061 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:38.427061 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:10:38.427061 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:38.427061 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:10:38.428320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:38.428320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:10:38.428320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:10:38.428320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:10:38.428320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:38.428320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:10:38.428320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:38.428320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:10:38.428320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:38.428320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:10:38.428320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:38.428320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:10:38.428320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:38.428320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:10:38.428320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:10:38.428923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:38.428923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:10:38.428923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:38.428923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:10:38.428923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:10:38.428923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:38.428923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:10:38.428923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:38.428923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:10:38.429854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:38.429854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:10:38.429854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:10:38.429854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:10:38.429854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:10:38.429854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:10:38.429854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:10:38.429854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:10:38.429854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:10:38.429854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:38.429854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:10:38.430306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:38.430306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:10:38.430306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:38.430306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:10:38.430306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:38.430306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:10:38.430306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:38.430306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:10:38.430306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:10:38.430306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:38.430306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:10:38.430306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:38.430306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:10:38.430306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:10:38.431052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:38.431052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:10:38.431052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:38.431052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:10:38.431694 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:38.431694 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:10:38.431694 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:10:38.431694 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:10:38.431694 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:10:38.431694 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:10:38.431694 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:10:38.431694 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:10:38.431694 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:10:38.431694 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:10:38.431694 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:10:38.431694 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:10:38.431694 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:10:38.431694 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:10:38.431694 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:10:38.431694 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:10:38.431694 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:10:38.432280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:10:38.432280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:10:38.432280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:10:38.432280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:38.432280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:10:38.432280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:38.432280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:10:38.432280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:38.432280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:10:38.432280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:38.432280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:10:38.432280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:38.432280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:10:38.432280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:10:38.432280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:38.432280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:10:38.432280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:38.432280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:10:38.432280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:10:38.432996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:38.432996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:10:38.432996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:38.432996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:10:38.440341 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:10:38.440654 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:10:38.440845 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:10:38.441014 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:10:38.442215 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:10:38.442523 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:10:38.442684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:38.443360 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:10:38.443360 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:38.443360 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:10:38.443360 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:10:38.443360 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:38.443360 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:10:38.443360 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:38.443360 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:10:38.444689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:38.444689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:10:38.444689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:10:38.444689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:10:38.444689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:10:38.444689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:10:38.444689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:10:38.444689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:38.444689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:10:38.444689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:38.444689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:10:38.444689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:38.445555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:10:38.445555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:38.445555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:10:38.445555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:38.445555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:10:38.445555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:10:38.445555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:38.445555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:10:38.445555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:38.445555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:10:38.445555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:10:38.445555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:38.445555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:10:38.446373 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:38.446373 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:10:38.446517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:38.446517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:10:38.446517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:10:38.446517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:10:38.446517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:10:38.446517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:10:38.446517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:10:38.446517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:10:38.446517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:10:38.446517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:10:38.446517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:10:38.446517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:10:38.446517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:10:38.446517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:10:38.446517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:10:38.446517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:10:38.448063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:10:38.448063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:10:38.448063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:38.448063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:10:38.448063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:38.448063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:10:38.448063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:38.448063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:10:38.448063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:38.448063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:10:38.448063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:38.448063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:10:38.448063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:10:38.448063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:38.448063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:10:38.448063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:38.448063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:10:38.448063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:10:38.448063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:38.449450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:10:38.449450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:38.449450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:10:42.890099 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:10:42.890701 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:10:42.890701 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:10:42.891578 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:10:43.000296 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:10:43.000765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:43.001593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:10:43.001593 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:43.001593 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:10:43.001593 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:10:43.001593 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:43.001593 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:10:43.001593 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:43.001593 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:10:43.002422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.111 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:43.002422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:10:43.002422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:10:43.002422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:10:43.002422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:10:43.002422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:10:43.002422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:10:43.002422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:43.002422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:10:43.002422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:43.002422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:10:43.002422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:43.002800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:10:43.002800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:43.002800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:10:43.002800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:43.002800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:10:43.002800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:10:43.002800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:43.002800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:10:43.002800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:43.002800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:10:43.002800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:10:43.002800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:43.002800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:10:43.003135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:43.003135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:10:43.003491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:43.003491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:10:43.003491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:10:43.003491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:10:43.003491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:10:43.003491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:10:43.003491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:10:43.003491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:10:43.003491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:10:43.003491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:10:43.003491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:10:43.003491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:10:43.003491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:10:43.003491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:10:43.003491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:10:43.003491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:10:43.003870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:10:43.003870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:10:43.003870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:43.003870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:10:43.003870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:43.003870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:10:43.003870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:43.003870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:10:43.003870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:43.003870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:10:43.003870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:43.003870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:10:43.003870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:10:43.003870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:43.003870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:10:43.003870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:43.003870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:10:43.003870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:10:43.003870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:43.004271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:10:43.004271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:43.004271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:10:43.404631 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:10:43.405354 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:10:43.405560 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:10:43.406398 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:10:43.406593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:43.407226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:10:43.407226 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:43.407226 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:10:43.407226 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:10:43.407226 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:43.407226 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:10:43.407226 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:43.407226 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:10:43.408825 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:43.408825 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:10:43.408825 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:10:43.408825 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:10:43.408825 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:10:43.408825 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:10:43.408825 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:10:43.408825 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:43.408825 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:10:43.408825 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:43.408825 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:10:43.408825 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:43.409925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:10:43.409925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:43.409925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:10:43.409925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:43.409925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:10:43.409925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:10:43.409925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:43.409925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:10:43.409925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:43.409925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:10:43.409925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:10:43.409925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:43.409925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:10:43.410531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:43.410531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:10:43.410531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:43.410531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:10:43.410531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:10:43.410531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:10:43.410531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:10:43.410531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:10:43.410531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:10:43.410531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:10:43.410531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:10:43.410531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:10:43.410531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:10:43.410531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:10:43.410531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:10:43.411355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:10:43.411355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:10:43.411355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:10:43.411355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:10:43.411355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:10:43.411355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:43.411355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:10:43.411355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:43.411355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:10:43.411355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:43.411355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:10:43.411355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:43.411355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:10:43.411355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:43.411355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:10:43.411355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:10:43.411355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:43.411355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:10:43.411355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:43.413503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:10:43.413503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:10:43.413503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:43.413503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:10:43.413503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:43.413503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:10:43.413503 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:10:43.415041 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:10:43.415041 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:10:43.415041 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:10:43.415292 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:10:43.415825 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:43.416923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:10:43.416923 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:43.416923 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:10:43.416923 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:10:43.416923 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:43.416923 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:10:43.416923 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:43.416923 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:10:43.417753 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:43.417753 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:10:43.417753 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:10:43.417753 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:10:43.417753 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:43.417753 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:10:43.417753 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:43.417753 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:10:43.417753 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:43.417753 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:10:43.417753 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:43.417753 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:10:43.417753 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:43.417753 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:10:43.417753 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:10:43.418517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:43.418517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:10:43.418517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:43.418517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:10:43.418517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:10:43.418517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:43.418517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:10:43.418517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:43.418517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:10:43.418956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:43.418956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:10:43.418956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:10:43.418956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:10:43.418956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:10:43.418956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:10:43.418956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:10:43.418956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:10:43.418956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:10:43.418956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:10:43.418956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:43.418956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:10:43.419712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:43.419712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:10:43.419712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:43.419712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:10:43.419712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:43.419712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:10:43.419712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:43.419712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:10:43.419712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:10:43.419712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:43.419712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:10:43.419712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:43.419712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:10:43.419712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:10:43.420435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:43.420435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:10:43.420435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:43.420435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:10:43.420435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:43.420435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:10:43.420435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:10:43.420435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:10:43.420435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:10:43.420435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:10:43.420435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:10:43.420435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:10:43.420435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:10:43.420435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:10:43.421076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:10:43.421076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:10:43.421076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:10:43.421076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:10:43.421076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:10:43.421076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:10:43.421076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:10:43.421076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:10:43.421076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:10:43.421076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:10:43.421076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:10:43.421076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:43.421076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:10:43.421076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:43.421076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:10:43.421076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:43.421076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:10:43.421076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:43.421076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:10:43.421922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:43.421922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:10:43.421922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:10:43.421922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:43.421922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:10:43.421922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:43.421922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:10:43.421922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:10:43.421922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:43.421922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:10:43.421922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:43.421922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:10:43.424760 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:10:43.425099 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:10:43.425214 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:10:43.425343 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:10:43.425794 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:10:43.426114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:43.426701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:10:43.426701 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:43.426701 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:10:43.426701 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:10:43.426701 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:43.426701 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:10:43.426701 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:43.426701 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:10:43.427487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:43.427487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:10:43.427487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:10:43.427487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:10:43.427487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:43.427487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:10:43.427487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:43.427487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:10:43.427487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:43.427487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:10:43.427487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:43.427487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:10:43.427487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:43.427487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:10:43.427487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:10:43.427936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:43.427936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:10:43.427936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:43.427936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:10:43.427936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:10:43.427936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:43.427936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:10:43.427936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:43.427936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:10:43.428595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:43.428595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:10:43.428595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:10:43.428595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:10:43.428595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:10:43.428595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:10:43.428595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:10:43.428595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:10:43.428595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:10:43.428595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:43.428595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:10:43.428956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:43.428956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:10:43.428956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:43.428956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:10:43.428956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:43.428956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:10:43.428956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:43.428956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:10:43.428956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:10:43.428956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:43.428956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:10:43.428956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:43.428956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:10:43.428956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:10:43.429337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:43.429337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:10:43.429337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:43.429337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:10:43.430407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:43.430407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:10:43.430407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:10:43.430407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:10:43.430407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:10:43.430407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:10:43.430407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:10:43.430407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:10:43.430407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:10:43.430407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:10:43.430407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:10:43.430407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:10:43.430407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:10:43.430407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:10:43.430407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:10:43.430407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:10:43.430407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:10:43.430878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:10:43.430878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:10:43.430878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:10:43.430878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:43.430878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:10:43.430878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:43.430878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:10:43.430878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:43.430878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:10:43.430878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:43.430878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:10:43.430878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:43.430878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:10:43.430878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:10:43.430878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:43.430878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:10:43.430878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:43.430878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:10:43.430878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:10:43.432997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:43.432997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:10:43.432997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:43.432997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:10:43.441103 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:10:43.441377 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:10:43.441490 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:10:43.441640 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:10:43.442536 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:10:43.443383 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:10:43.443813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:43.444908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:10:43.444908 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:43.444908 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:10:43.444908 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:10:43.444908 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:43.444908 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:10:43.444908 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:43.444908 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:10:43.446388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:43.446388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:10:43.446388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:10:43.446388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:10:43.446388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:10:43.446388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:10:43.446388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:10:43.446388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:43.446388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:10:43.446388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:43.446388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:10:43.446388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:43.447026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:10:43.447026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:43.447026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:10:43.447026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:43.447026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:10:43.447026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:10:43.447026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:43.447026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:10:43.447026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:43.447026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:10:43.447026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:10:43.447026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:43.447026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:10:43.447645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:43.447645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:10:43.448670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:43.448670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:10:43.448670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:10:43.448670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:10:43.448670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:10:43.448670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:10:43.448670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:10:43.448670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:10:43.448670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:10:43.448670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:10:43.448670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:10:43.448670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:10:43.448670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:10:43.448670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:10:43.448670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:10:43.448670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:10:43.449345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:10:43.449345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:10:43.449345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:43.449345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:10:43.449345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:43.449345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:10:43.449345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:43.449345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:10:43.449345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:43.449345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:10:43.449345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:43.449345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:10:43.449345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:10:43.449345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:43.449345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:10:43.449345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:43.449345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:10:43.449345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:10:43.449345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:43.449840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:10:43.449840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:43.449840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:10:47.899632 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:10:47.899632 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:10:47.899632 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:10:47.899632 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:10:47.927610 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:10:47.927992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:47.928815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:10:47.928815 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:47.928815 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:10:47.928815 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:10:47.928815 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:47.928815 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:10:47.928815 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:47.928815 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:10:47.929926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:47.929926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:10:47.929926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:10:47.929926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:10:47.929926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:10:47.929926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:10:47.929926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:10:47.929926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:47.929926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:10:47.929926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:47.929926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:10:47.929926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:47.930234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:10:47.930234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:47.930234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:10:47.930234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:47.930234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:10:47.930234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:10:47.930234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:47.930234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:10:47.930234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:47.930234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:10:47.930234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:10:47.930234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:47.930234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:10:47.930543 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:47.930543 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:10:47.931033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:47.931033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:10:47.931033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:10:47.931033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:10:47.931033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:10:47.931033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:10:47.931033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:10:47.931033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:10:47.931033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:10:47.931033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:10:47.931033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:10:47.931033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:10:47.931033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:10:47.931033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:10:47.931033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:10:47.931033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:10:47.934625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:10:47.934625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:10:47.934625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:47.934625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:10:47.934625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:47.934625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:10:47.934625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:47.934625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:10:47.934625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:47.934625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:10:47.934625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:47.934625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:10:47.934625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:10:47.934625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:47.934625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:10:47.934625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:47.934625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:10:47.934625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:10:47.934625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:47.935486 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:10:47.935486 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:47.935486 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:10:48.405512 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:10:48.406254 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:10:48.406454 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:10:48.407823 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:10:48.407823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:48.408544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:10:48.408544 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:48.408544 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:10:48.408544 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:10:48.408544 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:48.408544 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:10:48.408544 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:48.408544 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:10:48.410358 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:48.410358 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:10:48.410358 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:10:48.410358 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:10:48.410358 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:10:48.410358 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:10:48.410358 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:10:48.410358 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:48.410358 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:10:48.410358 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:48.410358 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:10:48.410358 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:48.410931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:10:48.410931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:48.410931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:10:48.410931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:48.410931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:10:48.410931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:10:48.410931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:48.410931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:10:48.410931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:48.410931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:10:48.410931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:10:48.410931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:48.410931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:10:48.411474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:48.411474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:10:48.412229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:48.412229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:10:48.412229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:10:48.412229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:10:48.412229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:10:48.412229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:10:48.412229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:10:48.412229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:10:48.412229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:10:48.412229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:10:48.412229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:10:48.412229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:10:48.412229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:10:48.412229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:10:48.412229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:10:48.412229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:10:48.413005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:10:48.413005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:10:48.413005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:48.413005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:10:48.413005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:48.413005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:10:48.413005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:48.413005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:10:48.413005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:48.413005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:10:48.413005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:48.413005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:10:48.413005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:10:48.413005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:48.413005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:10:48.413005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:48.413005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:10:48.413005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:10:48.413005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:48.414398 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:10:48.414398 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:48.414398 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:10:48.414398 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:10:48.414398 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:10:48.414398 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:10:48.414398 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:10:48.414750 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:10:48.415011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:48.415546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:10:48.415546 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:48.415546 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:10:48.415546 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:10:48.415546 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:48.415546 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:10:48.415546 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:48.415546 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:10:48.416385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:48.416385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:10:48.416385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:10:48.416385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:10:48.416385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:48.416385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:10:48.416385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:48.416385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:10:48.416385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:48.416385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:10:48.416385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:48.416385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:10:48.416385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:48.416385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:10:48.416385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:10:48.417087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:48.417087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:10:48.417087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:48.417087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:10:48.417087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:10:48.417087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:48.417087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:10:48.417087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:48.417087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:10:48.417452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:48.417452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:10:48.417452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:10:48.417452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:10:48.417452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:10:48.417452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:10:48.417452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:10:48.417452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:10:48.417452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:10:48.417452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:10:48.417452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:48.417452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:10:48.417810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:48.417810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:10:48.417810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:48.417810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:10:48.417810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:48.417810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:10:48.417810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:48.417810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:10:48.417810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:10:48.417810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:48.417810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:10:48.417810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:48.417810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:10:48.417810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:10:48.418164 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:48.418164 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:10:48.418164 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:48.418164 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:10:48.418415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:48.418415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:10:48.418415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:10:48.418415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:10:48.418415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:10:48.418415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:10:48.418415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:10:48.418415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:10:48.418415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:10:48.418415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:10:48.418415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:10:48.418415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:10:48.418415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:10:48.418415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:10:48.418415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:10:48.418415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:10:48.418986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:10:48.418986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:10:48.418986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:10:48.418986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:10:48.418986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:10:48.418986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:48.418986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:10:48.418986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:48.418986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:10:48.418986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:48.418986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:10:48.418986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:48.418986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:10:48.418986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:48.418986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:10:48.418986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:10:48.418986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:48.418986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:10:48.419429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:48.419429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:10:48.419429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:10:48.419429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:48.419429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:10:48.419429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:48.419429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:10:48.425174 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:10:48.425856 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:10:48.425856 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:10:48.425856 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:10:48.426131 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:10:48.426501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:48.427275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:10:48.427275 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:48.427275 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:10:48.427275 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:10:48.427275 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:48.427275 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:10:48.427275 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:48.427275 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:10:48.428083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:48.428083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:10:48.428083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:10:48.428083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:10:48.428083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:48.428083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:10:48.428083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:48.428083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:10:48.428083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:48.428083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:10:48.428083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:48.428083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:10:48.428083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:48.428083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:10:48.428083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:10:48.428463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:48.428463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:10:48.428463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:48.428463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:10:48.428463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:10:48.428463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:48.428463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:10:48.428463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:48.428463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:10:48.429047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:48.429047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:10:48.429047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:10:48.429047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:10:48.429047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:10:48.429047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:10:48.429047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:10:48.429047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:10:48.429047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:10:48.429047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:48.429047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:10:48.429348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:48.429348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:10:48.429348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:48.429348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:10:48.429348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:48.429348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:10:48.429348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:48.429348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:10:48.429348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:10:48.429348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:48.429348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:10:48.429348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:48.429348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:10:48.429348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:10:48.429697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:48.429697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:10:48.429697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:48.429697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:10:48.430378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:48.430378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:10:48.430378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:10:48.430378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:10:48.430378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:10:48.430378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:10:48.430378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:10:48.430378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:10:48.430378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:10:48.430378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:10:48.430378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:10:48.430378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:10:48.430378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:10:48.430378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:10:48.430378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:10:48.430378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:10:48.430378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:10:48.430833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:10:48.430833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:10:48.430833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:10:48.430833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:48.430833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:10:48.430833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:48.430833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:10:48.430833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:48.430833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:10:48.430833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:48.430833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:10:48.430833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:48.430833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:10:48.430833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:10:48.430833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:48.430833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:10:48.430833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:48.430833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:10:48.430833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:10:48.431299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:48.431299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:10:48.431299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:48.431299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:10:48.441567 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:10:48.442089 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:10:48.442310 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:10:48.442608 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:10:48.443092 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:10:48.443383 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:10:48.443519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:48.444305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:10:48.444305 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:48.444305 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:10:48.444305 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:10:48.444305 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:48.444305 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:10:48.444305 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:48.444305 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:10:48.446116 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:48.446116 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:10:48.446116 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:10:48.446116 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:10:48.446116 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:10:48.446116 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:10:48.446116 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:10:48.446116 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:48.446116 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:10:48.446116 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:48.446116 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:10:48.446116 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:48.446536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:10:48.446536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:48.446536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:10:48.446536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:48.446536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:10:48.446536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:10:48.446536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:48.446536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:10:48.446536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:48.446536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:10:48.446536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:10:48.446536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:48.446536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:10:48.446965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:48.446965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:10:48.448194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:48.448194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:10:48.448194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:10:48.448194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:10:48.448194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:10:48.448194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:10:48.448194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:10:48.448194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:10:48.448194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:10:48.448194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:10:48.448194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:10:48.448194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:10:48.448194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:10:48.448194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:10:48.448194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:10:48.448194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:10:48.448655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:10:48.448655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:10:48.448655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:48.448655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:10:48.448655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:48.448655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:10:48.448655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:48.448655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:10:48.448655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:48.448655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:10:48.448655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:48.448655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:10:48.448655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:10:48.448655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:48.448655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:10:48.448655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:48.448655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:10:48.448655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:10:48.448655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:48.449223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:10:48.449223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:48.449223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:10:52.891138 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:10:52.892229 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:10:52.892348 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:10:52.897737 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:10:52.922960 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:10:52.923358 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:52.924282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:10:52.924282 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:52.924282 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:10:52.924282 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:10:52.924282 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:52.924282 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:10:52.924282 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:52.924282 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:10:52.925174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.032 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:52.925174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:10:52.925174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:10:52.925174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:10:52.925174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:10:52.925174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:10:52.925174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:10:52.925174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:52.925174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:10:52.925174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:52.925174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:10:52.925174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:52.925493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:10:52.925493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:52.925493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:10:52.925493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:52.925493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:10:52.925493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:10:52.925493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:52.925493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:10:52.925493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:52.925493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:10:52.925493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:10:52.925493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:52.925493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:10:52.925833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:52.925833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:10:52.926223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:52.926223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:10:52.926223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:10:52.926223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:10:52.926223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:10:52.926223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:10:52.926223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:10:52.926223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:10:52.926223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:10:52.926223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:10:52.926223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:10:52.926223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:10:52.926223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:10:52.926223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:10:52.926223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:10:52.926223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:10:52.926557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:10:52.926557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:10:52.926557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:52.926557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:10:52.926557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:52.926557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:10:52.926557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:52.926557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:10:52.926557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:52.926557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:10:52.926557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:52.926557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:10:52.926557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:10:52.926557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:52.926557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:10:52.926557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:52.926557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:10:52.926557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:10:52.926557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:52.926936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:10:52.926936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:52.926936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:10:53.406316 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:10:53.407079 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:10:53.407332 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:10:53.408292 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:10:53.408614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:53.409600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:10:53.409600 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:53.409600 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:10:53.409600 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:10:53.409600 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:53.409600 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:10:53.409600 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:53.409600 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:10:53.411225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:53.411225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:10:53.411225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:10:53.411225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:10:53.411225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:10:53.411225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:10:53.411225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:10:53.411225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:53.411225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:10:53.411225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:53.411225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:10:53.411225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:53.411603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:10:53.411603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:53.411603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:10:53.411603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:53.411603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:10:53.411603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:10:53.411603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:53.411603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:10:53.411603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:53.411603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:10:53.411603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:10:53.411603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:53.411603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:10:53.412061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:53.412061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:10:53.413541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:53.413541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:10:53.413541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:10:53.413541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:10:53.413541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:10:53.413541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:10:53.413541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:10:53.413541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:10:53.413541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:10:53.413541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:10:53.413541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:10:53.413541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:10:53.413541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:10:53.413541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:10:53.413541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:10:53.413541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:10:53.414413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:10:53.414413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:10:53.414413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:53.414413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:10:53.414413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:53.414413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:10:53.414413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:53.414413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:10:53.414413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:53.414413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:10:53.414413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:53.414413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:10:53.414413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:10:53.414413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:53.414413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:10:53.414413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:53.414413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:10:53.414413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:10:53.414413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:53.415345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:10:53.415345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:53.415345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:10:53.415345 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:10:53.415345 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:10:53.415345 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:10:53.415345 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:10:53.415732 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:10:53.415972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:53.417017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:10:53.417017 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:53.417017 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:10:53.417017 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:10:53.417017 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:53.417017 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:10:53.417017 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:53.417017 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:10:53.418256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:53.418256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:10:53.418256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:10:53.418256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:10:53.418256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:53.418256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:10:53.418256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:53.418256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:10:53.418256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:53.418256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:10:53.418256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:53.418256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:10:53.418256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:53.418256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:10:53.418256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:10:53.418875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:53.418875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:10:53.418875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:53.418875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:10:53.418875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:10:53.418875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:53.418875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:10:53.418875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:53.418875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:10:53.419691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:53.419691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:10:53.419691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:10:53.419691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:10:53.419691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:10:53.419691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:10:53.419691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:10:53.419691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:10:53.419691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:10:53.419691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:10:53.419691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:53.419691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:10:53.420198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:53.420198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:10:53.420198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:53.420198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:10:53.420198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:53.420198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:10:53.420198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:53.420198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:10:53.420198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:10:53.420198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:53.420198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:10:53.420198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:53.420198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:10:53.420198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:10:53.420666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:53.420666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:10:53.420666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:53.420666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:10:53.421464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:53.421464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:10:53.421464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:10:53.421464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:10:53.421464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:10:53.421464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:10:53.421464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:10:53.421464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:10:53.421464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:10:53.421464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:10:53.421464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:10:53.421464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:10:53.421464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:10:53.421464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:10:53.421464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:10:53.421464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:10:53.421966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:10:53.421966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:10:53.421966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:10:53.421966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:10:53.421966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:10:53.421966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:53.421966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:10:53.421966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:53.421966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:10:53.421966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:53.421966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:10:53.421966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:53.421966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:10:53.421966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:53.421966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:10:53.421966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:10:53.421966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:53.421966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:10:53.422457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:53.422457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:10:53.422457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:10:53.422457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:53.422457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:10:53.422457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:53.422457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:10:53.425524 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:10:53.425753 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:10:53.425885 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:10:53.426028 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:10:53.426338 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:10:53.426549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:53.427015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:10:53.427015 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:53.427015 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:10:53.427015 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:10:53.427015 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:53.427015 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:10:53.427015 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:53.427015 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:10:53.427814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:53.427814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:10:53.427814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:10:53.427814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:10:53.427814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:53.427814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:10:53.427814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:53.427814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:10:53.427814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:53.427814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:10:53.427814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:53.427814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:10:53.427814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:53.427814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:10:53.427814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:10:53.428228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:53.428228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:10:53.428228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:53.428228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:10:53.428228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:10:53.428228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:53.428228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:10:53.428228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:53.428228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:10:53.428753 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:53.428753 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:10:53.428753 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:10:53.428753 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:10:53.428753 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:10:53.428753 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:10:53.428753 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:10:53.428753 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:10:53.428753 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:10:53.428753 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:53.428753 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:10:53.429084 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:53.429084 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:10:53.429084 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:53.429084 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:10:53.429084 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:53.429084 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:10:53.429084 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:53.429084 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:10:53.429084 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:10:53.429084 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:53.429084 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:10:53.429084 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:53.429084 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:10:53.429084 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:10:53.429457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:53.429457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:10:53.429457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:53.429457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:10:53.429849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:53.429849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:10:53.429849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:10:53.429849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:10:53.429849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:10:53.429849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:10:53.429849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:10:53.429849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:10:53.429849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:10:53.429849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:10:53.429849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:10:53.429849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:10:53.429849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:10:53.429849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:10:53.429849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:10:53.429849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:10:53.429849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:10:53.430342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:10:53.430342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:10:53.430342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:10:53.430342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:53.430342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:10:53.430342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:53.430342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:10:53.430342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:53.430342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:10:53.430342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:53.430342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:10:53.430342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:53.430342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:10:53.430342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:10:53.430342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:53.430342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:10:53.430342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:53.430342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:10:53.430342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:10:53.430828 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:53.430828 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:10:53.430828 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:53.430828 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:10:53.442089 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:10:53.442510 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:10:53.442768 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:10:53.443051 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:10:53.443471 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:10:53.443940 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:10:53.444199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:53.445138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:10:53.445138 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:53.445138 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:10:53.445138 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:10:53.445138 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:53.445138 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:10:53.445138 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:53.445138 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:10:53.446673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:53.446673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:10:53.446673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:10:53.446673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:10:53.446673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:10:53.446673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:10:53.446673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:10:53.446673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:53.446673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:10:53.446673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:53.446673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:10:53.446673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:53.447306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:10:53.447306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:53.447306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:10:53.447306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:53.447306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:10:53.447306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:10:53.447306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:53.447306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:10:53.447306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:53.447306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:10:53.447306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:10:53.447306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:53.447306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:10:53.447916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:53.447916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:10:53.448501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:53.448501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:10:53.448501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:10:53.448501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:10:53.448501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:10:53.448501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:10:53.448501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:10:53.448501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:10:53.448501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:10:53.448501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:10:53.448501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:10:53.448501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:10:53.448501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:10:53.448501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:10:53.448501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:10:53.448501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:10:53.449653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:10:53.449653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:10:53.449653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:53.449653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:10:53.449653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:53.449653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:10:53.449653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:53.449653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:10:53.449653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:53.449653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:10:53.449653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:53.449653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:10:53.449653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:10:53.449653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:53.449653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:10:53.449653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:53.449653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:10:53.449653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:10:53.449653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:53.450798 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:10:53.450798 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:53.450798 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:10:57.893149 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:10:57.894488 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:10:57.894696 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:10:57.896442 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:10:57.934786 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:10:57.935221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:57.936382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:10:57.936382 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:57.936382 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:10:57.936382 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:10:57.936382 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:57.936382 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:10:57.936382 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:57.936382 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:10:57.937978 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:57.937978 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:10:57.937978 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:10:57.937978 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:10:57.937978 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:10:57.937978 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:10:57.937978 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:10:57.937978 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:57.937978 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:10:57.937978 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:57.937978 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:10:57.937978 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:57.938883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:10:57.938883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:57.938883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:10:57.938883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:57.938883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:10:57.938883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:10:57.938883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:57.938883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:10:57.938883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:57.938883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:10:57.938883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:10:57.938883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:57.938883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:10:57.939594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:57.939594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:10:57.939594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:57.939594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:10:57.939594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:10:57.939594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:10:57.939594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:10:57.939594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:10:57.939594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:10:57.939594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:10:57.939594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:10:57.939594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:10:57.939594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:10:57.939594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:10:57.939594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:10:57.940615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:10:57.940615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:10:57.940615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:10:57.940615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:10:57.940615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:10:57.940615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:57.940615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:10:57.940615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:57.940615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:10:57.940615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:57.940615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:10:57.940615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:57.940615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:10:57.940615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:57.940615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:10:57.940615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:10:57.940615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:57.940615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:10:57.940615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:57.941457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:10:57.941457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:10:57.941457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:57.941457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:10:57.941457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:57.941457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:10:58.407240 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:10:58.408942 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:10:58.408942 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:10:58.409563 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:10:58.409921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:58.410953 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:10:58.410953 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:58.410953 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:10:58.410953 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:10:58.410953 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:58.410953 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:10:58.410953 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:58.410953 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:10:58.412778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:58.412778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:10:58.412778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:10:58.412778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:10:58.412778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:10:58.412778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:10:58.412778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:10:58.412778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:58.412778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:10:58.412778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:58.412778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:10:58.412778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:58.413580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:10:58.413580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:58.413580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:10:58.413580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:58.413580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:10:58.413580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:10:58.413580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:58.413580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:10:58.413580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:58.413580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:10:58.413580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:10:58.413580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:58.413580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:10:58.414634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:58.414634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:10:58.415208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:58.415208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:10:58.415208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:10:58.415208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:10:58.415208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:10:58.415208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:10:58.415208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:10:58.415208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:10:58.415208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:10:58.415208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:10:58.415208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:10:58.415208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:10:58.415208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:10:58.415208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:10:58.415208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:10:58.415208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:10:58.416073 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:10:58.416073 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:10:58.416073 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:58.416073 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:10:58.416073 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:58.416073 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:10:58.416073 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:58.416073 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:10:58.416073 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:58.416073 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:10:58.416073 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:58.416073 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:10:58.416073 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:10:58.416073 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:58.416073 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:10:58.416073 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:58.416073 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:10:58.416073 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:10:58.416073 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:58.416845 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:10:58.416845 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:58.416845 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:10:58.416845 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:10:58.416845 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:10:58.416845 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:10:58.416845 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:10:58.416845 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:10:58.417190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:58.417945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:10:58.417945 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:58.417945 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:10:58.417945 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:10:58.417945 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:58.417945 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:10:58.417945 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:58.417945 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:10:58.418858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:58.418858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:10:58.418858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:10:58.418858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:10:58.418858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:58.418858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:10:58.418858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:58.418858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:10:58.418858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:58.418858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:10:58.418858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:58.418858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:10:58.418858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:58.418858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:10:58.418858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:10:58.419466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:58.419466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:10:58.419466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:58.419466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:10:58.419466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:10:58.419466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:58.419466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:10:58.419466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:58.419466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:10:58.420019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:58.420019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:10:58.420019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:10:58.420019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:10:58.420019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:10:58.420019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:10:58.420019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:10:58.420019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:10:58.420019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:10:58.420019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:10:58.420019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:58.420019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:10:58.420514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:58.420514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:10:58.420514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:58.420514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:10:58.420514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:58.420514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:10:58.420514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:58.420514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:10:58.420514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:10:58.420514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:58.420514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:10:58.420514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:58.420514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:10:58.420514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:10:58.421079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:58.421079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:10:58.421079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:58.421079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:10:58.421457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:58.421457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:10:58.421457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:10:58.421457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:10:58.421457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:10:58.421457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:10:58.421457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:10:58.421457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:10:58.421457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:10:58.421457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:10:58.421457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:10:58.421457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:10:58.421457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:10:58.421457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:10:58.421457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:10:58.421457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:10:58.421942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:10:58.421942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:10:58.421942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:10:58.421942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:10:58.421942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:10:58.421942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:58.421942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:10:58.421942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:58.421942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:10:58.421942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:58.421942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:10:58.421942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:58.421942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:10:58.421942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:58.421942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:10:58.421942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:10:58.421942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:58.421942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:10:58.422477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:58.422477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:10:58.422477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:10:58.422477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:58.422477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:10:58.422477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:58.422477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:10:58.425989 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:10:58.426355 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:10:58.426507 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:10:58.426683 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:10:58.427233 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:10:58.427588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:58.428358 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:10:58.428358 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:58.428358 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:10:58.428358 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:10:58.428358 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:58.428358 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:10:58.428358 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:58.428358 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:10:58.429265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:58.429265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:10:58.429265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:10:58.429265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:10:58.429265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:58.429265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:10:58.429265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:58.429265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:10:58.429265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:58.429265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:10:58.429265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:58.429265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:10:58.429265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:58.429265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:10:58.429265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:10:58.429868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:58.429868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:10:58.429868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:58.429868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:10:58.429868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:10:58.429868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:58.429868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:10:58.429868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:58.429868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:10:58.430499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:58.430499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:10:58.430499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:10:58.430499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:10:58.430499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:10:58.430499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:10:58.430499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:10:58.430499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:10:58.430499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:10:58.430499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:58.430499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:10:58.431152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:58.431152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:10:58.431152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:58.431152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:10:58.431152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:58.431152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:10:58.431152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:58.431152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:10:58.431152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:10:58.431152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:58.431152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:10:58.431152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:58.431152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:10:58.431152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:10:58.431681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:58.431681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:10:58.431681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:58.431681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:10:58.432025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:58.432025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:10:58.432025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:10:58.432025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:10:58.432025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:10:58.432025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:10:58.432025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:10:58.432025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:10:58.432025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:10:58.432025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:10:58.432025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:10:58.432025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:10:58.432025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:10:58.432025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:10:58.432025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:10:58.432025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:10:58.432025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:10:58.432517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:10:58.432517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:10:58.432517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:10:58.432517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:58.432517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:10:58.432517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:58.432517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:10:58.432517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:58.432517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:10:58.432517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:58.432517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:10:58.432517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:58.432517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:10:58.432517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:10:58.432517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:58.432517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:10:58.432517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:58.432517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:10:58.432517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:10:58.433024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:58.433024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:10:58.433024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:58.433024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:10:58.442582 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:10:58.442914 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:10:58.443063 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:10:58.443271 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:10:58.443693 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:10:58.444084 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:10:58.444258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:58.445085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:10:58.445085 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:58.445085 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:10:58.445085 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:10:58.445085 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:58.445085 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:10:58.445085 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:58.445085 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:10:58.446529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:58.446529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:10:58.446529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:10:58.446529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:10:58.446529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:10:58.446529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:10:58.446529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:10:58.446529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:58.446529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:10:58.446529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:58.446529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:10:58.446529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:58.447229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:10:58.447229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:58.447229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:10:58.447229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:58.447229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:10:58.447229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:10:58.447229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:58.447229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:10:58.447229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:58.447229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:10:58.447229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:10:58.447229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:58.447229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:10:58.447778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:58.447778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:10:58.448256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:58.448256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:10:58.448256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:10:58.448256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:10:58.448256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:10:58.448256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:10:58.448256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:10:58.448256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:10:58.448256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:10:58.448256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:10:58.448256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:10:58.448256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:10:58.448256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:10:58.448256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:10:58.448256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:10:58.448256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:10:58.449007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:10:58.449007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:10:58.449007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:58.449007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:10:58.449007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:10:58.449007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:10:58.449007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:10:58.449007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:10:58.449007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:10:58.449007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:10:58.449007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:10:58.449007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:10:58.449007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:10:58.449007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:10:58.449007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:10:58.449007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:10:58.449007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:10:58.449007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:10:58.449007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:10:58.449783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:10:58.449783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:10:58.449783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:11:02.894587 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:11:02.895496 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:11:02.895496 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:11:02.896837 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:11:02.925925 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:11:02.926317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:02.927244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:11:02.927244 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:02.927244 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:11:02.927244 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:11:02.927244 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:02.927244 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:11:02.927244 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:02.927244 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:11:02.928165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.032 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:02.928165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:11:02.928165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:11:02.928165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:11:02.928165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:11:02.928165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:11:02.928165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:11:02.928165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:02.928165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:11:02.928165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:02.928165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:11:02.928165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:02.929323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:11:02.929323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:02.929323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:11:02.929323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:02.929323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:11:02.929323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:11:02.929323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:02.929323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:11:02.929323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:02.929323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:11:02.929323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:11:02.929323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:02.929323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:11:02.930658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:02.930658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:11:02.930658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:02.930658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:11:02.930658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:11:02.930658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:11:02.930658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:11:02.930658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:11:02.930658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:11:02.930658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:11:02.930658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:11:02.930658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:11:02.930658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:11:02.930658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:11:02.930658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:11:02.932384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:11:02.932384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:11:02.932384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:11:02.932384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:11:02.932384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:11:02.932384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:02.932384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:11:02.932384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:02.932384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:11:02.932384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:02.932384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:11:02.932384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:02.932384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:11:02.932384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:02.932384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:11:02.932384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:11:02.932384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:02.932384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:11:02.932384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:02.934094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:11:02.934094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:11:02.934094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:02.934094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:11:02.934094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:02.934094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:11:03.407913 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:11:03.408320 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:11:03.408563 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:11:03.409305 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:11:03.409525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:03.410232 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:11:03.410232 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:03.410232 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:11:03.410232 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:11:03.410232 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:03.410232 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:11:03.410232 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:03.410232 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:11:03.411973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:03.411973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:11:03.411973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:11:03.411973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:11:03.411973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:11:03.411973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:11:03.411973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:11:03.411973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:03.411973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:11:03.411973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:03.411973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:11:03.411973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:03.412612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:11:03.412612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:03.412612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:11:03.412612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:03.412612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:11:03.412612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:11:03.412612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:03.412612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:11:03.412612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:03.412612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:11:03.412612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:11:03.412612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:03.412612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:11:03.413279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:03.413279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:11:03.414199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:03.414199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:11:03.414199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:11:03.414199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:11:03.414199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:11:03.414199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:11:03.414199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:11:03.414199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:11:03.414199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:11:03.414199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:11:03.414199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:11:03.414199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:11:03.414199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:11:03.414199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:11:03.414199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:11:03.414199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:11:03.415122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:11:03.415122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:11:03.415122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:03.415122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:11:03.415122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:03.415122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:11:03.415122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:03.415122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:11:03.415122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:03.415122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:11:03.415122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:03.415122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:11:03.415122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:11:03.415122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:03.415122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:11:03.415122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:03.415122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:11:03.415122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:11:03.415122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:03.417008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:11:03.417008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:03.417008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:11:03.417008 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:11:03.417008 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:11:03.417008 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:11:03.417795 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:11:03.417795 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:11:03.418148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:03.418880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:11:03.418880 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:03.418880 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:11:03.418880 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:11:03.418880 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:03.418880 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:11:03.418880 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:03.418880 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:11:03.419847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:03.419847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:11:03.419847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:11:03.419847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:11:03.419847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:03.419847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:11:03.419847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:03.419847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:11:03.419847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:03.419847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:11:03.419847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:03.419847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:11:03.419847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:03.419847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:11:03.419847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:11:03.421399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:03.421399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:11:03.421399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:03.421399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:11:03.421399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:11:03.421399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:03.421399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:11:03.421399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:03.421399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:11:03.421399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:03.421399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:11:03.421399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:11:03.421399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:11:03.421399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:11:03.423161 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:11:03.423161 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:11:03.423161 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:11:03.423161 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:11:03.423161 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:11:03.423161 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:03.423161 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:11:03.423161 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:03.423161 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:11:03.423161 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:03.423161 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:11:03.423161 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:03.423161 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:11:03.423858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:03.423858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:11:03.423858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:11:03.423858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:03.423858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:11:03.423858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:03.423858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:11:03.423858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:11:03.423858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:03.423858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:11:03.423858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:03.423858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:11:03.424447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:03.424447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:11:03.424447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:11:03.424447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:11:03.424447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:11:03.424447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:11:03.424447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:11:03.424447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:11:03.424447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:11:03.424447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:11:03.424447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:11:03.424447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:11:03.424447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:11:03.424447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:11:03.424447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:11:03.424447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:11:03.425160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:11:03.425160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:11:03.425160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:11:03.425160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:11:03.425160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:11:03.425160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:03.425160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:11:03.425160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:03.425160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:11:03.425160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:03.425160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:11:03.425160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:03.425160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:11:03.425160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:03.425160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:11:03.425160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:11:03.425160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:03.425160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:11:03.425947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:03.425947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:11:03.425947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:11:03.425947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:03.425947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:11:03.425947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:03.425947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:11:03.426343 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:11:03.426551 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:11:03.426665 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:11:03.426834 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:11:03.427083 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:11:03.427280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:03.427688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:11:03.427688 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:03.427688 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:11:03.427688 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:11:03.427688 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:03.427688 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:11:03.427688 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:03.427688 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:11:03.428516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:03.428516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:11:03.428516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:11:03.428516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:11:03.428516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:03.428516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:11:03.428516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:03.428516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:11:03.428516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:03.428516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:11:03.428516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:03.428516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:11:03.428516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:03.428516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:11:03.428516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:11:03.429234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:03.429234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:11:03.429234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:03.429234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:11:03.429234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:11:03.429234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:03.429234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:11:03.429234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:03.429234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:11:03.429633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:03.429633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:11:03.429633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:11:03.429633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:11:03.429633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:11:03.429633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:11:03.429633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:11:03.429633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:11:03.429633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:11:03.429633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:03.429633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:11:03.430255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:03.430255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:11:03.430255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:03.430255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:11:03.430255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:03.430255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:11:03.430255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:03.430255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:11:03.430255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:11:03.430255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:03.430255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:11:03.430255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:03.430255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:11:03.430255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:11:03.430995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:03.430995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:11:03.430995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:03.430995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:11:03.430995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:03.430995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:11:03.430995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:11:03.430995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:11:03.430995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:11:03.430995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:11:03.430995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:11:03.430995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:11:03.430995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:11:03.430995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:11:03.431598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:11:03.431598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:11:03.431598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:11:03.431598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:11:03.431598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:11:03.431598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:11:03.431598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:11:03.431598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:11:03.431598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:11:03.431598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:11:03.431598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:03.431598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:11:03.431598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:03.431598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:11:03.431598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:03.431598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:11:03.431598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:03.431598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:11:03.431598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:03.432459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:11:03.432459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:11:03.432459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:03.432459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:11:03.432459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:03.432459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:11:03.432459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:11:03.432459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:03.432459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:11:03.432459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:03.432459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:11:03.442912 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:11:03.443137 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:11:03.443346 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:11:03.443576 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:11:03.444057 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:11:03.444491 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:11:03.444717 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:03.445594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:11:03.445594 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:03.445594 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:11:03.445594 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:11:03.445594 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:03.445594 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:11:03.445594 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:03.445594 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:11:03.447063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:03.447063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:11:03.447063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:11:03.447063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:11:03.447063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:11:03.447063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:11:03.447063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:11:03.447063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:03.447063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:11:03.447063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:03.447063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:11:03.447063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:03.447547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:11:03.447547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:03.447547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:11:03.447547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:03.447547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:11:03.447547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:11:03.447547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:03.447547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:11:03.447547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:03.447547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:11:03.447547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:11:03.447547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:03.447547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:11:03.448078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:03.448078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:11:03.448643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:03.448643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:11:03.448643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:11:03.448643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:11:03.448643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:11:03.448643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:11:03.448643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:11:03.448643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:11:03.448643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:11:03.448643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:11:03.448643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:11:03.448643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:11:03.448643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:11:03.448643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:11:03.448643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:11:03.448643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:11:03.449339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:11:03.449339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:11:03.449339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:03.449339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:11:03.449339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:03.449339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:11:03.449339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:03.449339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:11:03.449339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:03.449339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:11:03.449339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:03.449339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:11:03.449339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:11:03.449339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:03.449339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:11:03.449339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:03.449339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:11:03.449339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:11:03.449339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:03.450079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:11:03.450079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:03.450079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:11:07.895366 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:11:07.896257 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:11:07.896462 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:11:07.897941 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:11:07.938478 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:11:07.939044 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:07.940056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:11:07.940056 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:07.940056 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:11:07.940056 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:11:07.940056 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:07.940056 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:11:07.940056 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:07.940056 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:11:07.941167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.044 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:07.941167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:11:07.941167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:11:07.941167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:11:07.941167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:11:07.941167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:11:07.941167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:11:07.941167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:07.941167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:11:07.941167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:07.941167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:11:07.941167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:07.942401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:11:07.942401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:07.942401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:11:07.942401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:07.942401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:11:07.942401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:11:07.942401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:07.942401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:11:07.942401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:07.942401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:11:07.942401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:11:07.942401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:07.942401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:11:07.943340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:07.943340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:11:07.943340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:07.943340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:11:07.943340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:11:07.943340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:11:07.943340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:11:07.943340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:11:07.943340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:11:07.943340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:11:07.943340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:11:07.943340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:11:07.943340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:11:07.943340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:11:07.943340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:11:07.946262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:11:07.946262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:11:07.946262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:11:07.946262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:11:07.946262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:11:07.946262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:07.946262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:11:07.946262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:07.946262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:11:07.946262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:07.946262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:11:07.946262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:07.946262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:11:07.946262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:07.946262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:11:07.946262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:11:07.946262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:07.946262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:11:07.946262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:07.947354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:11:07.947354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:11:07.947354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:07.947354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:11:07.947354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:07.947354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:11:08.408926 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:11:08.409545 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:11:08.409773 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:11:08.410656 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:11:08.411101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:08.412047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:11:08.412047 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:08.412047 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:11:08.412047 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:11:08.412047 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:08.412047 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:11:08.412047 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:08.412047 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:11:08.413680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:08.413680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:11:08.413680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:11:08.413680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:11:08.413680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:11:08.413680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:11:08.413680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:11:08.413680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:08.413680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:11:08.413680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:08.413680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:11:08.413680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:08.414301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:11:08.414301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:08.414301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:11:08.414301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:08.414301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:11:08.414301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:11:08.414301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:08.414301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:11:08.414301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:08.414301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:11:08.414301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:11:08.414301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:08.414301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:11:08.414945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:08.414945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:11:08.415370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:08.415370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:11:08.415370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:11:08.415370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:11:08.415370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:11:08.415370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:11:08.415370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:11:08.415370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:11:08.415370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:11:08.415370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:11:08.415370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:11:08.415370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:11:08.415370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:11:08.415370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:11:08.415370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:11:08.415370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:11:08.416088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:11:08.416088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:11:08.416088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:08.416088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:11:08.416088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:08.416088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:11:08.416088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:08.416088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:11:08.416088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:08.416088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:11:08.416088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:08.416088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:11:08.416088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:11:08.416088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:08.416088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:11:08.416088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:08.416088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:11:08.416088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:11:08.416088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:08.416983 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:11:08.416983 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:08.416983 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:11:08.416983 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:11:08.417221 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:11:08.417221 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:11:08.417416 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:11:08.417830 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:11:08.418087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:08.418732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:11:08.418732 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:08.418732 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:11:08.418732 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:11:08.418732 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:08.418732 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:11:08.418732 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:08.418732 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:11:08.420071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:08.420071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:11:08.420071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:11:08.420071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:11:08.420071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:08.420071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:11:08.420071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:08.420071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:11:08.420071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:08.420071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:11:08.420071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:08.420071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:11:08.420071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:08.420071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:11:08.420071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:11:08.420496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:08.420496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:11:08.420496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:08.420496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:11:08.420496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:11:08.420496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:08.420496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:11:08.420496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:08.420496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:11:08.421591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:08.421591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:11:08.421591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:11:08.421591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:11:08.421591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:11:08.421591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:11:08.421591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:11:08.421591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:11:08.421591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:11:08.421591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:11:08.421591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:08.421591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:11:08.421948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:08.421948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:11:08.421948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:08.421948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:11:08.421948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:08.421948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:11:08.421948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:08.421948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:11:08.421948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:11:08.421948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:08.421948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:11:08.421948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:08.421948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:11:08.421948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:11:08.422290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:08.422290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:11:08.422290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:08.422290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:11:08.422599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:08.422599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:11:08.422599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:11:08.422599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:11:08.422599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:11:08.422599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:11:08.422599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:11:08.422599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:11:08.422599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:11:08.422599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:11:08.422599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:11:08.422599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:11:08.422599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:11:08.422599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:11:08.422599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:11:08.422599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:11:08.423157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:11:08.423157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:11:08.423157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:11:08.423157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:11:08.423157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:11:08.423157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:08.423157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:11:08.423157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:08.423157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:11:08.423157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:08.423157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:11:08.423157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:08.423157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:11:08.423157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:08.423157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:11:08.423157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:11:08.423157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:08.423157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:11:08.423591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:08.423591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:11:08.423591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:11:08.423591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:08.423591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:11:08.423591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:08.423591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:11:08.426705 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:11:08.427016 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:11:08.427130 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:11:08.427256 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:11:08.427620 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:11:08.427992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:08.428555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:11:08.428555 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:08.428555 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:11:08.428555 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:11:08.428555 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:08.428555 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:11:08.428555 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:08.428555 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:11:08.429353 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:08.429353 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:11:08.429353 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:11:08.429353 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:11:08.429353 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:08.429353 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:11:08.429353 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:08.429353 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:11:08.429353 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:08.429353 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:11:08.429353 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:08.429353 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:11:08.429353 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:08.429353 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:11:08.429353 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:11:08.429731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:08.429731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:11:08.429731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:08.429731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:11:08.429731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:11:08.429731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:08.429731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:11:08.429731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:08.429731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:11:08.430212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:08.430212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:11:08.430212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:11:08.430212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:11:08.430212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:11:08.430212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:11:08.430212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:11:08.430212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:11:08.430212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:11:08.430212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:08.430212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:11:08.430523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:08.430523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:11:08.430523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:08.430523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:11:08.430523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:08.430523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:11:08.430523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:08.430523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:11:08.430523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:11:08.430523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:08.430523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:11:08.430523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:08.430523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:11:08.430523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:11:08.430888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:08.430888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:11:08.430888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:08.430888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:11:08.431253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:08.431253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:11:08.431253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:11:08.431253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:11:08.431253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:11:08.431253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:11:08.431253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:11:08.431253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:11:08.431253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:11:08.431253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:11:08.431253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:11:08.431253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:11:08.431253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:11:08.431253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:11:08.431253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:11:08.431253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:11:08.431253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:11:08.431851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:11:08.431851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:11:08.431851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:11:08.431851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:08.431851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:11:08.431851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:08.431851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:11:08.431851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:08.431851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:11:08.431851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:08.431851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:11:08.431851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:08.431851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:11:08.431851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:11:08.431851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:08.431851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:11:08.431851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:08.431851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:11:08.431851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:11:08.432363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:08.432363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:11:08.432363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:08.432363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:11:08.443207 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:11:08.443570 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:11:08.443762 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:11:08.444232 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:11:08.444338 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:11:08.444570 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:11:08.444678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:08.445353 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:11:08.445353 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:08.445353 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:11:08.445353 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:11:08.445353 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:08.445353 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:11:08.445353 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:08.445353 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:11:08.446677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:08.446677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:11:08.446677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:11:08.446677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:11:08.446677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:11:08.446677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:11:08.446677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:11:08.446677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:08.446677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:11:08.446677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:08.446677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:11:08.446677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:08.447331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:11:08.447331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:08.447331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:11:08.447331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:08.447331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:11:08.447331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:11:08.447331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:08.447331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:11:08.447331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:08.447331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:11:08.447331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:11:08.447331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:08.447331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:11:08.448151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:08.448151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:11:08.448286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:08.448286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:11:08.448286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:11:08.448286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:11:08.448286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:11:08.448286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:11:08.448286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:11:08.448286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:11:08.448286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:11:08.448286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:11:08.448286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:11:08.448286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:11:08.448286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:11:08.448286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:11:08.448286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:11:08.448286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:11:08.449067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:11:08.449067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:11:08.449067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:08.449067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:11:08.449067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:08.449067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:11:08.449067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:08.449067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:11:08.449067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:08.449067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:11:08.449067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:08.449067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:11:08.449067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:11:08.449067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:08.449067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:11:08.449067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:08.449067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:11:08.449067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:11:08.449067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:08.450887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:11:08.450887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:08.450887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:11:12.896323 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:11:12.897146 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:11:12.897228 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:11:12.898139 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:11:12.925462 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:11:12.925855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:12.926621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:11:12.926621 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:12.926621 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:11:12.926621 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:11:12.926621 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:12.926621 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:11:12.926621 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:12.926621 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:11:12.927291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.029 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:12.927291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:11:12.927291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:11:12.927291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:11:12.927291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:11:12.927291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:11:12.927291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:11:12.927291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:12.927291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:11:12.927291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:12.927291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:11:12.927291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:12.927813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:11:12.927813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:12.927813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:11:12.927813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:12.927813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:11:12.927813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:11:12.927813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:12.927813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:11:12.927813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:12.927813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:11:12.927813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:11:12.927813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:12.927813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:11:12.928571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:12.928571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:11:12.928571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:12.928571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:11:12.928571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:11:12.928571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:11:12.928571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:11:12.928571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:11:12.928571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:11:12.928571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:11:12.928571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:11:12.928571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:11:12.928571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:11:12.928571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:11:12.928571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:11:12.929119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:11:12.929119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:11:12.929119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:11:12.929119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:11:12.929119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:11:12.929119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:12.929119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:11:12.929119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:12.929119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:11:12.929119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:12.929119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:11:12.929119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:12.929119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:11:12.929119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:12.929119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:11:12.929119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:11:12.929119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:12.929119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:11:12.929119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:12.929725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:11:12.929725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:11:12.929725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:12.929725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:11:12.929725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:12.929725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:11:13.409502 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:11:13.410195 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:11:13.410407 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:11:13.411178 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:11:13.411422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:13.412137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:11:13.412137 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:13.412137 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:11:13.412137 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:11:13.412137 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:13.412137 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:11:13.412137 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:13.412137 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:11:13.413739 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:13.413739 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:11:13.413739 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:11:13.413739 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:11:13.413739 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:11:13.413739 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:11:13.413739 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:11:13.413739 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:13.413739 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:11:13.413739 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:13.413739 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:11:13.413739 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:13.414413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:11:13.414413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:13.414413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:11:13.414413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:13.414413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:11:13.414413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:11:13.414413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:13.414413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:11:13.414413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:13.414413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:11:13.414413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:11:13.414413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:13.414413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:11:13.415058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:13.415058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:11:13.415613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:13.415613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:11:13.415613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:11:13.415613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:11:13.415613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:11:13.415613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:11:13.415613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:11:13.415613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:11:13.415613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:11:13.415613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:11:13.415613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:11:13.415613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:11:13.415613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:11:13.415613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:11:13.415613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:11:13.415613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:11:13.416580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:11:13.416580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:11:13.416580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:13.416580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:11:13.416580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:13.416580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:11:13.416580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:13.416580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:11:13.416580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:13.416580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:11:13.416580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:13.416580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:11:13.416580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:11:13.416580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:13.416580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:11:13.416580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:13.416580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:11:13.416580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:11:13.416580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:13.417564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:11:13.417564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:13.417564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:11:13.417564 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:11:13.417564 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:11:13.417925 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:11:13.417925 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:11:13.418271 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:11:13.418469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:13.419074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:11:13.419074 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:13.419074 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:11:13.419074 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:11:13.419074 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:13.419074 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:11:13.419074 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:13.419074 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:11:13.420334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:13.420334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:11:13.420334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:11:13.420334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:11:13.420334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:13.420334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:11:13.420334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:13.420334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:11:13.420334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:13.420334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:11:13.420334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:13.420334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:11:13.420334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:13.420334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:11:13.420334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:11:13.421051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:13.421051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:11:13.421051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:13.421051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:11:13.421051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:11:13.421051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:13.421051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:11:13.421051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:13.421051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:11:13.421764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:13.421764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:11:13.421764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:11:13.421764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:11:13.421764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:11:13.421764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:11:13.421764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:11:13.421764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:11:13.421764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:11:13.421764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:11:13.421764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:13.421764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:11:13.422342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:13.422342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:11:13.422342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:13.422342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:11:13.422342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:13.422342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:11:13.422342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:13.422342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:11:13.422342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:11:13.422342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:13.422342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:11:13.422342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:13.422342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:11:13.422342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:11:13.422829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:13.422829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:11:13.422829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:13.422829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:11:13.423183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:13.423183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:11:13.423183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:11:13.423183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:11:13.423183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:11:13.423183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:11:13.423183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:11:13.423183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:11:13.423183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:11:13.423183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:11:13.423183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:11:13.423183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:11:13.423183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:11:13.423183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:11:13.423183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:11:13.423183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:11:13.423598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:11:13.423598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:11:13.423598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:11:13.423598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:11:13.423598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:11:13.423598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:13.423598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:11:13.423598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:13.423598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:11:13.423598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:13.423598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:11:13.423598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:13.423598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:11:13.423598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:13.423598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:11:13.423598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:11:13.423598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:13.423598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:11:13.424184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:13.424184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:11:13.424184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:11:13.424184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:13.424184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:11:13.424184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:13.424184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:11:13.427775 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:11:13.427775 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:11:13.427775 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:11:13.427960 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:11:13.428292 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:11:13.428569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:13.429184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:11:13.429184 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:13.429184 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:11:13.429184 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:11:13.429184 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:13.429184 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:11:13.429184 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:13.429184 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:11:13.429961 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:13.429961 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:11:13.429961 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:11:13.429961 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:11:13.429961 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:13.429961 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:11:13.429961 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:13.429961 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:11:13.429961 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:13.429961 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:11:13.429961 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:13.429961 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:11:13.429961 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:13.429961 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:11:13.429961 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:11:13.430392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:13.430392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:11:13.430392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:13.430392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:11:13.430392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:11:13.430392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:13.430392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:11:13.430392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:13.430392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:11:13.432768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:13.432768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:11:13.432768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:11:13.432768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:11:13.432768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:11:13.432768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:11:13.432768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:11:13.432768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:11:13.432768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:11:13.432768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:13.432768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:11:13.433166 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:13.433166 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:11:13.433166 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:13.433166 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:11:13.433166 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:13.433166 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:11:13.433166 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:13.433166 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:11:13.433166 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:11:13.433166 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:13.433166 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:11:13.433166 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:13.433166 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:11:13.433166 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:11:13.433589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:13.433589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:11:13.433589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:13.433589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:11:13.433589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:13.433589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:11:13.433589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:11:13.433589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:11:13.433589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:11:13.433589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:11:13.433589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:11:13.433589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:11:13.433589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:11:13.433589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:11:13.433922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:11:13.433922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:11:13.433922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:11:13.433922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:11:13.433922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:11:13.433922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:11:13.433922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:11:13.433922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:11:13.433922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:11:13.433922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:11:13.433922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:13.433922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:11:13.433922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:13.433922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:11:13.433922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:13.433922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:11:13.433922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:13.433922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:11:13.433922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:13.434264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:11:13.434264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:11:13.434264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:13.434264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:11:13.434264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:13.434264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:11:13.434264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:11:13.434264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:13.434264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:11:13.434264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:13.434264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:11:13.446019 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:11:13.446473 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:11:13.446684 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:11:13.447211 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:11:13.447664 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:11:13.447955 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:11:13.448076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:13.448845 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:11:13.448845 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:13.448845 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:11:13.448845 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:11:13.448845 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:13.448845 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:11:13.448845 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:13.448845 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:11:13.450391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:13.450391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:11:13.450391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:11:13.450391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:11:13.450391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:11:13.450391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:11:13.450391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:11:13.450391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:13.450391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:11:13.450391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:13.450391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:11:13.450391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:13.450686 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:11:13.450686 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:13.450686 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:11:13.450686 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:13.450686 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:11:13.450686 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:11:13.450686 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:13.450686 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:11:13.450686 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:13.450686 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:11:13.450686 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:11:13.450686 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:13.450686 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:11:13.451081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:13.451081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:11:13.452262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:13.452262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:11:13.452262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:11:13.452262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:11:13.452262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:11:13.452262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:11:13.452262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:11:13.452262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:11:13.452262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:11:13.452262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:11:13.452262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:11:13.452262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:11:13.452262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:11:13.452262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:11:13.452262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:11:13.452262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:11:13.452704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:11:13.452704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:11:13.452704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:13.452704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:11:13.452704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:13.452704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:11:13.452704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:13.452704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:11:13.452704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:13.452704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:11:13.452704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:13.452704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:11:13.452704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:11:13.452704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:13.452704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:11:13.452704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:13.452704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:11:13.452704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:11:13.452704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:13.455944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:11:13.455944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:13.455944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:11:17.897110 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:11:17.897878 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:11:17.897878 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:11:17.898686 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:11:17.936080 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:11:17.936478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:17.937360 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:11:17.937360 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:17.937360 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:11:17.937360 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:11:17.937360 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:17.937360 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:11:17.937360 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:17.937360 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:11:17.938690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:17.938690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:11:17.938690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:11:17.938690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:11:17.938690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:11:17.938690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:11:17.938690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:11:17.938690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:17.938690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:11:17.938690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:17.938690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:11:17.938690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:17.939392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:11:17.939392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:17.939392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:11:17.939392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:17.939392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:11:17.939392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:11:17.939392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:17.939392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:11:17.939392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:17.939392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:11:17.939392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:11:17.939392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:17.939392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:11:17.940440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:17.940440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:11:17.940440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:17.940440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:11:17.940440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:11:17.940440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:11:17.940440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:11:17.940440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:11:17.940440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:11:17.940440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:11:17.940440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:11:17.940440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:11:17.940440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:11:17.940440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:11:17.940440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:11:17.941205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:11:17.941205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:11:17.941205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:11:17.941205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:11:17.941205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:11:17.941205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:17.941205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:11:17.941205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:17.941205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:11:17.941205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:17.941205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:11:17.941205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:17.941205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:11:17.941205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:17.941205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:11:17.941205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:11:17.941205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:17.941205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:11:17.941205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:17.941997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:11:17.941997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:11:17.941997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:17.941997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:11:17.941997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:17.941997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:11:18.410294 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:11:18.411073 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:11:18.411288 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:11:18.413403 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:11:18.413735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:18.414887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:11:18.414887 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:18.414887 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:11:18.414887 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:11:18.414887 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:18.414887 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:11:18.414887 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:18.414887 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:11:18.416633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:18.416633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:11:18.416633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:11:18.416633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:11:18.416633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:11:18.416633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:11:18.416633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:11:18.416633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:18.416633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:11:18.416633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:18.416633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:11:18.416633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:18.417163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:11:18.417163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:18.417163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:11:18.417163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:18.417163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:11:18.417163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:11:18.417163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:18.417163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:11:18.417163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:18.417163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:11:18.417163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:11:18.417163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:18.417163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:11:18.417515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:18.417515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:11:18.419598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:18.419598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:11:18.419598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:11:18.419598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:11:18.419598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:11:18.419598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:11:18.419598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:11:18.419598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:11:18.419598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:11:18.419598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:11:18.419598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:11:18.419598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:11:18.419598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:11:18.419598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:11:18.419598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:11:18.419598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:11:18.420104 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:11:18.420104 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:11:18.420104 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:18.420104 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:11:18.420104 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:18.420104 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:11:18.420104 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:18.420104 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:11:18.420104 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:18.420104 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:11:18.420104 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:18.420104 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:11:18.420104 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:11:18.420104 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:18.420104 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:11:18.420104 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:18.420104 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:11:18.420104 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:11:18.420104 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:18.420598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:11:18.420598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:18.420598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:11:18.420598 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:11:18.420598 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:11:18.420598 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:11:18.420885 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:11:18.422909 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:11:18.423150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:18.423712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:11:18.423712 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:18.423712 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:11:18.423712 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:11:18.423712 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:18.423712 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:11:18.423712 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:18.423712 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:11:18.424550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:18.424550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:11:18.424550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:11:18.424550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:11:18.424550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:18.424550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:11:18.424550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:18.424550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:11:18.424550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:18.424550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:11:18.424550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:18.424550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:11:18.424550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:18.424550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:11:18.424550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:11:18.424978 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:18.424978 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:11:18.424978 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:18.424978 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:11:18.424978 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:11:18.424978 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:18.424978 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:11:18.424978 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:18.424978 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:11:18.425431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:18.425431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:11:18.425431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:11:18.425431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:11:18.425431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:11:18.425431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:11:18.425431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:11:18.425431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:11:18.425431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:11:18.425431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:11:18.425431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:18.425431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:11:18.425836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:18.425836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:11:18.425836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:18.425836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:11:18.425836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:18.425836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:11:18.425836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:18.425836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:11:18.425836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:11:18.425836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:18.425836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:11:18.425836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:18.425836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:11:18.425836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:11:18.426256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:18.426256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:11:18.426256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:18.426256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:11:18.426639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:18.426639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:11:18.426639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:11:18.426639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:11:18.426639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:11:18.426639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:11:18.426639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:11:18.426639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:11:18.426639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:11:18.426639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:11:18.426639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:11:18.426639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:11:18.426639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:11:18.426639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:11:18.426639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:11:18.426639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:11:18.427249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:11:18.427249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:11:18.427249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:11:18.427249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:11:18.427249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:11:18.427249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:18.427249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:11:18.427249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:18.427249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:11:18.427249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:18.427249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:11:18.427249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:18.427249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:11:18.427249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:18.427249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:11:18.427249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:11:18.427249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:18.427249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:11:18.427862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:18.427862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:11:18.427862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:11:18.427862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:18.427862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:11:18.427862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:18.427862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:11:18.427862 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:11:18.428108 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:11:18.428218 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:11:18.428302 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:11:18.428649 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:11:18.428901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:18.429394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:11:18.429394 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:18.429394 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:11:18.429394 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:11:18.429394 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:18.429394 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:11:18.429394 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:18.429394 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:11:18.430147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:18.430147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:11:18.430147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:11:18.430147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:11:18.430147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:18.430147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:11:18.430147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:18.430147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:11:18.430147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:18.430147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:11:18.430147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:18.430147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:11:18.430147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:18.430147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:11:18.430147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:11:18.430607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:18.430607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:11:18.430607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:18.430607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:11:18.430607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:11:18.430607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:18.430607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:11:18.430607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:18.430607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:11:18.431028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:18.431028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:11:18.431028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:11:18.431028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:11:18.431028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:11:18.431028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:11:18.431028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:11:18.431028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:11:18.431028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:11:18.431028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:18.431028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:11:18.431377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:18.431377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:11:18.431377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:18.431377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:11:18.431377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:18.431377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:11:18.431377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:18.431377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:11:18.431377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:11:18.431377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:18.431377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:11:18.431377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:18.431377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:11:18.431377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:11:18.431783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:18.431783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:11:18.431783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:18.431783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:11:18.432279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:18.432279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:11:18.432279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:11:18.432279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:11:18.432279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:11:18.432279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:11:18.432279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:11:18.432279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:11:18.432279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:11:18.432279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:11:18.432279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:11:18.432279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:11:18.432279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:11:18.432279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:11:18.432279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:11:18.432279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:11:18.432279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:11:18.432789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:11:18.432789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:11:18.432789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:11:18.432789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:18.432789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:11:18.432789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:18.432789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:11:18.432789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:18.432789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:11:18.432789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:18.432789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:11:18.432789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:18.432789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:11:18.432789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:11:18.432789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:18.432789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:11:18.432789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:18.432789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:11:18.432789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:11:18.433349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:18.433349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:11:18.433349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:18.433349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:11:18.446519 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:11:18.447105 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:11:18.447224 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:11:18.447444 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:11:18.447921 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:11:18.448150 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:11:18.448275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:18.448988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:11:18.448988 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:18.448988 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:11:18.448988 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:11:18.448988 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:18.448988 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:11:18.448988 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:18.448988 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:11:18.450537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:18.450537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:11:18.450537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:11:18.450537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:11:18.450537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:11:18.450537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:11:18.450537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:11:18.450537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:18.450537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:11:18.450537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:18.450537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:11:18.450537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:18.451027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:11:18.451027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:18.451027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:11:18.451027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:18.451027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:11:18.451027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:11:18.451027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:18.451027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:11:18.451027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:18.451027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:11:18.451027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:11:18.451027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:18.451027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:11:18.451361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:18.451361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:11:18.452111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:18.452111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:11:18.452111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:11:18.452111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:11:18.452111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:11:18.452111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:11:18.452111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:11:18.452111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:11:18.452111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:11:18.452111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:11:18.452111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:11:18.452111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:11:18.452111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:11:18.452111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:11:18.452111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:11:18.452111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:11:18.452529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:11:18.452529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:11:18.452529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:18.452529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:11:18.452529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:18.452529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:11:18.452529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:18.452529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:11:18.452529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:18.452529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:11:18.452529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:18.452529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:11:18.452529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:11:18.452529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:18.452529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:11:18.452529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:18.452529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:11:18.452529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:11:18.452529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:18.452997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:11:18.452997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:18.452997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:11:22.875784 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched CREATE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(uuid=2a41a1f6-da20-4ba0-bc90-c25111b216aa, enabled=[], external_ids={'neutron:is_ext_gw': 'True', 'neutron:network_name': 'neutron-d18a9fac-2ebf-4d1e-a86c-0f33fba8eeda', 'neutron:revision_number': '1', 'neutron:router_name': 'neutron-cab10384-14c5-492c-8bd0-061f5838310c', 'neutron:subnet_ids': 'bbd6a8cc-80e6-4336-a70b-1c962c561590 088bbd0f-7e81-4a76-8399-24d15c984009'}, gateway_chassis=[], ha_chassis_group=669e44a2-e3a3-4516-a4cf-2b2ebaefcb68, ipv6_prefix=[], ipv6_ra_configs={}, mac='fa:16:3e:51:04:94', name='lrp-af118f11-dc01-429a-b20a-e97b77b8d5ec', networks=['172.24.5.116/24', '2001:db8::1b7/64'], options={'gateway_mtu': '1430', 'reside-on-redirect-chassis': 'true'}, peer=[], status={}) old= {{(pid=106314) matches /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}}
May 09 09:11:22.898285 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:11:22.900924 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:11:22.900924 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:11:22.902902 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:11:22.933306 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:11:22.933659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:22.934592 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:11:22.934592 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:22.934592 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:11:22.934592 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:11:22.934592 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:22.934592 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:11:22.934592 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:22.934592 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:11:22.935629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:22.935629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:11:22.935629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:11:22.935629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:11:22.935629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:11:22.935629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:11:22.935629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:11:22.935629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:22.935629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:11:22.935629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:22.935629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:11:22.935629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:22.936080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:11:22.936080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:22.936080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:11:22.936080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:22.936080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:11:22.936080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:11:22.936080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:22.936080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:11:22.936080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:22.936080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:11:22.936080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:11:22.936080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:22.936080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:11:22.936442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:22.936442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:11:22.936442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:22.936442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:11:22.936442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:11:22.936442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:11:22.936442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:11:22.936442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:11:22.936442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:11:22.936442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:11:22.936442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:11:22.936442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:11:22.936442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:11:22.936442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:11:22.936442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:11:22.936891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:11:22.936891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:11:22.936891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:11:22.936891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:11:22.936891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:11:22.936891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:22.936891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:11:22.936891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:22.936891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:11:22.936891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:22.936891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:11:22.936891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:22.936891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:11:22.936891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:22.936891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:11:22.936891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:11:22.936891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:22.936891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:11:22.936891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:22.937476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:11:22.937476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:11:22.937476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:22.937476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:11:22.937476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:22.937476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:11:23.411277 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:11:23.412051 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:11:23.412270 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:11:23.413872 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:11:23.414416 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:11:23.414895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:23.416006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:11:23.416006 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:23.416006 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:11:23.416006 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:11:23.416006 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:23.416006 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:11:23.416006 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:23.416006 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:11:23.417718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:23.417718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:11:23.417718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:11:23.417718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:11:23.417718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:11:23.417718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:11:23.417718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:11:23.417718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:23.417718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:11:23.417718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:23.417718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:11:23.417718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:23.418074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:11:23.418074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:23.418074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:11:23.418074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:23.418074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:11:23.418074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:11:23.418074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:23.418074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:11:23.418074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:23.418074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:11:23.418074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:11:23.418074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:23.418074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:11:23.418406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:23.418406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:11:23.418406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:23.418406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:11:23.418406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:11:23.418406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:11:23.418406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:11:23.418406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:11:23.418406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:11:23.418406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:11:23.418406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:11:23.418406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:11:23.418406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:11:23.418406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:11:23.418406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:11:23.418822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:11:23.418822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:11:23.418822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:11:23.418822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:11:23.418822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:11:23.418822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:23.418822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:11:23.418822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:23.418822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:11:23.418822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:23.418822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:11:23.418822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:23.418822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:11:23.418822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:23.418822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:11:23.418822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:11:23.418822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:23.418822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:11:23.418822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:23.419277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:11:23.419277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:11:23.419277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:23.419277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:11:23.419277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:23.419277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:11:23.420899 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:11:23.420899 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:11:23.420899 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:11:23.421149 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:11:23.421574 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:11:23.422047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:23.423059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:11:23.423059 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:23.423059 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:11:23.423059 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:11:23.423059 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:23.423059 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:11:23.423059 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:23.423059 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:11:23.423915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:23.423915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:11:23.423915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:11:23.423915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:11:23.423915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:23.423915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:11:23.423915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:23.423915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:11:23.423915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:23.423915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:11:23.423915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:23.423915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:11:23.423915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:23.423915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:11:23.423915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:11:23.424357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:23.424357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:11:23.424357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:23.424357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:11:23.424357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:11:23.424357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:23.424357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:11:23.424357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:23.424357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:11:23.425099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:23.425099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:11:23.425099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:11:23.425099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:11:23.425099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:11:23.425099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:11:23.425099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:11:23.425099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:11:23.425099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:11:23.425099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:11:23.425099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:23.425099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:11:23.425435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:23.425435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:11:23.425435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:23.425435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:11:23.425435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:23.425435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:11:23.425435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:23.425435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:11:23.425435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:11:23.425435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:23.425435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:11:23.425435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:23.425435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:11:23.425435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:11:23.425915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:23.425915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:11:23.425915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:23.425915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:11:23.426470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:23.426470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:11:23.426470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:11:23.426470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:11:23.426470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:11:23.426470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:11:23.426470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:11:23.426470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:11:23.426470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:11:23.426470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:11:23.426470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:11:23.426470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:11:23.426470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:11:23.426470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:11:23.426470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:11:23.426470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:11:23.427590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:11:23.427590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:11:23.427590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:11:23.427590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:11:23.427590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:11:23.427590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:23.427590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:11:23.427590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:23.427590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:11:23.427590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:23.427590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:11:23.427590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:23.427590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:11:23.427590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:23.427590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:11:23.427590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:11:23.427590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:23.427590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:11:23.428566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:23.428566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:11:23.428566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:11:23.428566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:23.428566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:11:23.428566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:23.428566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:11:23.428566 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:11:23.428566 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:11:23.428850 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:11:23.428850 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:11:23.429051 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:11:23.429294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:23.429947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:11:23.429947 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:23.429947 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:11:23.429947 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:11:23.429947 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:23.429947 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:11:23.429947 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:23.429947 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:11:23.430702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:23.430702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:11:23.430702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:11:23.430702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:11:23.430702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:23.430702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:11:23.430702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:23.430702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:11:23.430702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:23.430702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:11:23.430702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:23.430702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:11:23.430702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:23.430702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:11:23.430702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:11:23.431249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:23.431249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:11:23.431249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:23.431249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:11:23.431249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:11:23.431249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:23.431249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:11:23.431249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:23.431249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:11:23.431686 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:23.431686 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:11:23.431686 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:11:23.431686 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:11:23.431686 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:11:23.431686 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:11:23.431686 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:11:23.431686 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:11:23.431686 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:11:23.431686 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:23.431686 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:11:23.432099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:23.432099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:11:23.432099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:23.432099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:11:23.432099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:23.432099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:11:23.432099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:23.432099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:11:23.432099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:11:23.432099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:23.432099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:11:23.432099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:23.432099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:11:23.432099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:11:23.432513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:23.432513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:11:23.432513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:23.432513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:11:23.432774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:23.432774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:11:23.432774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:11:23.432774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:11:23.432774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:11:23.432774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:11:23.432774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:11:23.432774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:11:23.432774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:11:23.432774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:11:23.432774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:11:23.432774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:11:23.432774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:11:23.432774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:11:23.432774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:11:23.432774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:11:23.432774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:11:23.433264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:11:23.433264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:11:23.433264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:11:23.433264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:23.433264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:11:23.433264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:23.433264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:11:23.433264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:23.433264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:11:23.433264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:23.433264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:11:23.433264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:23.433264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:11:23.433264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:11:23.433264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:23.433264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:11:23.433264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:23.433264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:11:23.433264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:11:23.433815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:23.433815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:11:23.433815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:23.433815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:11:23.447044 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:11:23.447866 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:11:23.448058 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:11:23.448435 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:11:23.449396 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:11:23.449855 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:11:23.450095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:23.451075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:11:23.451075 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:23.451075 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:11:23.451075 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:11:23.451075 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:23.451075 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:11:23.451075 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:23.451075 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:11:23.452139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:23.452139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:11:23.452139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:11:23.452139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:11:23.452139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:11:23.452139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:11:23.452139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:11:23.452139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:23.452139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:11:23.452139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:23.452139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:11:23.452139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:23.452774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:11:23.452774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:23.452774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:11:23.452774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:23.452774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:11:23.452774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:11:23.452774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:23.452774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:11:23.452774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:23.452774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:11:23.452774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:11:23.452774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:23.452774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:11:23.453547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:23.453547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:11:23.453547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:23.453547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:11:23.453547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:11:23.453547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:11:23.453547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:11:23.453547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:11:23.453547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:11:23.453547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:11:23.453547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:11:23.453547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:11:23.453547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:11:23.453547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:11:23.453547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:11:23.454333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:11:23.454333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:11:23.454333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:11:23.454333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:11:23.454333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:11:23.454333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:23.454333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:11:23.454333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:23.454333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:11:23.454333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:23.454333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:11:23.454333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:23.454333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:11:23.454333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:23.454333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:11:23.454333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:11:23.454333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:23.454333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:11:23.454333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:23.455269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:11:23.455269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:11:23.455269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:23.455269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:11:23.455269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:23.455269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:11:26.008989 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched CREATE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(uuid=3604037c-6b0e-4e9b-884e-9d4fa9d97ae5, enabled=[], external_ids={'neutron:is_ext_gw': 'False', 'neutron:network_name': 'neutron-433617e8-cdf8-414a-8cc6-63a7b5d0c678', 'neutron:revision_number': '1', 'neutron:router_name': 'neutron-cab10384-14c5-492c-8bd0-061f5838310c', 'neutron:subnet_ids': '9187f065-9264-427a-8df8-672a6981191d'}, gateway_chassis=[], ha_chassis_group=[], ipv6_prefix=[], ipv6_ra_configs={}, mac='fa:16:3e:f0:b9:0f', name='lrp-909427fa-14e8-473a-944e-ccb829c01271', networks=['192.168.24.1/24'], options={}, peer=[], status={}) old= {{(pid=106314) matches /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}}
May 09 09:11:26.042771 np0000173033 neutron-ovn-maintenance-worker[106314]: INFO neutron.common.ovn.utils [-] HA Chassis Group neutron-433617e8-cdf8-414a-8cc6-63a7b5d0c678 synchronized; highest priority chassis 674084fc-c085-45f6-9e8a-02cd5b8dd8a3
May 09 09:11:26.043362 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): HAChassisGroupAddCommand(_result=None, name=neutron-433617e8-cdf8-414a-8cc6-63a7b5d0c678, may_exist=True, columns={'external_ids': {'neutron:availability_zone_hints': '', 'neutron:network_id': '433617e8-cdf8-414a-8cc6-63a7b5d0c678', 'neutron:router_id': 'cab10384-14c5-492c-8bd0-061f5838310c'}}) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:11:26.043889 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): HAChassisGroupAddChassisCommand(_result=None, hcg=neutron-433617e8-cdf8-414a-8cc6-63a7b5d0c678, chassis=674084fc-c085-45f6-9e8a-02cd5b8dd8a3, priority=1, columns={}) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:11:26.044248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:26.045286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
May 09 09:11:26.045286 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:26.045286 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:11:26.045286 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:11:26.045286 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:26.045286 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:11:26.045286 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:26.045286 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:11:26.046453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.event [-] Unexpected exception in notify_loop: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:26.046453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.event Traceback (most recent call last):
May 09 09:11:26.046453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/event.py", line 185, in notify_loop
May 09 09:11:26.046453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.event     match.run(event, row, updates)
May 09 09:11:26.046453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.event   File "/opt/stack/neutron/neutron/services/ovn_l3/ovsdb_monitor.py", line 86, in run
May 09 09:11:26.046453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.event     self.l3_plugin._ovn_client.link_network_ha_chassis_group(
May 09 09:11:26.046453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.event   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2416, in link_network_ha_chassis_group
May 09 09:11:26.046453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.event     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:11:26.046453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.event   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:26.046453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.event     next(self.gen)
May 09 09:11:26.046453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.event   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:26.046453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.event     with super().transaction(*args, **kwargs) as t:
May 09 09:11:26.046453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.event   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:26.046453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.event     next(self.gen)
May 09 09:11:26.046453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:26.046453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.event     with self.create_transaction(
May 09 09:11:26.046453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:26.046453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.event     self.result = self.commit()
May 09 09:11:26.046453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.event                   ^^^^^^^^^^^^^
May 09 09:11:26.047106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:26.047106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.event     raise result.ex
May 09 09:11:26.047106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:26.047106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.event     txn.results.put(txn.do_commit())
May 09 09:11:26.047106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.event                     ^^^^^^^^^^^^^^^
May 09 09:11:26.047106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:26.047106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.event     raise RuntimeError(msg)
May 09 09:11:26.047106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.event RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:26.047106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.event 
May 09 09:11:27.900287 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:11:27.901096 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:11:27.902596 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:11:27.903002 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:11:27.943510 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:11:27.943908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:27.945339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:11:27.945339 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:27.945339 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:11:27.945339 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:11:27.945339 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:27.945339 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:11:27.945339 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:27.945339 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:11:27.948152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.043 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:27.948152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:11:27.948152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:11:27.948152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:11:27.948152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:11:27.948152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:11:27.948152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:11:27.948152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:27.948152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:11:27.948152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:27.948152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:11:27.948152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:27.949007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:11:27.949007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:27.949007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:11:27.949007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:27.949007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:11:27.949007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:11:27.949007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:27.949007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:11:27.949007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:27.949007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:11:27.949007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:11:27.949007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:27.949007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:11:27.949824 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:27.949824 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:11:27.949983 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:27.949983 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:11:27.949983 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:11:27.949983 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:11:27.949983 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:11:27.949983 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:11:27.949983 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:11:27.949983 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:11:27.949983 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:11:27.949983 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:11:27.949983 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:11:27.949983 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:11:27.949983 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:11:27.949983 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:11:27.949983 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:11:27.949983 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:11:27.950622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:11:27.950622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:11:27.950622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:27.950622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:11:27.950622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:27.950622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:11:27.950622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:27.950622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:11:27.950622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:27.950622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:11:27.950622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:27.950622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:11:27.950622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:11:27.950622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:27.950622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:11:27.950622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:27.950622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:11:27.950622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:11:27.950622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:27.951723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:11:27.951723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:27.951723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:11:28.411984 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:11:28.412638 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:11:28.412904 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:11:28.414224 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:11:28.414224 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:11:28.414348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:28.415611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:11:28.415611 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:28.415611 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:11:28.415611 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:11:28.415611 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:28.415611 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:11:28.415611 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:28.415611 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:11:28.416781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:28.416781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:11:28.416781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:11:28.416781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:11:28.416781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:11:28.416781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:11:28.416781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:11:28.416781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:28.416781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:11:28.416781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:28.416781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:11:28.416781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:28.417374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:11:28.417374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:28.417374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:11:28.417374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:28.417374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:11:28.417374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:11:28.417374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:28.417374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:11:28.417374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:28.417374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:11:28.417374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:11:28.417374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:28.417374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:11:28.417735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:28.417735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:11:28.417876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:28.417876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:11:28.417876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:11:28.417876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:11:28.417876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:11:28.417876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:11:28.417876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:11:28.417876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:11:28.417876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:11:28.417876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:11:28.417876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:11:28.417876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:11:28.417876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:11:28.417876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:11:28.417876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:11:28.417876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:11:28.418459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:11:28.418459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:11:28.418459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:28.418459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:11:28.418459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:28.418459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:11:28.418459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:28.418459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:11:28.418459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:28.418459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:11:28.418459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:28.418459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:11:28.418459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:11:28.418459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:28.418459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:11:28.418459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:28.418459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:11:28.418459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:11:28.418459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:28.419020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:11:28.419020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:28.419020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:11:28.420928 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:11:28.421384 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:11:28.421629 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:11:28.421875 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:11:28.422475 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:11:28.423153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:28.423817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:11:28.423817 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:28.423817 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:11:28.423817 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:11:28.423817 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:28.423817 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:11:28.423817 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:28.423817 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:11:28.424847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:28.424847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:11:28.424847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:11:28.424847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:11:28.424847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:28.424847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:11:28.424847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:28.424847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:11:28.424847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:28.424847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:11:28.424847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:28.424847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:11:28.424847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:28.424847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:11:28.424847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:11:28.425319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:28.425319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:11:28.425319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:28.425319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:11:28.425319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:11:28.425319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:28.425319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:11:28.425319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:28.425319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:11:28.425812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:28.425812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:11:28.425812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:11:28.425812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:11:28.425812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:11:28.425812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:11:28.425812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:11:28.425812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:11:28.425812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:11:28.425812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:11:28.425812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:28.425812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:11:28.426208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:28.426208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:11:28.426208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:28.426208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:11:28.426208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:28.426208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:11:28.426208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:28.426208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:11:28.426208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:11:28.426208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:28.426208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:11:28.426208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:28.426208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:11:28.426208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:11:28.426575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:28.426575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:11:28.426575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:28.426575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:11:28.427031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:28.427031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:11:28.427031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:11:28.427031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:11:28.427031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:11:28.427031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:11:28.427031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:11:28.427031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:11:28.427031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:11:28.427031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:11:28.427031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:11:28.427031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:11:28.427031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:11:28.427031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:11:28.427031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:11:28.427031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:11:28.427511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:11:28.427511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:11:28.427511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:11:28.427511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:11:28.427511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:11:28.427511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:28.427511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:11:28.427511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:28.427511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:11:28.427511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:28.427511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:11:28.427511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:28.427511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:11:28.427511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:28.427511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:11:28.427511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:11:28.427511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:28.427511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:11:28.428128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:28.428128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:11:28.428128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:11:28.428128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:28.428128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:11:28.428128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:28.428128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:11:28.428782 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:11:28.429110 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:11:28.429251 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:11:28.429471 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:11:28.431245 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:11:28.431469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:28.431951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:11:28.431951 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:28.431951 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:11:28.431951 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:11:28.431951 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:28.431951 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:11:28.431951 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:28.431951 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:11:28.432728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:28.432728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:11:28.432728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:11:28.432728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:11:28.432728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:28.432728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:11:28.432728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:28.432728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:11:28.432728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:28.432728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:11:28.432728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:28.432728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:11:28.432728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:28.432728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:11:28.432728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:11:28.433189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:28.433189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:11:28.433189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:28.433189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:11:28.433189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:11:28.433189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:28.433189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:11:28.433189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:28.433189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:11:28.433618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:28.433618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:11:28.433618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:11:28.433618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:11:28.433618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:11:28.433618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:11:28.433618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:11:28.433618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:11:28.433618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:11:28.433618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:28.433618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:11:28.433981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:28.433981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:11:28.433981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:28.433981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:11:28.433981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:28.433981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:11:28.433981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:28.433981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:11:28.433981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:11:28.433981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:28.433981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:11:28.433981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:28.433981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:11:28.433981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:11:28.434333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:28.434333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:11:28.434333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:28.434333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:11:28.434759 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:28.434759 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:11:28.434759 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:11:28.434759 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:11:28.434759 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:11:28.434759 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:11:28.434759 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:11:28.434759 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:11:28.434759 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:11:28.434759 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:11:28.434759 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:11:28.434759 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:11:28.434759 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:11:28.434759 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:11:28.434759 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:11:28.434759 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:11:28.434759 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:11:28.435242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:11:28.435242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:11:28.435242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:11:28.435242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:28.435242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:11:28.435242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:28.435242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:11:28.435242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:28.435242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:11:28.435242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:28.435242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:11:28.435242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:28.435242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:11:28.435242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:11:28.435242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:28.435242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:11:28.435242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:28.435242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:11:28.435242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:11:28.435712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:28.435712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:11:28.435712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:28.435712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:11:28.447653 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:11:28.448126 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:11:28.448368 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:11:28.448628 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:11:28.449410 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:11:28.449410 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:11:28.449699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:28.450486 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:11:28.450486 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:28.450486 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:11:28.450486 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:11:28.450486 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:28.450486 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:11:28.450486 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:28.450486 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:11:28.451701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:28.451701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:11:28.451701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:11:28.451701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:11:28.451701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:11:28.451701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:11:28.451701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:11:28.451701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:28.451701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:11:28.451701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:28.451701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:11:28.451701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:28.452118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:11:28.452118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:28.452118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:11:28.452118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:28.452118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:11:28.452118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:11:28.452118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:28.452118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:11:28.452118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:28.452118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:11:28.452118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:11:28.452118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:28.452118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:11:28.452449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:28.452449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:11:28.452821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:28.452821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:11:28.452821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:11:28.452821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:11:28.452821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:11:28.452821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:11:28.452821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:11:28.452821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:11:28.452821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:11:28.452821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:11:28.452821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:11:28.452821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:11:28.452821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:11:28.452821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:11:28.452821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:11:28.452821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:11:28.453275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:11:28.453275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:11:28.453275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:28.453275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:11:28.453275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:28.453275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:11:28.453275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:28.453275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:11:28.453275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:28.453275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:11:28.453275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:28.453275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:11:28.453275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:11:28.453275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:28.453275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:11:28.453275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:28.453275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:11:28.453275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:11:28.453275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:28.453721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:11:28.453721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:28.453721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:11:32.901448 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:11:32.902369 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:11:32.902369 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:11:32.903652 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:11:32.935928 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:11:32.936183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:32.936679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:11:32.936679 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:32.936679 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:11:32.936679 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:11:32.936679 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:32.936679 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:11:32.936679 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:32.936679 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:11:32.937648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:32.937648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:11:32.937648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:11:32.937648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:11:32.937648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:11:32.937648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:11:32.937648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:11:32.937648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:32.937648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:11:32.937648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:32.937648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:11:32.937648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:32.938335 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:11:32.938335 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:32.938335 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:11:32.938335 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:32.938335 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:11:32.938335 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:11:32.938335 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:32.938335 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:11:32.938335 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:32.938335 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:11:32.938335 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:11:32.938335 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:32.938335 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:11:32.939005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:32.939005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:11:32.939005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:32.939005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:11:32.939005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:11:32.939005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:11:32.939005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:11:32.939005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:11:32.939005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:11:32.939005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:11:32.939005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:11:32.939005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:11:32.939005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:11:32.939005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:11:32.939005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:11:32.940066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:11:32.940066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:11:32.940066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:11:32.940066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:11:32.940066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:11:32.940066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:32.940066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:11:32.940066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:32.940066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:11:32.940066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:32.940066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:11:32.940066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:32.940066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:11:32.940066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:32.940066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:11:32.940066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:11:32.940066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:32.940066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:11:32.940066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:32.942062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:11:32.942062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:11:32.942062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:32.942062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:11:32.942062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:32.942062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:11:33.412689 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:11:33.413081 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:11:33.413213 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:11:33.413942 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:11:33.414243 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:11:33.414474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:33.415293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:11:33.415293 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:33.415293 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:11:33.415293 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:11:33.415293 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:33.415293 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:11:33.415293 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:33.415293 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:11:33.416887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:33.416887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:11:33.416887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:11:33.416887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:11:33.416887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:11:33.416887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:11:33.416887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:11:33.416887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:33.416887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:11:33.416887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:33.416887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:11:33.416887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:33.417530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:11:33.417530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:33.417530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:11:33.417530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:33.417530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:11:33.417530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:11:33.417530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:33.417530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:11:33.417530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:33.417530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:11:33.417530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:11:33.417530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:33.417530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:11:33.418101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:33.418101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:11:33.419037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:33.419037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:11:33.419037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:11:33.419037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:11:33.419037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:11:33.419037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:11:33.419037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:11:33.419037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:11:33.419037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:11:33.419037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:11:33.419037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:11:33.419037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:11:33.419037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:11:33.419037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:11:33.419037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:11:33.419037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:11:33.419939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:11:33.419939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:11:33.419939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:33.419939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:11:33.419939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:33.419939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:11:33.419939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:33.419939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:11:33.419939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:33.419939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:11:33.419939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:33.419939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:11:33.419939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:11:33.419939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:33.419939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:11:33.419939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:33.419939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:11:33.419939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:11:33.419939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:33.420773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:11:33.420773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:33.420773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:11:33.421612 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:11:33.422069 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:11:33.422181 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:11:33.422294 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:11:33.422544 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:11:33.422723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:33.423271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:11:33.423271 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:33.423271 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:11:33.423271 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:11:33.423271 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:33.423271 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:11:33.423271 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:33.423271 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:11:33.424331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:33.424331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:11:33.424331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:11:33.424331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:11:33.424331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:33.424331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:11:33.424331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:33.424331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:11:33.424331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:33.424331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:11:33.424331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:33.424331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:11:33.424331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:33.424331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:11:33.424331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:11:33.424914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:33.424914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:11:33.424914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:33.424914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:11:33.424914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:11:33.424914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:33.424914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:11:33.424914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:33.424914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:11:33.425371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:33.425371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:11:33.425371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:11:33.425371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:11:33.425371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:11:33.425371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:11:33.425371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:11:33.425371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:11:33.425371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:11:33.425371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:11:33.425371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:33.425371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:11:33.425729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:33.425729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:11:33.425729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:33.425729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:11:33.425729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:33.425729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:11:33.425729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:33.425729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:11:33.425729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:11:33.425729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:33.425729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:11:33.425729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:33.425729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:11:33.425729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:11:33.426078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:33.426078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:11:33.426078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:33.426078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:11:33.426368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:33.426368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:11:33.426368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:11:33.426368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:11:33.426368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:11:33.426368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:11:33.426368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:11:33.426368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:11:33.426368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:11:33.426368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:11:33.426368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:11:33.426368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:11:33.426368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:11:33.426368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:11:33.426368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:11:33.426368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:11:33.426837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:11:33.426837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:11:33.426837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:11:33.426837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:11:33.426837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:11:33.426837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:33.426837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:11:33.426837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:33.426837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:11:33.426837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:33.426837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:11:33.426837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:33.426837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:11:33.426837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:33.426837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:11:33.426837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:11:33.426837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:33.426837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:11:33.427381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:33.427381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:11:33.427381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:11:33.427381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:33.427381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:11:33.427381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:33.427381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:11:33.429096 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:11:33.429348 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:11:33.429463 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:11:33.429580 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:11:33.430780 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:11:33.430780 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:33.430964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:11:33.430964 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:33.430964 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:11:33.430964 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:11:33.430964 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:33.430964 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:11:33.430964 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:33.430964 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:11:33.431906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:33.431906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:11:33.431906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:11:33.431906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:11:33.431906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:33.431906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:11:33.431906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:33.431906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:11:33.431906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:33.431906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:11:33.431906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:33.431906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:11:33.431906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:33.431906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:11:33.431906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:11:33.432477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:33.432477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:11:33.432477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:33.432477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:11:33.432477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:11:33.432477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:33.432477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:11:33.432477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:33.432477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:11:33.433226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:33.433226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:11:33.433226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:11:33.433226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:11:33.433226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:11:33.433226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:11:33.433226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:11:33.433226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:11:33.433226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:11:33.433226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:33.433226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:11:33.433586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:33.433586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:11:33.433586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:33.433586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:11:33.433586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:33.433586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:11:33.433586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:33.433586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:11:33.433586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:11:33.433586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:33.433586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:11:33.433586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:33.433586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:11:33.433586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:11:33.433953 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:33.433953 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:11:33.433953 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:33.433953 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:11:33.434304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:33.434304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:11:33.434304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:11:33.434304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:11:33.434304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:11:33.434304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:11:33.434304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:11:33.434304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:11:33.434304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:11:33.434304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:11:33.434304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:11:33.434304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:11:33.434304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:11:33.434304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:11:33.434304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:11:33.434304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:11:33.434304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:11:33.434784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:11:33.434784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:11:33.434784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:11:33.434784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:33.434784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:11:33.434784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:33.434784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:11:33.434784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:33.434784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:11:33.434784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:33.434784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:11:33.434784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:33.434784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:11:33.434784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:11:33.434784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:33.434784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:11:33.434784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:33.434784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:11:33.434784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:11:33.435264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:33.435264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:11:33.435264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:33.435264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:11:33.448148 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:11:33.448647 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:11:33.448881 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:11:33.449203 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:11:33.449988 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:11:33.450239 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:11:33.450478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:33.451320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:11:33.451320 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:33.451320 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:11:33.451320 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:11:33.451320 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:33.451320 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:11:33.451320 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:33.451320 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:11:33.452779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:33.452779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:11:33.452779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:11:33.452779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:11:33.452779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:11:33.452779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:11:33.452779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:11:33.452779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:33.452779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:11:33.452779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:33.452779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:11:33.452779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:33.453377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:11:33.453377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:33.453377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:11:33.453377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:33.453377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:11:33.453377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:11:33.453377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:33.453377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:11:33.453377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:33.453377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:11:33.453377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:11:33.453377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:33.453377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:11:33.453981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:33.453981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:11:33.454483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:33.454483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:11:33.454483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:11:33.454483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:11:33.454483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:11:33.454483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:11:33.454483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:11:33.454483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:11:33.454483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:11:33.454483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:11:33.454483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:11:33.454483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:11:33.454483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:11:33.454483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:11:33.454483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:11:33.454483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:11:33.455320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:11:33.455320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:11:33.455320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:33.455320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:11:33.455320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:33.455320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:11:33.455320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:33.455320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:11:33.455320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:33.455320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:11:33.455320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:33.455320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:11:33.455320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:11:33.455320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:33.455320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:11:33.455320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:33.455320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:11:33.455320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:11:33.455320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:33.457003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:11:33.457003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:33.457003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:11:37.902225 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:11:37.903012 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:11:37.903114 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:11:37.904139 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:11:37.940503 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:11:37.940996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:37.941960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:11:37.941960 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:37.941960 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:11:37.941960 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:11:37.941960 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:37.941960 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:11:37.941960 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:37.941960 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:11:37.942735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:37.942735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:11:37.942735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:11:37.942735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:11:37.942735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:11:37.942735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:11:37.942735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:11:37.942735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:37.942735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:11:37.942735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:37.942735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:11:37.942735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:37.943143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:11:37.943143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:37.943143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:11:37.943143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:37.943143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:11:37.943143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:11:37.943143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:37.943143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:11:37.943143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:37.943143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:11:37.943143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:11:37.943143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:37.943143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:11:37.943501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:37.943501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:11:37.943565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:37.943565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:11:37.943565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:11:37.943565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:11:37.943565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:11:37.943565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:11:37.943565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:11:37.943565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:11:37.943565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:11:37.943565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:11:37.943565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:11:37.943565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:11:37.943565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:11:37.943565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:11:37.943565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:11:37.943565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:11:37.944061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:11:37.944061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:11:37.944061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:37.944061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:11:37.944061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:37.944061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:11:37.944061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:37.944061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:11:37.944061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:37.944061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:11:37.944061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:37.944061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:11:37.944061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:11:37.944061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:37.944061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:11:37.944061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:37.944061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:11:37.944061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:11:37.944061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:37.944674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:11:37.944674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:37.944674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:11:38.413298 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:11:38.414017 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:11:38.414313 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:11:38.415770 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:11:38.416211 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:11:38.416562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:38.417545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:11:38.417545 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:38.417545 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:11:38.417545 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:11:38.417545 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:38.417545 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:11:38.417545 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:38.417545 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:11:38.418615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:38.418615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:11:38.418615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:11:38.418615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:11:38.418615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:11:38.418615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:11:38.418615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:11:38.418615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:38.418615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:11:38.418615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:38.418615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:11:38.418615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:38.419047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:11:38.419047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:38.419047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:11:38.419047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:38.419047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:11:38.419047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:11:38.419047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:38.419047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:11:38.419047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:38.419047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:11:38.419047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:11:38.419047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:38.419047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:11:38.419411 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:38.419411 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:11:38.419700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:38.419700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:11:38.419700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:11:38.419700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:11:38.419700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:11:38.419700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:11:38.419700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:11:38.419700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:11:38.419700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:11:38.419700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:11:38.419700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:11:38.419700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:11:38.419700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:11:38.419700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:11:38.419700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:11:38.419700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:11:38.420150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:11:38.420150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:11:38.420150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:38.420150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:11:38.420150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:38.420150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:11:38.420150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:38.420150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:11:38.420150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:38.420150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:11:38.420150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:38.420150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:11:38.420150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:11:38.420150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:38.420150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:11:38.420150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:38.420150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:11:38.420150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:11:38.420150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:38.420678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:11:38.420678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:38.420678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:11:38.422111 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:11:38.422378 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:11:38.422493 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:11:38.422614 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:11:38.423159 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:11:38.423465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:38.424245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:11:38.424245 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:38.424245 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:11:38.424245 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:11:38.424245 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:38.424245 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:11:38.424245 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:38.424245 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:11:38.425026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:38.425026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:11:38.425026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:11:38.425026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:11:38.425026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:38.425026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:11:38.425026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:38.425026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:11:38.425026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:38.425026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:11:38.425026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:38.425026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:11:38.425026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:38.425026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:11:38.425026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:11:38.425447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:38.425447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:11:38.425447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:38.425447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:11:38.425447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:11:38.425447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:38.425447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:11:38.425447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:38.425447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:11:38.425994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:38.425994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:11:38.425994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:11:38.425994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:11:38.425994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:11:38.425994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:11:38.425994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:11:38.425994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:11:38.425994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:11:38.425994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:11:38.425994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:38.425994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:11:38.426365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:38.426365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:11:38.426365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:38.426365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:11:38.426365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:38.426365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:11:38.426365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:38.426365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:11:38.426365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:11:38.426365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:38.426365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:11:38.426365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:38.426365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:11:38.426365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:11:38.426770 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:38.426770 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:11:38.426770 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:38.426770 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:11:38.427049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:38.427049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:11:38.427049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:11:38.427049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:11:38.427049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:11:38.427049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:11:38.427049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:11:38.427049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:11:38.427049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:11:38.427049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:11:38.427049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:11:38.427049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:11:38.427049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:11:38.427049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:11:38.427049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:11:38.427049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:11:38.427495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:11:38.427495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:11:38.427495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:11:38.427495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:11:38.427495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:11:38.427495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:38.427495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:11:38.427495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:38.427495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:11:38.427495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:38.427495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:11:38.427495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:38.427495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:11:38.427495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:38.427495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:11:38.427495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:11:38.427495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:38.427495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:11:38.428028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:38.428028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:11:38.428028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:11:38.428028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:38.428028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:11:38.428028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:38.428028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:11:38.429406 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:11:38.429642 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:11:38.429751 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:11:38.429895 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:11:38.430355 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:11:38.430719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:38.431688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:11:38.431688 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:38.431688 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:11:38.431688 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:11:38.431688 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:38.431688 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:11:38.431688 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:38.431688 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:11:38.432604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:38.432604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:11:38.432604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:11:38.432604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:11:38.432604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:38.432604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:11:38.432604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:38.432604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:11:38.432604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:38.432604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:11:38.432604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:38.432604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:11:38.432604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:38.432604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:11:38.432604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:11:38.433011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:38.433011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:11:38.433011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:38.433011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:11:38.433011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:11:38.433011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:38.433011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:11:38.433011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:38.433011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:11:38.433467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:38.433467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:11:38.433467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:11:38.433467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:11:38.433467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:11:38.433467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:11:38.433467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:11:38.433467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:11:38.433467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:11:38.433467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:38.433467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:11:38.433804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:38.433804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:11:38.433804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:38.433804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:11:38.433804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:38.433804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:11:38.433804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:38.433804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:11:38.433804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:11:38.433804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:38.433804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:11:38.433804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:38.433804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:11:38.433804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:11:38.434179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:38.434179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:11:38.434179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:38.434179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:11:38.434502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:38.434502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:11:38.434502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:11:38.434502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:11:38.434502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:11:38.434502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:11:38.434502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:11:38.434502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:11:38.434502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:11:38.434502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:11:38.434502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:11:38.434502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:11:38.434502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:11:38.434502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:11:38.434502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:11:38.434502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:11:38.434502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:11:38.434991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:11:38.434991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:11:38.434991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:11:38.434991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:38.434991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:11:38.434991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:38.434991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:11:38.434991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:38.434991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:11:38.434991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:38.434991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:11:38.434991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:38.434991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:11:38.434991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:11:38.434991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:38.434991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:11:38.434991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:38.434991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:11:38.434991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:11:38.435489 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:38.435489 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:11:38.435489 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:38.435489 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:11:38.448541 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:11:38.448933 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:11:38.449116 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:11:38.449349 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:11:38.449915 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:11:38.450272 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:11:38.450518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:38.451380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:11:38.451380 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:38.451380 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:11:38.451380 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:11:38.451380 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:38.451380 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:11:38.451380 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:38.451380 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:11:38.452725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:38.452725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:11:38.452725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:11:38.452725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:11:38.452725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:11:38.452725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:11:38.452725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:11:38.452725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:38.452725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:11:38.452725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:38.452725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:11:38.452725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:38.453361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:11:38.453361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:38.453361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:11:38.453361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:38.453361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:11:38.453361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:11:38.453361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:38.453361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:11:38.453361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:38.453361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:11:38.453361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:11:38.453361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:38.453361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:11:38.453930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:38.453930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:11:38.454301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:38.454301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:11:38.454301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:11:38.454301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:11:38.454301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:11:38.454301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:11:38.454301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:11:38.454301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:11:38.454301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:11:38.454301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:11:38.454301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:11:38.454301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:11:38.454301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:11:38.454301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:11:38.454301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:11:38.454301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:11:38.455024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:11:38.455024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:11:38.455024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:38.455024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:11:38.455024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:38.455024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:11:38.455024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:38.455024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:11:38.455024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:38.455024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:11:38.455024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:38.455024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:11:38.455024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:11:38.455024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:38.455024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:11:38.455024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:38.455024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:11:38.455024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:11:38.455024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:38.455872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:11:38.455872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:38.455872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:11:42.902945 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:11:42.903976 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:11:42.904158 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:11:42.905607 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:11:42.907793 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG dbcounter [-] [106314] Writing DB stats neutron:SELECT=273 {{(pid=106314) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 09 09:11:42.948846 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:11:42.949296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:42.950397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:11:42.950397 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:42.950397 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:11:42.950397 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:11:42.950397 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:42.950397 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:11:42.950397 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:42.950397 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:11:42.951645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.046 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:42.951645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:11:42.951645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:11:42.951645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:11:42.951645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:11:42.951645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:11:42.951645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:11:42.951645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:42.951645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:11:42.951645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:42.951645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:11:42.951645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:42.952275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:11:42.952275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:42.952275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:11:42.952275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:42.952275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:11:42.952275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:11:42.952275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:42.952275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:11:42.952275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:42.952275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:11:42.952275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:11:42.952275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:42.952275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:11:42.952886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:42.952886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:11:42.952886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:42.952886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:11:42.952886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:11:42.952886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:11:42.952886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:11:42.952886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:11:42.952886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:11:42.952886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:11:42.952886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:11:42.952886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:11:42.952886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:11:42.952886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:11:42.952886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:11:42.953585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:11:42.953585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:11:42.953585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:11:42.953585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:11:42.953585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:11:42.953585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:42.953585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:11:42.953585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:42.953585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:11:42.953585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:42.953585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:11:42.953585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:42.953585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:11:42.953585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:42.953585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:11:42.953585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:11:42.953585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:42.953585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:11:42.953585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:42.954515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:11:42.954515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:11:42.954515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:42.954515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:11:42.954515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:42.954515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:11:43.414069 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:11:43.414771 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:11:43.415021 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:11:43.415926 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:11:43.416118 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:11:43.416259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:43.416983 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:11:43.416983 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:43.416983 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:11:43.416983 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:11:43.416983 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:43.416983 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:11:43.416983 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:43.416983 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:11:43.418579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:43.418579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:11:43.418579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:11:43.418579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:11:43.418579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:11:43.418579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:11:43.418579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:11:43.418579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:43.418579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:11:43.418579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:43.418579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:11:43.418579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:43.419132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:11:43.419132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:43.419132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:11:43.419132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:43.419132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:11:43.419132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:11:43.419132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:43.419132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:11:43.419132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:43.419132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:11:43.419132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:11:43.419132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:43.419132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:11:43.419676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:43.419676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:11:43.419801 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:43.419801 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:11:43.419801 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:11:43.419801 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:11:43.419801 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:11:43.419801 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:11:43.419801 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:11:43.419801 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:11:43.419801 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:11:43.419801 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:11:43.419801 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:11:43.419801 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:11:43.419801 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:11:43.419801 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:11:43.419801 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:11:43.419801 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:11:43.420463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:11:43.420463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:11:43.420463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:43.420463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:11:43.420463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:43.420463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:11:43.420463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:43.420463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:11:43.420463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:43.420463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:11:43.420463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:43.420463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:11:43.420463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:11:43.420463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:43.420463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:11:43.420463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:43.420463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:11:43.420463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:11:43.420463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:43.421306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:11:43.421306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:43.421306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:11:43.422660 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:11:43.422989 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:11:43.423098 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:11:43.423218 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:11:43.423680 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:11:43.424001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:43.424689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:11:43.424689 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:43.424689 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:11:43.424689 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:11:43.424689 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:43.424689 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:11:43.424689 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:43.424689 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:11:43.425546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:43.425546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:11:43.425546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:11:43.425546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:11:43.425546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:43.425546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:11:43.425546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:43.425546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:11:43.425546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:43.425546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:11:43.425546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:43.425546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:11:43.425546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:43.425546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:11:43.425546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:11:43.425943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:43.425943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:11:43.425943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:43.425943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:11:43.425943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:11:43.425943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:43.425943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:11:43.425943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:43.425943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:11:43.426400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:43.426400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:11:43.426400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:11:43.426400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:11:43.426400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:11:43.426400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:11:43.426400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:11:43.426400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:11:43.426400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:11:43.426400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:11:43.426400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:43.426400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:11:43.427509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:43.427509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:11:43.427509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:43.427509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:11:43.427509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:43.427509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:11:43.427509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:43.427509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:11:43.427509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:11:43.427509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:43.427509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:11:43.427509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:43.427509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:11:43.427509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:11:43.427962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:43.427962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:11:43.427962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:43.427962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:11:43.427962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:43.427962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:11:43.427962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:11:43.427962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:11:43.427962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:11:43.427962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:11:43.427962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:11:43.427962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:11:43.427962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:11:43.427962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:11:43.428372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:11:43.428372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:11:43.428372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:11:43.428372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:11:43.428372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:11:43.428372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:11:43.428372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:11:43.428372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:11:43.428372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:11:43.428372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:11:43.428372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:11:43.428372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:43.428372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:11:43.428372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:43.428372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:11:43.428372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:43.428372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:11:43.428372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:43.428372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:11:43.428951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:43.428951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:11:43.428951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:11:43.428951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:43.428951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:11:43.428951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:43.428951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:11:43.428951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:11:43.428951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:43.428951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:11:43.428951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:43.428951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:11:43.429846 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:11:43.430155 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:11:43.430299 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:11:43.430432 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:11:43.430869 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:11:43.431225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:43.431924 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:11:43.431924 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:43.431924 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:11:43.431924 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:11:43.431924 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:43.431924 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:11:43.431924 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:43.431924 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:11:43.432640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:43.432640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:11:43.432640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:11:43.432640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:11:43.432640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:43.432640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:11:43.432640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:43.432640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:11:43.432640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:43.432640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:11:43.432640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:43.432640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:11:43.432640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:43.432640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:11:43.432640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:11:43.433110 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:43.433110 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:11:43.433110 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:43.433110 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:11:43.433110 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:11:43.433110 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:43.433110 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:11:43.433110 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:43.433110 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:11:43.433616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:43.433616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:11:43.433616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:11:43.433616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:11:43.433616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:11:43.433616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:11:43.433616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:11:43.433616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:11:43.433616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:11:43.433616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:43.433616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:11:43.433944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:43.433944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:11:43.433944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:43.433944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:11:43.433944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:43.433944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:11:43.433944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:43.433944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:11:43.433944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:11:43.433944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:43.433944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:11:43.433944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:43.433944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:11:43.433944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:11:43.434355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:43.434355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:11:43.434355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:43.434355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:11:43.434715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:43.434715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:11:43.434715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:11:43.434715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:11:43.434715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:11:43.434715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:11:43.434715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:11:43.434715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:11:43.434715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:11:43.434715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:11:43.434715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:11:43.434715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:11:43.434715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:11:43.434715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:11:43.434715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:11:43.434715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:11:43.434715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:11:43.435228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:11:43.435228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:11:43.435228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:11:43.435228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:43.435228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:11:43.435228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:43.435228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:11:43.435228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:43.435228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:11:43.435228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:43.435228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:11:43.435228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:43.435228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:11:43.435228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:11:43.435228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:43.435228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:11:43.435228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:43.435228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:11:43.435228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:11:43.435774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:43.435774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:11:43.435774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:43.435774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:11:43.449093 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:11:43.449568 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:11:43.449787 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:11:43.450106 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:11:43.451969 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:11:43.452149 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:11:43.452405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:43.453629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:11:43.453629 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:43.453629 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:11:43.453629 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:11:43.453629 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:43.453629 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:11:43.453629 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:43.453629 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:11:43.455210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:43.455210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:11:43.455210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:11:43.455210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:11:43.455210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:11:43.455210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:11:43.455210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:11:43.455210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:43.455210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:11:43.455210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:43.455210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:11:43.455210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:43.455827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:11:43.455827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:43.455827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:11:43.455827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:43.455827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:11:43.455827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:11:43.455827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:43.455827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:11:43.455827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:43.455827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:11:43.455827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:11:43.455827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:43.455827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:11:43.456372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:43.456372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:11:43.457405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:43.457405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:11:43.457405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:11:43.457405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:11:43.457405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:11:43.457405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:11:43.457405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:11:43.457405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:11:43.457405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:11:43.457405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:11:43.457405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:11:43.457405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:11:43.457405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:11:43.457405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:11:43.457405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:11:43.457405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:11:43.458160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:11:43.458160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:11:43.458160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:43.458160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:11:43.458160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:43.458160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:11:43.458160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:43.458160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:11:43.458160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:43.458160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:11:43.458160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:43.458160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:11:43.458160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:11:43.458160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:43.458160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:11:43.458160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:43.458160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:11:43.458160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:11:43.458160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:43.458637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:11:43.458637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:43.458637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:11:47.903843 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:11:47.904747 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:11:47.904747 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:11:47.905913 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:11:47.939967 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:11:47.941700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:47.941700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:11:47.941700 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:47.941700 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:11:47.941700 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:11:47.941700 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:47.941700 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:11:47.941700 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:47.941700 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:11:47.942394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:47.942394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:11:47.942394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:11:47.942394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:11:47.942394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:11:47.942394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:11:47.942394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:11:47.942394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:47.942394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:11:47.942394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:47.942394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:11:47.942394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:47.943258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:11:47.943258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:47.943258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:11:47.943258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:47.943258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:11:47.943258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:11:47.943258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:47.943258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:11:47.943258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:47.943258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:11:47.943258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:11:47.943258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:47.943258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:11:47.945159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:47.945159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:11:47.945159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:47.945159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:11:47.945159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:11:47.945159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:11:47.945159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:11:47.945159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:11:47.945159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:11:47.945159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:11:47.945159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:11:47.945159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:11:47.945159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:11:47.945159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:11:47.945159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:11:47.946016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:11:47.946016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:11:47.946016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:11:47.946016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:11:47.946016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:11:47.946016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:47.946016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:11:47.946016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:47.946016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:11:47.946016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:47.946016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:11:47.946016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:47.946016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:11:47.946016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:47.946016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:11:47.946016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:11:47.946016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:47.946016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:11:47.946016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:47.947021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:11:47.947021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:11:47.947021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:47.947021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:11:47.947021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:47.947021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:11:48.415183 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:11:48.415534 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:11:48.415858 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:11:48.417036 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:11:48.417418 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:11:48.417692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:48.418650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:11:48.418650 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:48.418650 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:11:48.418650 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:11:48.418650 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:48.418650 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:11:48.418650 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:48.418650 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:11:48.420398 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:48.420398 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:11:48.420398 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:11:48.420398 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:11:48.420398 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:11:48.420398 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:11:48.420398 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:11:48.420398 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:48.420398 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:11:48.420398 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:48.420398 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:11:48.420398 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:48.421017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:11:48.421017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:48.421017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:11:48.421017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:48.421017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:11:48.421017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:11:48.421017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:48.421017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:11:48.421017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:48.421017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:11:48.421017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:11:48.421017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:48.421017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:11:48.421540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:48.421540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:11:48.422204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:48.422204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:11:48.422204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:11:48.422204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:11:48.422204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:11:48.422204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:11:48.422204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:11:48.422204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:11:48.422204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:11:48.422204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:11:48.422204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:11:48.422204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:11:48.422204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:11:48.422204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:11:48.422204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:11:48.422204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:11:48.422901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:11:48.422901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:11:48.422901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:48.422901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:11:48.422901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:48.422901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:11:48.422901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:48.422901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:11:48.422901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:48.422901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:11:48.422901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:48.422901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:11:48.422901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:11:48.422901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:48.422901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:11:48.422901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:48.422901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:11:48.422901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:11:48.422901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:48.423416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:11:48.423416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:48.423416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:11:48.424988 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:11:48.425330 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:11:48.425472 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:11:48.425648 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:11:48.426161 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:11:48.426399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:48.427023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:11:48.427023 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:48.427023 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:11:48.427023 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:11:48.427023 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:48.427023 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:11:48.427023 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:48.427023 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:11:48.428185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:48.428185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:11:48.428185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:11:48.428185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:11:48.428185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:48.428185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:11:48.428185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:48.428185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:11:48.428185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:48.428185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:11:48.428185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:48.428185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:11:48.428185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:48.428185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:11:48.428185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:11:48.428725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:48.428725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:11:48.428725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:48.428725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:11:48.428725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:11:48.428725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:48.428725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:11:48.428725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:48.428725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:11:48.429329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:48.429329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:11:48.429329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:11:48.429329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:11:48.429329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:11:48.429329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:11:48.429329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:11:48.429329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:11:48.429329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:11:48.429329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:11:48.429329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:48.429329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:11:48.429715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:48.429715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:11:48.429715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:48.429715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:11:48.429715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:48.429715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:11:48.429715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:48.429715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:11:48.429715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:11:48.429715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:48.429715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:11:48.429715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:48.429715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:11:48.429715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:11:48.430113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:48.430113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:11:48.430113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:48.430113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:11:48.431316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:48.431316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:11:48.431316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:11:48.431316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:11:48.431316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:11:48.431316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:11:48.431316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:11:48.431316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:11:48.431316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:11:48.431316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:11:48.431316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:11:48.431316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:11:48.431316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:11:48.431316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:11:48.431316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:11:48.431316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:11:48.432311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:11:48.432311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:11:48.432311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:11:48.432311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:11:48.432311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:11:48.432311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:48.432311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:11:48.432311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:48.432311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:11:48.432311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:48.432311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:11:48.432311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:48.432311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:11:48.432311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:48.432311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:11:48.432311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:11:48.432311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:48.432311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:11:48.432823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:48.432823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:11:48.432823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:11:48.432823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:48.432823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:11:48.432823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:48.432823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:11:48.432823 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:11:48.432823 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:11:48.432823 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:11:48.433106 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:11:48.433231 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:11:48.433616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:48.434555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:11:48.434555 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:48.434555 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:11:48.434555 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:11:48.434555 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:48.434555 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:11:48.434555 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:48.434555 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:11:48.436646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:48.436646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:11:48.436646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:11:48.436646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:11:48.436646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:48.436646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:11:48.436646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:48.436646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:11:48.436646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:48.436646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:11:48.436646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:48.436646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:11:48.436646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:48.436646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:11:48.436646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:11:48.437371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:48.437371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:11:48.437371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:48.437371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:11:48.437371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:11:48.437371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:48.437371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:11:48.437371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:48.437371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:11:48.438180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:48.438180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:11:48.438180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:11:48.438180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:11:48.438180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:11:48.438180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:11:48.438180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:11:48.438180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:11:48.438180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:11:48.438180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:48.438180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:11:48.438640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:48.438640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:11:48.438640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:48.438640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:11:48.438640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:48.438640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:11:48.438640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:48.438640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:11:48.438640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:11:48.438640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:48.438640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:11:48.438640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:48.438640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:11:48.438640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:11:48.439236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:48.439236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:11:48.439236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:48.439236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:11:48.439882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:48.439882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:11:48.439882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:11:48.439882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:11:48.439882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:11:48.439882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:11:48.439882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:11:48.439882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:11:48.439882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:11:48.439882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:11:48.439882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:11:48.439882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:11:48.439882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:11:48.439882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:11:48.439882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:11:48.439882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:11:48.439882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:11:48.440586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:11:48.440586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:11:48.440586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:11:48.440586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:48.440586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:11:48.440586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:48.440586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:11:48.440586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:48.440586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:11:48.440586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:48.440586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:11:48.440586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:48.440586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:11:48.440586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:11:48.440586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:48.440586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:11:48.440586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:48.440586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:11:48.440586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:11:48.441317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:48.441317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:11:48.441317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:48.441317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:11:48.449608 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:11:48.450075 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:11:48.450353 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:11:48.450681 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:11:48.451555 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:11:48.452080 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:11:48.452437 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:48.453555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:11:48.453555 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:48.453555 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:11:48.453555 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:11:48.453555 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:48.453555 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:11:48.453555 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:48.453555 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:11:48.455251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:48.455251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:11:48.455251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:11:48.455251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:11:48.455251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:11:48.455251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:11:48.455251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:11:48.455251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:48.455251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:11:48.455251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:48.455251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:11:48.455251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:48.455739 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:11:48.455739 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:48.455739 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:11:48.455739 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:48.455739 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:11:48.455739 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:11:48.455739 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:48.455739 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:11:48.455739 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:48.455739 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:11:48.455739 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:11:48.455739 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:48.455739 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:11:48.456214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:48.456214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:11:48.457696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:48.457696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:11:48.457696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:11:48.457696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:11:48.457696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:11:48.457696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:11:48.457696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:11:48.457696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:11:48.457696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:11:48.457696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:11:48.457696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:11:48.457696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:11:48.457696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:11:48.457696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:11:48.457696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:11:48.457696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:11:48.458952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:11:48.458952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:11:48.458952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:48.458952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:11:48.458952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:48.458952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:11:48.458952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:48.458952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:11:48.458952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:48.458952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:11:48.458952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:48.458952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:11:48.458952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:11:48.458952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:48.458952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:11:48.458952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:48.458952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:11:48.458952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:11:48.458952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:48.459907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:11:48.459907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:48.459907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:11:52.904526 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:11:52.906170 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:11:52.906633 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:11:52.908390 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:11:52.951965 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:11:52.952373 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:52.956154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:11:52.956154 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:52.956154 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:11:52.956154 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:11:52.956154 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:52.956154 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:11:52.956154 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:52.956154 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:11:52.956154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.047 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:52.956154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:11:52.956154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:11:52.956154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:11:52.956154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:11:52.956154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:11:52.956154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:11:52.957292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:52.957292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:11:52.957292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:52.957292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:11:52.957292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:52.957292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:11:52.957292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:52.957292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:11:52.957292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:52.957292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:11:52.957292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:11:52.957292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:52.957292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:11:52.957292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:52.958472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:11:52.958472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:11:52.958472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:52.958472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:11:52.958472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:52.958472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:11:52.958472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:52.958472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:11:52.958472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:11:52.958472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:11:52.958472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:11:52.958472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:11:52.958472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:11:52.958472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:11:52.959379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:11:52.959379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:11:52.959379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:11:52.959379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:11:52.959379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:11:52.959379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:11:52.959379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:11:52.959379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:11:52.959379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:11:52.959379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:11:52.959379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:52.959379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:11:52.959379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:52.959379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:11:52.959379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:52.959379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:11:52.959379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:52.959379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:11:52.959379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:52.959379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:11:52.959379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:11:52.960318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:52.960318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:11:52.960318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:52.960318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:11:52.960318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:11:52.960318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:52.960318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:11:52.960318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:52.960318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:11:53.416053 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:11:53.417192 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:11:53.417705 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:11:53.419172 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:11:53.419496 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:11:53.419717 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:53.423792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:11:53.423792 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:53.423792 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:11:53.423792 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:11:53.423792 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:53.423792 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:11:53.423792 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:53.423792 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:11:53.423792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:53.423792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:11:53.423792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:11:53.423792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:11:53.423792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:11:53.423792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:11:53.423792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:11:53.424224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:53.424224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:11:53.424224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:53.424224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:11:53.424224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:53.424224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:11:53.424224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:53.424224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:11:53.424224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:53.424224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:11:53.424224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:11:53.424224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:53.424224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:11:53.424224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:53.424569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:11:53.424569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:11:53.424569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:53.424569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:11:53.424569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:53.424569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:11:53.425146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:53.425146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:11:53.425146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:11:53.425146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:11:53.425146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:11:53.425146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:11:53.425146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:11:53.425146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:11:53.425146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:11:53.425146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:11:53.425146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:11:53.425146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:11:53.425146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:11:53.425146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:11:53.425146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:11:53.425146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:11:53.426268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:11:53.426268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:11:53.426268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:53.426268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:11:53.426268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:53.426268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:11:53.426268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:53.426268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:11:53.426268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:53.426268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:11:53.426268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:53.426268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:11:53.426268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:11:53.426268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:53.426268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:11:53.426268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:53.426268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:11:53.426268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:11:53.426268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:53.427440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:11:53.427440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:53.427440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:11:53.427904 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:11:53.428657 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:11:53.428657 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:11:53.428657 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:11:53.428944 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:11:53.429082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:53.431733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:11:53.431733 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:53.431733 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:11:53.431733 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:11:53.431733 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:53.431733 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:11:53.431733 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:53.431733 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:11:53.431733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:53.431733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:11:53.431733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:11:53.431733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:11:53.431733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:53.431733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:11:53.431733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:53.431733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:11:53.431733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:53.431733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:11:53.432643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:53.432643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:11:53.432643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:53.432643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:11:53.432643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:11:53.432643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:53.432643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:11:53.432643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:53.432643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:11:53.432643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:11:53.432643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:53.432643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:11:53.432643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:53.432643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:11:53.432643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:53.433362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:11:53.433362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:11:53.433362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:11:53.433362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:11:53.433362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:11:53.433362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:11:53.433362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:11:53.433362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:11:53.433362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:11:53.433362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:53.433362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:11:53.433362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:53.433362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:11:53.434016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:53.434016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:11:53.434016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:53.434016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:11:53.434016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:53.434016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:11:53.434016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:11:53.434016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:53.434016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:11:53.434016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:53.434016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:11:53.434016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:11:53.434016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:53.434016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:11:53.434628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:53.434628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:11:53.434628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:53.434628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:11:53.434628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:11:53.434628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:11:53.434628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:11:53.434628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:11:53.434628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:11:53.434628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:11:53.434628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:11:53.434628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:11:53.434628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:11:53.434628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:11:53.434628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:11:53.435334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:11:53.435334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:11:53.435334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:11:53.435334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:11:53.435334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:11:53.435334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:11:53.435334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:11:53.435334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:11:53.435334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:53.435334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:11:53.435334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:53.435334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:11:53.435334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:53.435334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:11:53.435334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:53.435334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:11:53.435334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:53.435334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:11:53.435334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:11:53.436196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:53.436196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:11:53.436196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:53.436196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:11:53.436196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:11:53.436196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:53.436196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:11:53.436196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:53.436196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:11:53.436196 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:11:53.436849 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:11:53.437037 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:11:53.437223 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:11:53.437596 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:11:53.437798 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:53.440194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:11:53.440194 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:53.440194 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:11:53.440194 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:11:53.440194 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:53.440194 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:11:53.440194 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:53.440194 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:11:53.440194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:53.440194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:11:53.440194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:11:53.440194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:11:53.440194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:53.440194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:11:53.440194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:53.440194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:11:53.440194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:53.440194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:11:53.440885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:53.440885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:11:53.440885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:53.440885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:11:53.440885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:11:53.440885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:53.440885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:11:53.440885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:53.440885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:11:53.440885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:11:53.440885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:53.440885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:11:53.440885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:53.440885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:11:53.440885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:53.441163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:11:53.441163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:11:53.441163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:11:53.441163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:11:53.441163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:11:53.441163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:11:53.441163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:11:53.441163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:11:53.441163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:53.441163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:11:53.441163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:53.441163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:11:53.441163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:53.441396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:11:53.441396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:53.441396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:11:53.441396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:53.441396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:11:53.441396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:11:53.441396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:53.441396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:11:53.441396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:53.441396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:11:53.441396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:11:53.441396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:53.441396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:11:53.441614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:53.441614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:11:53.441614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:53.441614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:11:53.441614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:11:53.441614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:11:53.441614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:11:53.441614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:11:53.441614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:11:53.441614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:11:53.441614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:11:53.441614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:11:53.441614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:11:53.441614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:11:53.441614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:11:53.441958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:11:53.441958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:11:53.441958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:11:53.441958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:11:53.441958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:11:53.441958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:11:53.441958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:11:53.441958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:53.441958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:11:53.441958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:53.441958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:11:53.441958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:53.441958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:11:53.441958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:53.441958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:11:53.441958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:53.441958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:11:53.441958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:11:53.441958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:53.442331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:11:53.442331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:53.442331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:11:53.442331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:11:53.442331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:53.442331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:11:53.442331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:53.442331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:11:53.450212 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:11:53.450624 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:11:53.450900 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:11:53.451191 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:11:53.451641 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:11:53.451829 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:11:53.451922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:53.454360 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:11:53.454360 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:53.454360 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:11:53.454360 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:11:53.454360 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:53.454360 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:11:53.454360 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:53.454360 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:11:53.454360 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:53.454360 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:11:53.454360 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:11:53.454360 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:11:53.454360 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:11:53.454360 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:11:53.454360 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:11:53.455357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:53.455357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:11:53.455357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:53.455357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:11:53.455357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:53.455357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:11:53.455357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:53.455357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:11:53.455357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:53.455357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:11:53.455357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:11:53.455357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:53.455357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:11:53.455357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:53.456258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:11:53.456258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:11:53.456258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:53.456258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:11:53.456258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:53.456258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:11:53.456258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:53.456258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:11:53.456258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:11:53.456258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:11:53.456258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:11:53.456258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:11:53.456258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:11:53.456258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:11:53.457160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:11:53.457160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:11:53.457160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:11:53.457160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:11:53.457160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:11:53.457160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:11:53.457160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:11:53.457160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:11:53.457160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:11:53.457160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:11:53.457160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:53.457160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:11:53.457160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:53.457160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:11:53.457160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:53.457160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:11:53.457160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:53.457160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:11:53.457160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:53.457160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:11:53.458010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:11:53.458010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:53.458010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:11:53.458010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:53.458010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:11:53.458010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:11:53.458010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:53.458010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:11:53.458010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:53.458010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:11:57.906213 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:11:57.907011 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:11:57.907011 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:11:57.908150 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:11:57.943297 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:11:57.943760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:57.944683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:11:57.944683 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:57.944683 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:11:57.944683 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:11:57.944683 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:57.944683 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:11:57.944683 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:57.944683 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:11:57.945508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:57.945508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:11:57.945508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:11:57.945508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:11:57.945508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:11:57.945508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:11:57.945508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:11:57.945508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:57.945508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:11:57.945508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:57.945508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:11:57.945508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:57.946148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:11:57.946148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:57.946148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:11:57.946148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:57.946148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:11:57.946148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:11:57.946148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:57.946148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:11:57.946148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:57.946148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:11:57.946148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:11:57.946148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:57.946148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:11:57.946788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:57.946788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:11:57.947280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:57.947280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:11:57.947280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:11:57.947280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:11:57.947280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:11:57.947280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:11:57.947280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:11:57.947280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:11:57.947280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:11:57.947280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:11:57.947280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:11:57.947280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:11:57.947280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:11:57.947280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:11:57.947280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:11:57.947280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:11:57.947963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:11:57.947963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:11:57.947963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:57.947963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:11:57.947963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:57.947963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:11:57.947963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:57.947963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:11:57.947963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:57.947963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:11:57.947963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:57.947963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:11:57.947963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:11:57.947963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:57.947963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:11:57.947963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:57.947963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:11:57.947963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:11:57.947963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:57.948463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:11:57.948463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:57.948463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:11:58.417019 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:11:58.417892 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:11:58.418156 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:11:58.419240 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:11:58.419626 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:11:58.419911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:58.420877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:11:58.420877 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:58.420877 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:11:58.420877 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:11:58.420877 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:58.420877 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:11:58.420877 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:58.420877 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:11:58.422492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:58.422492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:11:58.422492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:11:58.422492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:11:58.422492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:11:58.422492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:11:58.422492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:11:58.422492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:58.422492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:11:58.422492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:58.422492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:11:58.422492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:58.422975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:11:58.422975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:58.422975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:11:58.422975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:58.422975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:11:58.422975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:11:58.422975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:58.422975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:11:58.422975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:58.422975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:11:58.422975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:11:58.422975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:58.422975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:11:58.423319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:58.423319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:11:58.423874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:58.423874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:11:58.423874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:11:58.423874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:11:58.423874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:11:58.423874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:11:58.423874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:11:58.423874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:11:58.423874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:11:58.423874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:11:58.423874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:11:58.423874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:11:58.423874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:11:58.423874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:11:58.423874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:11:58.423874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:11:58.424301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:11:58.424301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:11:58.424301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:58.424301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:11:58.424301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:58.424301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:11:58.424301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:58.424301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:11:58.424301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:58.424301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:11:58.424301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:58.424301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:11:58.424301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:11:58.424301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:58.424301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:11:58.424301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:58.424301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:11:58.424301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:11:58.424301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:58.424795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:11:58.424795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:58.424795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:11:58.428435 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:11:58.428672 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:11:58.428812 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:11:58.428919 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:11:58.429289 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:11:58.429483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:58.430067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:11:58.430067 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:58.430067 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:11:58.430067 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:11:58.430067 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:58.430067 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:11:58.430067 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:58.430067 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:11:58.430837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:58.430837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:11:58.430837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:11:58.430837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:11:58.430837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:58.430837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:11:58.430837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:58.430837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:11:58.430837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:58.430837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:11:58.430837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:58.430837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:11:58.430837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:58.430837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:11:58.430837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:11:58.431241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:58.431241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:11:58.431241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:58.431241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:11:58.431241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:11:58.431241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:58.431241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:11:58.431241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:58.431241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:11:58.431641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:58.431641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:11:58.431641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:11:58.431641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:11:58.431641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:11:58.431641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:11:58.431641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:11:58.431641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:11:58.431641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:11:58.431641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:11:58.431641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:58.431641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:11:58.431972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:58.431972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:11:58.431972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:58.431972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:11:58.431972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:58.431972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:11:58.431972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:58.431972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:11:58.431972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:11:58.431972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:58.431972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:11:58.431972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:58.431972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:11:58.431972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:11:58.432309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:58.432309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:11:58.432309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:58.432309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:11:58.432706 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:58.432706 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:11:58.432706 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:11:58.432706 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:11:58.432706 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:11:58.432706 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:11:58.432706 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:11:58.432706 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:11:58.432706 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:11:58.432706 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:11:58.432706 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:11:58.432706 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:11:58.432706 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:11:58.432706 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:11:58.432706 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:11:58.432706 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:11:58.433266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:11:58.433266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:11:58.433266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:11:58.433266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:11:58.433266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:11:58.433266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:58.433266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:11:58.433266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:58.433266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:11:58.433266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:58.433266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:11:58.433266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:58.433266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:11:58.433266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:58.433266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:11:58.433266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:11:58.433266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:58.433266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:11:58.433725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:58.433725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:11:58.433725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:11:58.433725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:58.433725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:11:58.433725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:58.433725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:11:58.437003 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:11:58.437235 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:11:58.437347 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:11:58.437476 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:11:58.438104 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:11:58.438470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:58.439348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:11:58.439348 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:58.439348 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:11:58.439348 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:11:58.439348 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:58.439348 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:11:58.439348 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:58.439348 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:11:58.440098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:58.440098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:11:58.440098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:11:58.440098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:11:58.440098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:58.440098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:11:58.440098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:58.440098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:11:58.440098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:58.440098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:11:58.440098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:58.440098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:11:58.440098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:58.440098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:11:58.440098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:11:58.440600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:58.440600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:11:58.440600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:58.440600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:11:58.440600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:11:58.440600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:58.440600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:11:58.440600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:58.440600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:11:58.440961 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:58.440961 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:11:58.440961 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:11:58.440961 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:11:58.440961 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:11:58.440961 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:11:58.440961 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:11:58.440961 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:11:58.440961 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:11:58.440961 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:58.440961 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:11:58.441313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:58.441313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:11:58.441313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:58.441313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:11:58.441313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:58.441313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:11:58.441313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:58.441313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:11:58.441313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:11:58.441313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:58.441313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:11:58.441313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:58.441313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:11:58.441313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:11:58.441653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:58.441653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:11:58.441653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:58.441653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:11:58.442451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:58.442451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:11:58.442451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:11:58.442451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:11:58.442451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:11:58.442451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:11:58.442451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:11:58.442451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:11:58.442451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:11:58.442451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:11:58.442451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:11:58.442451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:11:58.442451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:11:58.442451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:11:58.442451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:11:58.442451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:11:58.442451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:11:58.443165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:11:58.443165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:11:58.443165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:11:58.443165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:58.443165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:11:58.443165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:58.443165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:11:58.443165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:58.443165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:11:58.443165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:58.443165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:11:58.443165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:58.443165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:11:58.443165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:11:58.443165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:58.443165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:11:58.443165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:58.443165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:11:58.443165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:11:58.443712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:58.443712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:11:58.443712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:58.443712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:11:58.450828 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:11:58.451032 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:11:58.451133 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:11:58.451295 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:11:58.451612 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:11:58.451866 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:11:58.452013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:58.452499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:11:58.452499 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:58.452499 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:11:58.452499 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:11:58.452499 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:58.452499 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:11:58.452499 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:58.452499 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:11:58.453333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:58.453333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:11:58.453333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:11:58.453333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:11:58.453333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:11:58.453333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:11:58.453333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:11:58.453333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:58.453333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:11:58.453333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:58.453333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:11:58.453333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:58.453691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:11:58.453691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:58.453691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:11:58.453691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:58.453691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:11:58.453691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:11:58.453691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:58.453691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:11:58.453691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:58.453691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:11:58.453691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:11:58.453691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:58.453691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:11:58.454071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:58.454071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:11:58.454341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:58.454341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:11:58.454341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:11:58.454341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:11:58.454341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:11:58.454341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:11:58.454341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:11:58.454341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:11:58.454341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:11:58.454341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:11:58.454341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:11:58.454341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:11:58.454341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:11:58.454341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:11:58.454341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:11:58.454341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:11:58.454792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:11:58.454792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:11:58.454792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:58.454792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:11:58.454792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:11:58.454792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:11:58.454792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:11:58.454792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:11:58.454792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:11:58.454792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:11:58.454792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:11:58.454792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:11:58.454792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:11:58.454792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:11:58.454792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:11:58.454792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:11:58.454792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:11:58.454792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:11:58.454792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:11:58.455260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:11:58.455260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:11:58.455260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:12:02.906662 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:12:02.907682 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:12:02.907682 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:12:02.908463 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:12:02.946901 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:12:02.946901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:02.947538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:12:02.947538 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:02.947538 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:12:02.947538 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:12:02.947538 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:02.947538 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:12:02.947538 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:02.947538 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:12:02.948324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:02.948324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:12:02.948324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:12:02.948324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:12:02.948324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:12:02.948324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:12:02.948324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:12:02.948324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:02.948324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:12:02.948324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:02.948324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:12:02.948324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:02.948860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:12:02.948860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:02.948860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:12:02.948860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:02.948860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:12:02.948860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:12:02.948860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:02.948860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:12:02.948860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:02.948860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:12:02.948860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:12:02.948860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:02.948860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:12:02.949351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:02.949351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:12:02.949351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:02.949351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:12:02.949351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:12:02.949351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:12:02.949351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:12:02.949351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:12:02.949351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:12:02.949351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:12:02.949351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:12:02.949351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:12:02.949351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:12:02.949351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:12:02.949351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:12:02.950034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:12:02.950034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:12:02.950034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:12:02.950034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:12:02.950034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:12:02.950034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:02.950034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:12:02.950034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:02.950034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:12:02.950034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:02.950034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:12:02.950034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:02.950034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:12:02.950034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:02.950034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:12:02.950034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:12:02.950034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:02.950034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:12:02.950034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:02.950494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:12:02.950494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:12:02.950494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:02.950494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:12:02.950494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:02.950494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:12:03.417574 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:12:03.418025 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:12:03.418206 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:12:03.419151 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:12:03.419520 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:12:03.419908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:03.420848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:12:03.420848 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:03.420848 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:12:03.420848 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:12:03.420848 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:03.420848 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:12:03.420848 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:03.420848 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:12:03.422197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:03.422197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:12:03.422197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:12:03.422197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:12:03.422197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:12:03.422197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:12:03.422197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:12:03.422197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:03.422197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:12:03.422197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:03.422197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:12:03.422197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:03.422785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:12:03.422785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:03.422785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:12:03.422785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:03.422785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:12:03.422785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:12:03.422785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:03.422785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:12:03.422785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:03.422785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:12:03.422785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:12:03.422785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:03.422785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:12:03.423435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:03.423435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:12:03.424342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:03.424342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:12:03.424342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:12:03.424342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:12:03.424342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:12:03.424342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:12:03.424342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:12:03.424342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:12:03.424342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:12:03.424342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:12:03.424342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:12:03.424342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:12:03.424342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:12:03.424342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:12:03.424342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:12:03.424342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:12:03.425858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:12:03.425858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:12:03.425858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:03.425858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:12:03.425858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:03.425858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:12:03.425858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:03.425858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:12:03.425858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:03.425858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:12:03.425858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:03.425858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:12:03.425858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:12:03.425858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:03.425858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:12:03.425858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:03.425858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:12:03.425858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:12:03.425858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:03.426669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:12:03.426669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:03.426669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:12:03.428681 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:12:03.428886 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:12:03.428974 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:12:03.429087 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:12:03.429415 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:12:03.429645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:03.430129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:12:03.430129 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:03.430129 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:12:03.430129 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:12:03.430129 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:03.430129 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:12:03.430129 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:03.430129 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:12:03.430851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:03.430851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:12:03.430851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:12:03.430851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:12:03.430851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:03.430851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:12:03.430851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:03.430851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:12:03.430851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:03.430851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:12:03.430851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:03.430851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:12:03.430851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:03.430851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:12:03.430851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:12:03.431297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:03.431297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:12:03.431297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:03.431297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:12:03.431297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:12:03.431297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:03.431297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:12:03.431297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:03.431297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:12:03.431674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:03.431674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:12:03.431674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:12:03.431674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:12:03.431674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:12:03.431674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:12:03.431674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:12:03.431674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:12:03.431674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:12:03.431674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:12:03.431674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:03.431674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:12:03.432523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:03.432523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:12:03.432523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:03.432523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:12:03.432523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:03.432523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:12:03.432523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:03.432523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:12:03.432523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:12:03.432523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:03.432523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:12:03.432523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:03.432523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:12:03.432523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:12:03.433117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:03.433117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:12:03.433117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:03.433117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:12:03.433117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:03.433117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:12:03.433117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:12:03.433117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:12:03.433117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:12:03.433117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:12:03.433117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:12:03.433117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:12:03.433117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:12:03.433117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:12:03.433462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:12:03.433462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:12:03.433462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:12:03.433462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:12:03.433462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:12:03.433462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:12:03.433462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:12:03.433462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:12:03.433462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:12:03.433462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:12:03.433462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:12:03.433462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:03.433462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:12:03.433462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:03.433462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:12:03.433462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:03.433462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:12:03.433462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:03.433462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:12:03.433949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:03.433949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:12:03.433949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:12:03.433949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:03.433949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:12:03.433949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:03.433949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:12:03.433949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:12:03.433949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:03.433949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:12:03.433949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:03.433949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:12:03.488726 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:12:03.488726 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:12:03.489689 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:12:03.489919 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:12:03.490098 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:12:03.491043 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:12:03.491363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:03.492332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:12:03.492332 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:03.492332 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:12:03.492332 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:12:03.492332 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:03.492332 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:12:03.492332 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:03.492332 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:12:03.494115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:03.494115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:12:03.494115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:12:03.494115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:12:03.494115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:03.494115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:12:03.494115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:03.494115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:12:03.494115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:03.494115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:12:03.494115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:03.494115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:12:03.494115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:03.494115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:12:03.494115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:12:03.494826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:03.494826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:12:03.494826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:03.494826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:12:03.494826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:12:03.494826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:03.494826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:12:03.494826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:03.494826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:12:03.495208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:03.495208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:12:03.495208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:12:03.495208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:12:03.495208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:12:03.495208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:12:03.495208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:12:03.495208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:12:03.495208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:12:03.495208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:03.495208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:12:03.495700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:03.495700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:12:03.495700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:03.495700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:12:03.495700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:03.495700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:12:03.495700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:03.495700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:12:03.495700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:12:03.495700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:03.495700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:12:03.495700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:03.495700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:12:03.495700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:12:03.496348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:03.496348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:12:03.496348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:03.496348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:12:03.497129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:03.497129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:12:03.497129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:12:03.497129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:12:03.497129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:12:03.497129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:12:03.497129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:12:03.497129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:12:03.497129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:12:03.497129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:12:03.497129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:12:03.497129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:12:03.497129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:12:03.497129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:12:03.497129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:12:03.497129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:12:03.497129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:12:03.498008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:12:03.498008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:12:03.498008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:12:03.498008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:03.498008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:12:03.498008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:03.498008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:12:03.498008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:03.498008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:12:03.498008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:03.498008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:12:03.498008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:03.498008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:12:03.498008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:12:03.498008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:03.498008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:12:03.498008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:03.498008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:12:03.498008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:12:03.498935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:03.498935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:12:03.498935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:03.498935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:12:03.498935 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:12:03.498935 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:12:03.498935 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:12:03.498935 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:12:03.499428 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:12:03.499428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:03.499428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:12:03.499428 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:03.499428 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:12:03.499428 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:12:03.499428 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:03.499428 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:12:03.499428 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:03.499428 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:12:03.499987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:03.499987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:12:03.499987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:12:03.499987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:12:03.499987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:12:03.499987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:12:03.499987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:12:03.499987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:03.499987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:12:03.499987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:03.499987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:12:03.499987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:03.500773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:12:03.500773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:03.500773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:12:03.500773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:03.500773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:12:03.500773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:12:03.500773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:03.500773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:12:03.500773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:03.500773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:12:03.500773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:12:03.500773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:03.500773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:12:03.501276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:03.501276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:12:03.501276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:03.501276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:12:03.501276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:12:03.501276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:12:03.501276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:12:03.501276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:12:03.501276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:12:03.501276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:12:03.501276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:12:03.501276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:12:03.501276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:12:03.501276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:12:03.501276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:12:03.501715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:12:03.501715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:12:03.501715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:12:03.501715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:12:03.501715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:12:03.501715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:03.501715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:12:03.501715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:03.501715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:12:03.501715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:03.501715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:12:03.501715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:03.501715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:12:03.501715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:03.501715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:12:03.501715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:12:03.501715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:03.501715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:12:03.501715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:03.502314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:12:03.502314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:12:03.502314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:03.502314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:12:03.502314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:03.502314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:12:08.025137 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:12:08.025137 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:12:08.025137 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:12:08.025137 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:12:08.025137 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:12:08.025137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:08.027903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:12:08.027903 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:08.027903 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:12:08.027903 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:12:08.027903 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:08.027903 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:12:08.027903 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:08.027903 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:12:08.027903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.050 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:08.027903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:12:08.027903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:12:08.027903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:12:08.027903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:12:08.027903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:12:08.027903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:12:08.028680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:08.028680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:12:08.028680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:08.028680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:12:08.028680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:08.028680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:12:08.028680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:08.028680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:12:08.028680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:08.028680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:12:08.028680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:12:08.028680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:08.028680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:12:08.028680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:08.029095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:12:08.029095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:12:08.029095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:08.029095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:12:08.029095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:08.029095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:12:08.029095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:08.029095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:12:08.029095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:12:08.029095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:12:08.029095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:12:08.029095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:12:08.029095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:12:08.029095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:12:08.029511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:12:08.029511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:12:08.029511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:12:08.029511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:12:08.029511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:12:08.029511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:12:08.029511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:12:08.029511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:12:08.029511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:12:08.029511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:12:08.029511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:08.029511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:12:08.029511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:08.029511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:12:08.029511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:08.029511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:12:08.029511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:08.029511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:12:08.029511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:08.029511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:12:08.029511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:12:08.030243 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:08.030243 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:12:08.030243 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:08.030243 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:12:08.030243 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:12:08.030243 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:08.030243 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:12:08.030243 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:08.030243 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:12:08.418348 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:12:08.418702 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:12:08.419020 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:12:08.420213 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:12:08.420564 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:12:08.420918 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:08.421770 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:12:08.421770 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:08.421770 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:12:08.421770 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:12:08.421770 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:08.421770 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:12:08.421770 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:08.421770 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:12:08.423712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:08.423712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:12:08.423712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:12:08.423712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:12:08.423712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:12:08.423712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:12:08.423712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:12:08.423712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:08.423712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:12:08.423712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:08.423712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:12:08.423712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:08.424378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:12:08.424378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:08.424378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:12:08.424378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:08.424378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:12:08.424378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:12:08.424378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:08.424378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:12:08.424378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:08.424378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:12:08.424378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:12:08.424378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:08.424378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:12:08.425294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:08.425294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:12:08.426242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:08.426242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:12:08.426242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:12:08.426242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:12:08.426242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:12:08.426242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:12:08.426242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:12:08.426242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:12:08.426242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:12:08.426242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:12:08.426242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:12:08.426242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:12:08.426242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:12:08.426242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:12:08.426242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:12:08.426242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:12:08.427071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:12:08.427071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:12:08.427071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:08.427071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:12:08.427071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:08.427071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:12:08.427071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:08.427071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:12:08.427071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:08.427071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:12:08.427071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:08.427071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:12:08.427071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:12:08.427071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:08.427071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:12:08.427071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:08.427071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:12:08.427071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:12:08.427071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:08.427923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:12:08.427923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:08.427923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:12:08.429105 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:12:08.429442 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:12:08.429587 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:12:08.429751 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:12:08.430262 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:12:08.430473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:08.431038 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:12:08.431038 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:08.431038 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:12:08.431038 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:12:08.431038 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:08.431038 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:12:08.431038 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:08.431038 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:12:08.431928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:08.431928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:12:08.431928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:12:08.431928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:12:08.431928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:08.431928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:12:08.431928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:08.431928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:12:08.431928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:08.431928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:12:08.431928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:08.431928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:12:08.431928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:08.431928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:12:08.431928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:12:08.432325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:08.432325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:12:08.432325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:08.432325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:12:08.432325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:12:08.432325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:08.432325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:12:08.432325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:08.432325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:12:08.433098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:08.433098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:12:08.433098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:12:08.433098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:12:08.433098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:12:08.433098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:12:08.433098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:12:08.433098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:12:08.433098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:12:08.433098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:12:08.433098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:08.433098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:12:08.433425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:08.433425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:12:08.433425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:08.433425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:12:08.433425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:08.433425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:12:08.433425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:08.433425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:12:08.433425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:12:08.433425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:08.433425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:12:08.433425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:08.433425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:12:08.433425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:12:08.433841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:08.433841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:12:08.433841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:08.433841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:12:08.434428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:08.434428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:12:08.434428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:12:08.434428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:12:08.434428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:12:08.434428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:12:08.434428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:12:08.434428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:12:08.434428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:12:08.434428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:12:08.434428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:12:08.434428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:12:08.434428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:12:08.434428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:12:08.434428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:12:08.434428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:12:08.434861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:12:08.434861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:12:08.434861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:12:08.434861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:12:08.434861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:12:08.434861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:08.434861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:12:08.434861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:08.434861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:12:08.434861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:08.434861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:12:08.434861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:08.434861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:12:08.434861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:08.434861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:12:08.434861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:12:08.434861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:08.434861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:12:08.435295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:08.435295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:12:08.435295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:12:08.435295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:08.435295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:12:08.435295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:08.435295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:12:08.490015 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:12:08.490529 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:12:08.490736 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:12:08.491001 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:12:08.491606 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:12:08.492138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:08.493125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:12:08.493125 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:08.493125 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:12:08.493125 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:12:08.493125 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:08.493125 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:12:08.493125 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:08.493125 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:12:08.494457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:08.494457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:12:08.494457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:12:08.494457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:12:08.494457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:08.494457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:12:08.494457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:08.494457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:12:08.494457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:08.494457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:12:08.494457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:08.494457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:12:08.494457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:08.494457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:12:08.494457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:12:08.495289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:08.495289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:12:08.495289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:08.495289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:12:08.495289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:12:08.495289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:08.495289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:12:08.495289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:08.495289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:12:08.496216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:08.496216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:12:08.496216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:12:08.496216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:12:08.496216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:12:08.496216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:12:08.496216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:12:08.496216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:12:08.496216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:12:08.496216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:08.496216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:12:08.496522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:08.496522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:12:08.496522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:08.496522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:12:08.496522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:08.496522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:12:08.496522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:08.496522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:12:08.496522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:12:08.496522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:08.496522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:12:08.496522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:08.496522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:12:08.496522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:12:08.496969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:08.496969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:12:08.496969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:08.496969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:12:08.498494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:08.498494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:12:08.498494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:12:08.498494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:12:08.498494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:12:08.498494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:12:08.498494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:12:08.498494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:12:08.498494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:12:08.498494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:12:08.498494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:12:08.498494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:12:08.498494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:12:08.498494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:12:08.498494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:12:08.498494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:12:08.498494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:12:08.498980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:12:08.498980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:12:08.498980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:12:08.498980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:08.498980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:12:08.498980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:08.498980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:12:08.498980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:08.498980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:12:08.498980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:08.498980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:12:08.498980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:08.498980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:12:08.498980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:12:08.498980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:08.498980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:12:08.498980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:08.498980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:12:08.498980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:12:08.499468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:08.499468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:12:08.499468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:08.499468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:12:08.499468 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:12:08.499468 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:12:08.499468 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:12:08.499690 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:12:08.500023 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:12:08.500301 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:12:08.500494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:08.501154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:12:08.501154 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:08.501154 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:12:08.501154 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:12:08.501154 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:08.501154 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:12:08.501154 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:08.501154 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:12:08.502114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:08.502114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:12:08.502114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:12:08.502114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:12:08.502114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:12:08.502114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:12:08.502114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:12:08.502114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:08.502114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:12:08.502114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:08.502114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:12:08.502114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:08.502450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:12:08.502450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:08.502450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:12:08.502450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:08.502450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:12:08.502450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:12:08.502450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:08.502450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:12:08.502450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:08.502450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:12:08.502450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:12:08.502450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:08.502450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:12:08.502781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:08.502781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:12:08.503342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:08.503342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:12:08.503342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:12:08.503342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:12:08.503342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:12:08.503342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:12:08.503342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:12:08.503342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:12:08.503342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:12:08.503342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:12:08.503342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:12:08.503342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:12:08.503342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:12:08.503342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:12:08.503342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:12:08.503342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:12:08.503835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:12:08.503835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:12:08.503835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:08.503835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:12:08.503835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:08.503835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:12:08.503835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:08.503835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:12:08.503835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:08.503835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:12:08.503835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:08.503835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:12:08.503835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:12:08.503835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:08.503835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:12:08.503835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:08.503835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:12:08.503835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:12:08.503835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:08.504292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:12:08.504292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:08.504292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:12:12.908406 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:12:12.909436 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:12:12.909436 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:12:12.910506 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:12:12.942166 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:12:12.942675 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:12.943730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:12:12.943730 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:12.943730 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:12:12.943730 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:12:12.943730 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:12.943730 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:12:12.943730 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:12.943730 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:12:12.944665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:12.944665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:12:12.944665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:12:12.944665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:12:12.944665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:12:12.944665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:12:12.944665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:12:12.944665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:12.944665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:12:12.944665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:12.944665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:12:12.944665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:12.945150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:12:12.945150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:12.945150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:12:12.945150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:12.945150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:12:12.945150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:12:12.945150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:12.945150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:12:12.945150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:12.945150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:12:12.945150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:12:12.945150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:12.945150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:12:12.945602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:12.945602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:12:12.945602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:12.945602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:12:12.945602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:12:12.945602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:12:12.945602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:12:12.945602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:12:12.945602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:12:12.945602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:12:12.945602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:12:12.945602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:12:12.945602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:12:12.945602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:12:12.945602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:12:12.946099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:12:12.946099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:12:12.946099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:12:12.946099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:12:12.946099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:12:12.946099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:12.946099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:12:12.946099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:12.946099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:12:12.946099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:12.946099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:12:12.946099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:12.946099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:12:12.946099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:12.946099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:12:12.946099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:12:12.946099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:12.946099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:12:12.946099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:12.946595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:12:12.946595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:12:12.946595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:12.946595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:12:12.946595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:12.946595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:12:13.418958 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:12:13.420524 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:12:13.420652 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:12:13.421932 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:12:13.422363 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:12:13.422599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:13.423586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:12:13.423586 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:13.423586 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:12:13.423586 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:12:13.423586 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:13.423586 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:12:13.423586 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:13.423586 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:12:13.424963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:13.424963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:12:13.424963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:12:13.424963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:12:13.424963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:12:13.424963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:12:13.424963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:12:13.424963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:13.424963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:12:13.424963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:13.424963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:12:13.424963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:13.425362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:12:13.425362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:13.425362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:12:13.425362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:13.425362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:12:13.425362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:12:13.425362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:13.425362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:12:13.425362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:13.425362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:12:13.425362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:12:13.425362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:13.425362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:12:13.425735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:13.425735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:12:13.426095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:13.426095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:12:13.426095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:12:13.426095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:12:13.426095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:12:13.426095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:12:13.426095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:12:13.426095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:12:13.426095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:12:13.426095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:12:13.426095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:12:13.426095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:12:13.426095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:12:13.426095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:12:13.426095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:12:13.426095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:12:13.426558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:12:13.426558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:12:13.426558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:13.426558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:12:13.426558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:13.426558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:12:13.426558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:13.426558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:12:13.426558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:13.426558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:12:13.426558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:13.426558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:12:13.426558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:12:13.426558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:13.426558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:12:13.426558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:13.426558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:12:13.426558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:12:13.426558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:13.427127 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:12:13.427127 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:13.427127 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:12:13.429610 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:12:13.429895 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:12:13.430013 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:12:13.430308 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:12:13.430527 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:12:13.430794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:13.431373 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:12:13.431373 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:13.431373 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:12:13.431373 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:12:13.431373 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:13.431373 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:12:13.431373 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:13.431373 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:12:13.432212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:13.432212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:12:13.432212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:12:13.432212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:12:13.432212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:13.432212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:12:13.432212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:13.432212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:12:13.432212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:13.432212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:12:13.432212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:13.432212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:12:13.432212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:13.432212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:12:13.432212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:12:13.432659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:13.432659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:12:13.432659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:13.432659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:12:13.432659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:12:13.432659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:13.432659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:12:13.432659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:13.432659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:12:13.433062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:13.433062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:12:13.433062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:12:13.433062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:12:13.433062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:12:13.433062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:12:13.433062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:12:13.433062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:12:13.433062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:12:13.433062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:12:13.433062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:13.433062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:12:13.433456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:13.433456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:12:13.433456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:13.433456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:12:13.433456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:13.433456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:12:13.433456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:13.433456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:12:13.433456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:12:13.433456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:13.433456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:12:13.433456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:13.433456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:12:13.433456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:12:13.433875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:13.433875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:12:13.433875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:13.433875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:12:13.434126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:13.434126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:12:13.434126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:12:13.434126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:12:13.434126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:12:13.434126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:12:13.434126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:12:13.434126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:12:13.434126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:12:13.434126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:12:13.434126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:12:13.434126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:12:13.434126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:12:13.434126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:12:13.434126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:12:13.434126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:12:13.434687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:12:13.434687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:12:13.434687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:12:13.434687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:12:13.434687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:12:13.434687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:13.434687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:12:13.434687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:13.434687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:12:13.434687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:13.434687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:12:13.434687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:13.434687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:12:13.434687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:13.434687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:12:13.434687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:12:13.434687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:13.434687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:12:13.435183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:13.435183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:12:13.435183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:12:13.435183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:13.435183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:12:13.435183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:13.435183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:12:13.490983 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:12:13.491716 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:12:13.491882 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:12:13.492077 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:12:13.492655 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:12:13.493051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:13.493828 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:12:13.493828 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:13.493828 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:12:13.493828 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:12:13.493828 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:13.493828 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:12:13.493828 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:13.493828 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:12:13.494954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:13.494954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:12:13.494954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:12:13.494954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:12:13.494954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:13.494954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:12:13.494954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:13.494954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:12:13.494954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:13.494954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:12:13.494954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:13.494954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:12:13.494954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:13.494954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:12:13.494954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:12:13.495403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:13.495403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:12:13.495403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:13.495403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:12:13.495403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:12:13.495403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:13.495403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:12:13.495403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:13.495403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:12:13.495907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:13.495907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:12:13.495907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:12:13.495907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:12:13.495907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:12:13.495907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:12:13.495907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:12:13.495907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:12:13.495907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:12:13.495907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:13.495907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:12:13.496279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:13.496279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:12:13.496279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:13.496279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:12:13.496279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:13.496279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:12:13.496279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:13.496279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:12:13.496279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:12:13.496279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:13.496279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:12:13.496279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:13.496279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:12:13.496279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:12:13.496666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:13.496666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:12:13.496666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:13.496666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:12:13.497076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:13.497076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:12:13.497076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:12:13.497076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:12:13.497076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:12:13.497076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:12:13.497076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:12:13.497076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:12:13.497076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:12:13.497076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:12:13.497076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:12:13.497076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:12:13.497076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:12:13.497076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:12:13.497076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:12:13.497076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:12:13.497076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:12:13.497583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:12:13.497583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:12:13.497583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:12:13.497583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:13.497583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:12:13.497583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:13.497583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:12:13.497583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:13.497583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:12:13.497583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:13.497583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:12:13.497583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:13.497583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:12:13.497583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:12:13.497583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:13.497583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:12:13.497583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:13.497583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:12:13.497583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:12:13.498179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:13.498179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:12:13.498179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:13.498179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:12:13.499131 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:12:13.499731 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:12:13.499731 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:12:13.499731 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:12:13.500129 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:12:13.500596 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:12:13.500854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:13.501775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:12:13.501775 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:13.501775 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:12:13.501775 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:12:13.501775 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:13.501775 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:12:13.501775 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:13.501775 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:12:13.502665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:13.502665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:12:13.502665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:12:13.502665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:12:13.502665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:12:13.502665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:12:13.502665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:12:13.502665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:13.502665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:12:13.502665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:13.502665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:12:13.502665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:13.503237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:12:13.503237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:13.503237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:12:13.503237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:13.503237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:12:13.503237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:12:13.503237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:13.503237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:12:13.503237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:13.503237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:12:13.503237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:12:13.503237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:13.503237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:12:13.503837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:13.503837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:12:13.503837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:13.503837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:12:13.503837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:12:13.503837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:12:13.503837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:12:13.503837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:12:13.503837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:12:13.503837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:12:13.503837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:12:13.503837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:12:13.503837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:12:13.503837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:12:13.503837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:12:13.504446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:12:13.504446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:12:13.504446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:12:13.504446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:12:13.504446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:12:13.504446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:13.504446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:12:13.504446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:13.504446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:12:13.504446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:13.504446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:12:13.504446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:13.504446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:12:13.504446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:13.504446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:12:13.504446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:12:13.504446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:13.504446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:12:13.504446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:13.505200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:12:13.505200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:12:13.505200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:13.505200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:12:13.505200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:13.505200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:12:17.908985 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:12:17.909955 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:12:17.909955 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:12:17.911044 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:12:17.950917 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:12:17.951404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:17.952565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:12:17.952565 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:17.952565 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:12:17.952565 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:12:17.952565 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:17.952565 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:12:17.952565 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:17.952565 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:12:17.954643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.043 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:17.954643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:12:17.954643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:12:17.954643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:12:17.954643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:12:17.954643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:12:17.954643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:12:17.954643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:17.954643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:12:17.954643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:17.954643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:12:17.954643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:17.955106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:12:17.955106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:17.955106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:12:17.955106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:17.955106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:12:17.955106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:12:17.955106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:17.955106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:12:17.955106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:17.955106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:12:17.955106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:12:17.955106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:17.955106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:12:17.955472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:17.955472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:12:17.955472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:17.955472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:12:17.955472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:12:17.955472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:12:17.955472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:12:17.955472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:12:17.955472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:12:17.955472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:12:17.955472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:12:17.955472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:12:17.955472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:12:17.955472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:12:17.955472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:12:17.956103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:12:17.956103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:12:17.956103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:12:17.956103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:12:17.956103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:12:17.956103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:17.956103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:12:17.956103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:17.956103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:12:17.956103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:17.956103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:12:17.956103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:17.956103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:12:17.956103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:17.956103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:12:17.956103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:12:17.956103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:17.956103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:12:17.956103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:17.956825 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:12:17.956825 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:12:17.956825 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:17.956825 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:12:17.956825 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:17.956825 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:12:18.420406 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:12:18.420701 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:12:18.420881 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:12:18.424411 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:12:18.424666 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:12:18.424918 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:18.425839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:12:18.425839 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:18.425839 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:12:18.425839 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:12:18.425839 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:18.425839 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:12:18.425839 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:18.425839 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:12:18.427970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:18.427970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:12:18.427970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:12:18.427970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:12:18.427970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:12:18.427970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:12:18.427970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:12:18.427970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:18.427970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:12:18.427970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:18.427970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:12:18.427970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:18.428615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:12:18.428615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:18.428615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:12:18.428615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:18.428615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:12:18.428615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:12:18.428615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:18.428615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:12:18.428615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:18.428615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:12:18.428615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:12:18.428615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:18.428615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:12:18.429385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:18.429385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:12:18.430771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:18.430771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:12:18.430771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:12:18.430771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:12:18.430771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:12:18.430771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:12:18.430771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:12:18.430771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:12:18.430771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:12:18.430771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:12:18.430771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:12:18.430771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:12:18.430771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:12:18.430771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:12:18.430771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:12:18.430771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:12:18.431595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:12:18.431595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:12:18.431595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:18.431595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:12:18.431595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:18.431595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:12:18.431595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:18.431595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:12:18.431595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:18.431595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:12:18.431595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:18.431595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:12:18.431595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:12:18.431595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:18.431595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:12:18.431595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:18.431595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:12:18.431595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:12:18.431595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:18.432447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:12:18.432447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:18.432447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:12:18.432447 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:12:18.432447 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:12:18.432447 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:12:18.432447 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:12:18.432818 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:12:18.432818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:18.433418 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:12:18.433418 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:18.433418 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:12:18.433418 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:12:18.433418 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:18.433418 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:12:18.433418 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:18.433418 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:12:18.434026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:18.434026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:12:18.434026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:12:18.434026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:12:18.434026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:18.434026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:12:18.434026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:18.434026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:12:18.434026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:18.434026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:12:18.434026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:18.434026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:12:18.434026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:18.434026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:12:18.434026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:12:18.434723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:18.434723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:12:18.434723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:18.434723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:12:18.434723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:12:18.434723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:18.434723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:12:18.434723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:18.434723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:12:18.434723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:18.434723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:12:18.434723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:12:18.434723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:12:18.434723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:12:18.435376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:12:18.435376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:12:18.435376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:12:18.435376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:12:18.435376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:12:18.435376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:18.435376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:12:18.435376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:18.435376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:12:18.435376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:18.435376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:12:18.435376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:18.435376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:12:18.435964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:18.435964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:12:18.435964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:12:18.435964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:18.435964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:12:18.435964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:18.435964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:12:18.435964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:12:18.435964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:18.435964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:12:18.435964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:18.435964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:12:18.436497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:18.436497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:12:18.436497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:12:18.436497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:12:18.436497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:12:18.436497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:12:18.436497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:12:18.436497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:12:18.436497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:12:18.436497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:12:18.436497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:12:18.436497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:12:18.436497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:12:18.436497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:12:18.436497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:12:18.436497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:12:18.437270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:12:18.437270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:12:18.437270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:12:18.437270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:12:18.437270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:12:18.437270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:18.437270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:12:18.437270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:18.437270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:12:18.437270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:18.437270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:12:18.437270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:18.437270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:12:18.437270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:18.437270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:12:18.437270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:12:18.437270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:18.437270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:12:18.437881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:18.437881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:12:18.437881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:12:18.437881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:18.437881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:12:18.437881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:18.437881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:12:18.492295 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:12:18.492829 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:12:18.493086 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:12:18.493365 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:12:18.494348 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:12:18.494776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:18.495732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:12:18.495732 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:18.495732 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:12:18.495732 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:12:18.495732 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:18.495732 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:12:18.495732 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:18.495732 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:12:18.497387 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:18.497387 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:12:18.497387 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:12:18.497387 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:12:18.497387 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:18.497387 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:12:18.497387 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:18.497387 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:12:18.497387 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:18.497387 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:12:18.497387 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:18.497387 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:12:18.497387 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:18.497387 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:12:18.497387 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:12:18.497878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:18.497878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:12:18.497878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:18.497878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:12:18.497878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:12:18.497878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:18.497878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:12:18.497878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:18.497878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:12:18.499034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:18.499034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:12:18.499034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:12:18.499034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:12:18.499034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:12:18.499034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:12:18.499034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:12:18.499034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:12:18.499034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:12:18.499034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:18.499034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:12:18.499396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:18.499396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:12:18.499396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:18.499396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:12:18.499396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:18.499396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:12:18.499396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:18.499396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:12:18.499396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:12:18.499396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:18.499396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:12:18.499396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:18.499396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:12:18.499396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:12:18.499817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:18.499817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:12:18.499817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:18.499817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:12:18.501004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:18.501004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:12:18.501004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:12:18.501004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:12:18.501004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:12:18.501004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:12:18.501004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:12:18.501004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:12:18.501004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:12:18.501004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:12:18.501004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:12:18.501004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:12:18.501004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:12:18.501004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:12:18.501004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:12:18.501004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:12:18.501004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:12:18.501511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:12:18.501511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:12:18.501511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:12:18.501511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:18.501511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:12:18.501511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:18.501511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:12:18.501511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:18.501511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:12:18.501511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:18.501511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:12:18.501511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:18.501511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:12:18.501511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:12:18.501511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:18.501511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:12:18.501511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:18.501511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:12:18.501511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:12:18.502144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:18.502144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:12:18.502144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:18.502144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:12:18.502144 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:12:18.502144 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:12:18.502144 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:12:18.502144 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:12:18.502411 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:12:18.502411 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:12:18.502498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:18.503072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:12:18.503072 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:18.503072 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:12:18.503072 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:12:18.503072 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:18.503072 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:12:18.503072 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:18.503072 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:12:18.504013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:18.504013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:12:18.504013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:12:18.504013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:12:18.504013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:12:18.504013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:12:18.504013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:12:18.504013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:18.504013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:12:18.504013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:18.504013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:12:18.504013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:18.504423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:12:18.504423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:18.504423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:12:18.504423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:18.504423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:12:18.504423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:12:18.504423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:18.504423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:12:18.504423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:18.504423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:12:18.504423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:12:18.504423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:18.504423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:12:18.504898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:18.504898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:12:18.505089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:18.505089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:12:18.505089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:12:18.505089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:12:18.505089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:12:18.505089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:12:18.505089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:12:18.505089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:12:18.505089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:12:18.505089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:12:18.505089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:12:18.505089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:12:18.505089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:12:18.505089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:12:18.505089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:12:18.505089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:12:18.505550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:12:18.505550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:12:18.505550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:18.505550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:12:18.505550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:18.505550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:12:18.505550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:18.505550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:12:18.505550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:18.505550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:12:18.505550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:18.505550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:12:18.505550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:12:18.505550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:18.505550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:12:18.505550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:18.505550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:12:18.505550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:12:18.505550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:18.506111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:12:18.506111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:18.506111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:12:22.909584 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:12:22.910646 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:12:22.910904 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:12:22.912412 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:12:22.950880 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:12:22.951332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:22.952583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:12:22.952583 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:22.952583 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:12:22.952583 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:12:22.952583 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:22.952583 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:12:22.952583 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:22.952583 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:12:22.953637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:22.953637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:12:22.953637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:12:22.953637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:12:22.953637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:12:22.953637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:12:22.953637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:12:22.953637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:22.953637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:12:22.953637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:22.953637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:12:22.953637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:22.954274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:12:22.954274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:22.954274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:12:22.954274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:22.954274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:12:22.954274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:12:22.954274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:22.954274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:12:22.954274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:22.954274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:12:22.954274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:12:22.954274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:22.954274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:12:22.954775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:22.954775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:12:22.955082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:22.955082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:12:22.955082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:12:22.955082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:12:22.955082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:12:22.955082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:12:22.955082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:12:22.955082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:12:22.955082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:12:22.955082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:12:22.955082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:12:22.955082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:12:22.955082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:12:22.955082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:12:22.955082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:12:22.955082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:12:22.955889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:12:22.955889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:12:22.955889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:22.955889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:12:22.955889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:22.955889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:12:22.955889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:22.955889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:12:22.955889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:22.955889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:12:22.955889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:22.955889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:12:22.955889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:12:22.955889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:22.955889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:12:22.955889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:22.955889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:12:22.955889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:12:22.955889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:22.956472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:12:22.956472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:22.956472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:12:23.421076 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:12:23.422212 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:12:23.422643 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:12:23.424082 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:12:23.424446 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:12:23.424690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:23.425606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:12:23.425606 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:23.425606 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:12:23.425606 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:12:23.425606 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:23.425606 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:12:23.425606 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:23.425606 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:12:23.426897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:23.426897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:12:23.426897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:12:23.426897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:12:23.426897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:12:23.426897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:12:23.426897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:12:23.426897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:23.426897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:12:23.426897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:23.426897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:12:23.426897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:23.427687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:12:23.427687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:23.427687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:12:23.427687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:23.427687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:12:23.427687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:12:23.427687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:23.427687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:12:23.427687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:23.427687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:12:23.427687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:12:23.427687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:23.427687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:12:23.428317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:23.428317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:12:23.428317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:23.428317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:12:23.428317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:12:23.428317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:12:23.428317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:12:23.428317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:12:23.428317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:12:23.428317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:12:23.428317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:12:23.428317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:12:23.428317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:12:23.428317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:12:23.428317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:12:23.429055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:12:23.429055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:12:23.429055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:12:23.429055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:12:23.429055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:12:23.429055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:23.429055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:12:23.429055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:23.429055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:12:23.429055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:23.429055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:12:23.429055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:23.429055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:12:23.429055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:23.429055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:12:23.429055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:12:23.429055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:23.429055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:12:23.429055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:23.430221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:12:23.430221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:12:23.430221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:23.430221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:12:23.430221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:23.430221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:12:23.431549 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:12:23.432121 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:12:23.432237 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:12:23.432371 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:12:23.433559 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:12:23.434047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:23.435205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:12:23.435205 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:23.435205 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:12:23.435205 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:12:23.435205 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:23.435205 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:12:23.435205 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:23.435205 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:12:23.436250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:23.436250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:12:23.436250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:12:23.436250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:12:23.436250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:23.436250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:12:23.436250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:23.436250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:12:23.436250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:23.436250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:12:23.436250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:23.436250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:12:23.436250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:23.436250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:12:23.436250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:12:23.436666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:23.436666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:12:23.436666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:23.436666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:12:23.436666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:12:23.436666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:23.436666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:12:23.436666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:23.436666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:12:23.437125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:23.437125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:12:23.437125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:12:23.437125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:12:23.437125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:12:23.437125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:12:23.437125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:12:23.437125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:12:23.437125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:12:23.437125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:12:23.437125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:23.437125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:12:23.437509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:23.437509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:12:23.437509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:23.437509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:12:23.437509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:23.437509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:12:23.437509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:23.437509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:12:23.437509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:12:23.437509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:23.437509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:12:23.437509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:23.437509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:12:23.437509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:12:23.437937 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:23.437937 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:12:23.437937 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:23.437937 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:12:23.438219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:23.438219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:12:23.438219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:12:23.438219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:12:23.438219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:12:23.438219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:12:23.438219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:12:23.438219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:12:23.438219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:12:23.438219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:12:23.438219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:12:23.438219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:12:23.438219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:12:23.438219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:12:23.438219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:12:23.438219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:12:23.438702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:12:23.438702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:12:23.438702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:12:23.438702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:12:23.438702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:12:23.438702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:23.438702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:12:23.438702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:23.438702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:12:23.438702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:23.438702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:12:23.438702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:23.438702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:12:23.438702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:23.438702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:12:23.438702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:12:23.438702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:23.438702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:12:23.439288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:23.439288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:12:23.439288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:12:23.439288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:23.439288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:12:23.439288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:23.439288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:12:23.493014 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:12:23.493669 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:12:23.493902 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:12:23.494463 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:12:23.495371 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:12:23.495858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:23.497016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:12:23.497016 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:23.497016 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:12:23.497016 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:12:23.497016 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:23.497016 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:12:23.497016 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:23.497016 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:12:23.498901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:23.498901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:12:23.498901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:12:23.498901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:12:23.498901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:23.498901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:12:23.498901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:23.498901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:12:23.498901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:23.498901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:12:23.498901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:23.498901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:12:23.498901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:23.498901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:12:23.498901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:12:23.499606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:23.499606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:12:23.499606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:23.499606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:12:23.499606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:12:23.499606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:23.499606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:12:23.499606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:23.499606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:12:23.501100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:23.501100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:12:23.501100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:12:23.501100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:12:23.501100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:12:23.501100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:12:23.501100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:12:23.501100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:12:23.501100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:12:23.501100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:23.501100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:12:23.501442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:23.501442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:12:23.501442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:23.501442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:12:23.501442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:23.501442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:12:23.501442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:23.501442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:12:23.501442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:12:23.501442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:23.501442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:12:23.501442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:23.501442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:12:23.501442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:12:23.501894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:23.501894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:12:23.501894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:23.501894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:12:23.502558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:23.502558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:12:23.502558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:12:23.502558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:12:23.502558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:12:23.502558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:12:23.502558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:12:23.502558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:12:23.502558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:12:23.502558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:12:23.502558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:12:23.502558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:12:23.502558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:12:23.502558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:12:23.502558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:12:23.502558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:12:23.502558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:12:23.503167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:12:23.503167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:12:23.503167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:12:23.503167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:23.503167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:12:23.503167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:23.503167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:12:23.503167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:23.503167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:12:23.503167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:23.503167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:12:23.503167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:23.503167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:12:23.503167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:12:23.503167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:23.503167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:12:23.503167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:23.503167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:12:23.503167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:12:23.503775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:23.503775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:12:23.503775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:23.503775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:12:23.503775 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:12:23.503775 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:12:23.503775 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:12:23.503775 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:12:23.504043 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:12:23.504312 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:12:23.504475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:23.505145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:12:23.505145 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:23.505145 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:12:23.505145 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:12:23.505145 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:23.505145 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:12:23.505145 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:23.505145 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:12:23.506291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:23.506291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:12:23.506291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:12:23.506291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:12:23.506291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:12:23.506291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:12:23.506291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:12:23.506291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:23.506291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:12:23.506291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:23.506291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:12:23.506291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:23.506667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:12:23.506667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:23.506667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:12:23.506667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:23.506667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:12:23.506667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:12:23.506667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:23.506667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:12:23.506667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:23.506667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:12:23.506667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:12:23.506667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:23.506667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:12:23.507058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:23.507058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:12:23.507720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:23.507720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:12:23.507720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:12:23.507720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:12:23.507720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:12:23.507720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:12:23.507720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:12:23.507720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:12:23.507720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:12:23.507720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:12:23.507720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:12:23.507720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:12:23.507720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:12:23.507720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:12:23.507720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:12:23.507720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:12:23.508224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:12:23.508224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:12:23.508224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:23.508224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:12:23.508224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:23.508224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:12:23.508224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:23.508224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:12:23.508224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:23.508224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:12:23.508224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:23.508224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:12:23.508224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:12:23.508224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:23.508224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:12:23.508224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:23.508224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:12:23.508224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:12:23.508224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:23.508794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:12:23.508794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:23.508794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:12:27.911058 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:12:27.911896 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:12:27.911896 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:12:27.913315 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:12:27.944664 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:12:27.945265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:27.946406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:12:27.946406 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:27.946406 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:12:27.946406 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:12:27.946406 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:27.946406 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:12:27.946406 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:27.946406 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:12:27.947298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:27.947298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:12:27.947298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:12:27.947298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:12:27.947298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:12:27.947298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:12:27.947298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:12:27.947298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:27.947298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:12:27.947298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:27.947298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:12:27.947298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:27.947915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:12:27.947915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:27.947915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:12:27.947915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:27.947915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:12:27.947915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:12:27.947915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:27.947915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:12:27.947915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:27.947915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:12:27.947915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:12:27.947915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:27.947915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:12:27.948436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:27.948436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:12:27.948436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:27.948436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:12:27.948436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:12:27.948436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:12:27.948436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:12:27.948436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:12:27.948436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:12:27.948436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:12:27.948436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:12:27.948436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:12:27.948436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:12:27.948436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:12:27.948436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:12:27.949152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:12:27.949152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:12:27.949152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:12:27.949152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:12:27.949152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:12:27.949152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:27.949152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:12:27.949152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:27.949152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:12:27.949152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:27.949152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:12:27.949152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:27.949152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:12:27.949152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:27.949152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:12:27.949152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:12:27.949152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:27.949152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:12:27.949152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:27.950049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:12:27.950049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:12:27.950049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:27.950049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:12:27.950049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:27.950049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:12:28.421974 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:12:28.422650 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:12:28.422903 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:12:28.424531 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:12:28.425274 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:12:28.425419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:28.428078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:12:28.428078 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:28.428078 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:12:28.428078 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:12:28.428078 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:28.428078 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:12:28.428078 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:28.428078 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:12:28.429401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:28.429401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:12:28.429401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:12:28.429401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:12:28.429401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:12:28.429401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:12:28.429401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:12:28.429401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:28.429401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:12:28.429401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:28.429401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:12:28.429401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:28.430313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:12:28.430313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:28.430313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:12:28.430313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:28.430313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:12:28.430313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:12:28.430313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:28.430313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:12:28.430313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:28.430313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:12:28.430313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:12:28.430313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:28.430313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:12:28.430923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:28.430923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:12:28.430923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:28.430923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:12:28.430923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:12:28.430923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:12:28.430923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:12:28.430923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:12:28.430923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:12:28.430923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:12:28.430923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:12:28.430923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:12:28.430923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:12:28.430923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:12:28.430923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:12:28.431763 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:12:28.431763 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:12:28.431763 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:12:28.431763 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:12:28.431763 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:12:28.431763 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:28.431763 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:12:28.431763 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:28.431763 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:12:28.431763 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:28.431763 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:12:28.431763 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:28.431763 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:12:28.431763 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:28.431763 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:12:28.431763 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:12:28.431763 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:28.431763 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:12:28.431763 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:28.432820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:12:28.432820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:12:28.432820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:28.432820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:12:28.432820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:28.432820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:12:28.432820 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:12:28.432820 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:12:28.432820 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:12:28.432820 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:12:28.433138 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:12:28.433278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:28.433828 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:12:28.433828 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:28.433828 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:12:28.433828 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:12:28.433828 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:28.433828 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:12:28.433828 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:28.433828 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:12:28.434756 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:28.434756 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:12:28.434756 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:12:28.434756 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:12:28.434756 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:28.434756 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:12:28.434756 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:28.434756 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:12:28.434756 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:28.434756 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:12:28.434756 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:28.434756 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:12:28.434756 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:28.434756 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:12:28.434756 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:12:28.435343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:28.435343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:12:28.435343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:28.435343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:12:28.435343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:12:28.435343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:28.435343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:12:28.435343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:28.435343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:12:28.435992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:28.435992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:12:28.435992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:12:28.435992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:12:28.435992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:12:28.435992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:12:28.435992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:12:28.435992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:12:28.435992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:12:28.435992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:12:28.435992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:28.435992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:12:28.436381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:28.436381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:12:28.436381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:28.436381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:12:28.436381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:28.436381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:12:28.436381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:28.436381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:12:28.436381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:12:28.436381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:28.436381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:12:28.436381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:28.436381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:12:28.436381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:12:28.436804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:28.436804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:12:28.436804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:28.436804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:12:28.438013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:28.438013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:12:28.438013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:12:28.438013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:12:28.438013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:12:28.438013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:12:28.438013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:12:28.438013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:12:28.438013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:12:28.438013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:12:28.438013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:12:28.438013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:12:28.438013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:12:28.438013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:12:28.438013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:12:28.438013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:12:28.438662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:12:28.438662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:12:28.438662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:12:28.438662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:12:28.438662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:12:28.438662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:28.438662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:12:28.438662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:28.438662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:12:28.438662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:28.438662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:12:28.438662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:28.438662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:12:28.438662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:28.438662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:12:28.438662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:12:28.438662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:28.438662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:12:28.439170 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:28.439170 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:12:28.439170 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:12:28.439170 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:28.439170 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:12:28.439170 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:28.439170 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:12:28.493649 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:12:28.494228 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:12:28.494327 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:12:28.494506 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:12:28.495701 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:12:28.495701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:28.496559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:12:28.496559 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:28.496559 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:12:28.496559 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:12:28.496559 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:28.496559 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:12:28.496559 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:28.496559 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:12:28.497940 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:28.497940 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:12:28.497940 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:12:28.497940 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:12:28.497940 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:28.497940 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:12:28.497940 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:28.497940 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:12:28.497940 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:28.497940 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:12:28.497940 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:28.497940 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:12:28.497940 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:28.497940 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:12:28.497940 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:12:28.498723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:28.498723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:12:28.498723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:28.498723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:12:28.498723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:12:28.498723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:28.498723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:12:28.498723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:28.498723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:12:28.499474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:28.499474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:12:28.499474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:12:28.499474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:12:28.499474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:12:28.499474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:12:28.499474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:12:28.499474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:12:28.499474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:12:28.499474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:28.499474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:12:28.500111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:28.500111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:12:28.500111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:28.500111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:12:28.500111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:28.500111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:12:28.500111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:28.500111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:12:28.500111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:12:28.500111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:28.500111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:12:28.500111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:28.500111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:12:28.500111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:12:28.500838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:28.500838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:12:28.500838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:28.500838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:12:28.501528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:28.501528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:12:28.501528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:12:28.501528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:12:28.501528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:12:28.501528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:12:28.501528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:12:28.501528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:12:28.501528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:12:28.501528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:12:28.501528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:12:28.501528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:12:28.501528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:12:28.501528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:12:28.501528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:12:28.501528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:12:28.501528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:12:28.502298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:12:28.502298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:12:28.502298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:12:28.502298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:28.502298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:12:28.502298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:28.502298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:12:28.502298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:28.502298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:12:28.502298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:28.502298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:12:28.502298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:28.502298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:12:28.502298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:12:28.502298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:28.502298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:12:28.502298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:28.502298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:12:28.502298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:12:28.504148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:28.504148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:12:28.504148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:28.504148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:12:28.505473 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:12:28.505716 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:12:28.505846 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:12:28.506011 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:12:28.506536 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:12:28.506972 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:12:28.507407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:28.508096 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:12:28.508096 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:28.508096 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:12:28.508096 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:12:28.508096 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:28.508096 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:12:28.508096 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:28.508096 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:12:28.509131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:28.509131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:12:28.509131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:12:28.509131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:12:28.509131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:12:28.509131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:12:28.509131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:12:28.509131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:28.509131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:12:28.509131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:28.509131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:12:28.509131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:28.509524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:12:28.509524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:28.509524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:12:28.509524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:28.509524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:12:28.509524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:12:28.509524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:28.509524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:12:28.509524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:28.509524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:12:28.509524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:12:28.509524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:28.509524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:12:28.509878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:28.509878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:12:28.510050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:28.510050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:12:28.510050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:12:28.510050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:12:28.510050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:12:28.510050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:12:28.510050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:12:28.510050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:12:28.510050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:12:28.510050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:12:28.510050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:12:28.510050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:12:28.510050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:12:28.510050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:12:28.510050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:12:28.510050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:12:28.510513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:12:28.510513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:12:28.510513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:28.510513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:12:28.510513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:28.510513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:12:28.510513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:28.510513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:12:28.510513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:28.510513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:12:28.510513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:28.510513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:12:28.510513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:12:28.510513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:28.510513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:12:28.510513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:28.510513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:12:28.510513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:12:28.510513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:28.511060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:12:28.511060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:28.511060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:12:32.911804 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:12:32.912368 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:12:32.912368 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:12:32.913631 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:12:32.949020 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:12:32.949910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:32.950440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:12:32.950440 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:32.950440 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:12:32.950440 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:12:32.950440 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:32.950440 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:12:32.950440 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:32.950440 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:12:32.952191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:32.952191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:12:32.952191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:12:32.952191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:12:32.952191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:12:32.952191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:12:32.952191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:12:32.952191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:32.952191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:12:32.952191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:32.952191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:12:32.952191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:32.952700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:12:32.952700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:32.952700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:12:32.952700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:32.952700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:12:32.952700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:12:32.952700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:32.952700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:12:32.952700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:32.952700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:12:32.952700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:12:32.952700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:32.952700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:12:32.953135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:32.953135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:12:32.953135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:32.953135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:12:32.953135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:12:32.953135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:12:32.953135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:12:32.953135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:12:32.953135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:12:32.953135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:12:32.953135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:12:32.953135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:12:32.953135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:12:32.953135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:12:32.953135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:12:32.954521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:12:32.954521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:12:32.954521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:12:32.954521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:12:32.954521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:12:32.954521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:32.954521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:12:32.954521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:32.954521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:12:32.954521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:32.954521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:12:32.954521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:32.954521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:12:32.954521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:32.954521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:12:32.954521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:12:32.954521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:32.954521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:12:32.954521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:32.955627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:12:32.955627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:12:32.955627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:32.955627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:12:32.955627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:32.955627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:12:33.422719 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:12:33.423527 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:12:33.423724 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:12:33.425043 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:12:33.425401 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:12:33.425646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:33.426570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:12:33.426570 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:33.426570 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:12:33.426570 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:12:33.426570 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:33.426570 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:12:33.426570 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:33.426570 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:12:33.428259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:33.428259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:12:33.428259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:12:33.428259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:12:33.428259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:12:33.428259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:12:33.428259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:12:33.428259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:33.428259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:12:33.428259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:33.428259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:12:33.428259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:33.428885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:12:33.428885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:33.428885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:12:33.428885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:33.428885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:12:33.428885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:12:33.428885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:33.428885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:12:33.428885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:33.428885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:12:33.428885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:12:33.428885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:33.428885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:12:33.429431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:33.429431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:12:33.430221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:33.430221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:12:33.430221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:12:33.430221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:12:33.430221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:12:33.430221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:12:33.430221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:12:33.430221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:12:33.430221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:12:33.430221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:12:33.430221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:12:33.430221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:12:33.430221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:12:33.430221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:12:33.430221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:12:33.430221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:12:33.430997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:12:33.430997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:12:33.430997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:33.430997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:12:33.430997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:33.430997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:12:33.430997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:33.430997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:12:33.430997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:33.430997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:12:33.430997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:33.430997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:12:33.430997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:12:33.430997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:33.430997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:12:33.430997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:33.430997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:12:33.430997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:12:33.430997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:33.431823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:12:33.431823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:33.431823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:12:33.432416 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:12:33.432764 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:12:33.432911 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:12:33.433046 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:12:33.433510 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:12:33.433862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:33.434570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:12:33.434570 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:33.434570 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:12:33.434570 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:12:33.434570 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:33.434570 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:12:33.434570 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:33.434570 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:12:33.435404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:33.435404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:12:33.435404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:12:33.435404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:12:33.435404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:33.435404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:12:33.435404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:33.435404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:12:33.435404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:33.435404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:12:33.435404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:33.435404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:12:33.435404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:33.435404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:12:33.435404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:12:33.435916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:33.435916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:12:33.435916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:33.435916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:12:33.435916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:12:33.435916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:33.435916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:12:33.435916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:33.435916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:12:33.436251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:33.436251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:12:33.436251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:12:33.436251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:12:33.436251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:12:33.436251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:12:33.436251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:12:33.436251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:12:33.436251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:12:33.436251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:12:33.436251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:33.436251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:12:33.436617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:33.436617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:12:33.436617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:33.436617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:12:33.436617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:33.436617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:12:33.436617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:33.436617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:12:33.436617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:12:33.436617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:33.436617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:12:33.436617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:33.436617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:12:33.436617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:12:33.437078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:33.437078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:12:33.437078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:33.437078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:12:33.437319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:33.437319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:12:33.437319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:12:33.437319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:12:33.437319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:12:33.437319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:12:33.437319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:12:33.437319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:12:33.437319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:12:33.437319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:12:33.437319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:12:33.437319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:12:33.437319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:12:33.437319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:12:33.437319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:12:33.437319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:12:33.437778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:12:33.437778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:12:33.437778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:12:33.437778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:12:33.437778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:12:33.437778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:33.437778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:12:33.437778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:33.437778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:12:33.437778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:33.437778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:12:33.437778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:33.437778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:12:33.437778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:33.437778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:12:33.437778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:12:33.437778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:33.437778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:12:33.438252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:33.438252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:12:33.438252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:12:33.438252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:33.438252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:12:33.438252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:33.438252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:12:33.494483 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:12:33.494920 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:12:33.495151 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:12:33.495357 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:12:33.496085 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:12:33.496485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:33.497379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:12:33.497379 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:33.497379 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:12:33.497379 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:12:33.497379 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:33.497379 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:12:33.497379 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:33.497379 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:12:33.498575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:33.498575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:12:33.498575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:12:33.498575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:12:33.498575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:33.498575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:12:33.498575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:33.498575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:12:33.498575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:33.498575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:12:33.498575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:33.498575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:12:33.498575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:33.498575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:12:33.498575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:12:33.499372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:33.499372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:12:33.499372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:33.499372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:12:33.499372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:12:33.499372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:33.499372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:12:33.499372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:33.499372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:12:33.500110 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:33.500110 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:12:33.500110 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:12:33.500110 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:12:33.500110 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:12:33.500110 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:12:33.500110 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:12:33.500110 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:12:33.500110 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:12:33.500110 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:33.500110 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:12:33.500622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:33.500622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:12:33.500622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:33.500622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:12:33.500622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:33.500622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:12:33.500622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:33.500622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:12:33.500622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:12:33.500622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:33.500622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:12:33.500622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:33.500622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:12:33.500622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:12:33.501220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:33.501220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:12:33.501220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:33.501220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:12:33.502469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:33.502469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:12:33.502469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:12:33.502469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:12:33.502469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:12:33.502469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:12:33.502469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:12:33.502469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:12:33.502469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:12:33.502469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:12:33.502469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:12:33.502469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:12:33.502469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:12:33.502469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:12:33.502469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:12:33.502469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:12:33.502469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:12:33.502981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:12:33.502981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:12:33.502981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:12:33.502981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:33.502981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:12:33.502981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:33.502981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:12:33.502981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:33.502981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:12:33.502981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:33.502981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:12:33.502981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:33.502981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:12:33.502981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:12:33.502981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:33.502981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:12:33.502981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:33.502981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:12:33.502981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:12:33.503460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:33.503460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:12:33.503460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:33.503460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:12:33.505965 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:12:33.506307 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:12:33.506428 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:12:33.506579 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:12:33.507093 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:12:33.507455 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:12:33.507690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:33.508428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:12:33.508428 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:33.508428 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:12:33.508428 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:12:33.508428 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:33.508428 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:12:33.508428 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:33.508428 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:12:33.509289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:33.509289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:12:33.509289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:12:33.509289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:12:33.509289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:12:33.509289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:12:33.509289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:12:33.509289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:33.509289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:12:33.509289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:33.509289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:12:33.509289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:33.509662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:12:33.509662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:33.509662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:12:33.509662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:33.509662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:12:33.509662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:12:33.509662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:33.509662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:12:33.509662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:33.509662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:12:33.509662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:12:33.509662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:33.509662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:12:33.510049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:33.510049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:12:33.510269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:33.510269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:12:33.510269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:12:33.510269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:12:33.510269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:12:33.510269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:12:33.510269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:12:33.510269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:12:33.510269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:12:33.510269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:12:33.510269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:12:33.510269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:12:33.510269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:12:33.510269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:12:33.510269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:12:33.510269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:12:33.510724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:12:33.510724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:12:33.510724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:33.510724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:12:33.510724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:33.510724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:12:33.510724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:33.510724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:12:33.510724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:33.510724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:12:33.510724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:33.510724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:12:33.510724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:12:33.510724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:33.510724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:12:33.510724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:33.510724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:12:33.510724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:12:33.510724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:33.511326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:12:33.511326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:33.511326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:12:37.912466 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:12:37.913270 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:12:37.913363 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:12:37.914693 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:12:37.948142 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:12:37.948326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:37.948761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:12:37.948761 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:37.948761 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:12:37.948761 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:12:37.948761 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:37.948761 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:12:37.948761 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:37.948761 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:12:37.949384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:37.949384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:12:37.949384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:12:37.949384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:12:37.949384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:12:37.949384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:12:37.949384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:12:37.949384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:37.949384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:12:37.949384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:37.949384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:12:37.949384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:37.949995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:12:37.949995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:37.949995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:12:37.949995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:37.949995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:12:37.949995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:12:37.949995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:37.949995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:12:37.949995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:37.949995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:12:37.949995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:12:37.949995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:37.949995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:12:37.950588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:37.950588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:12:37.950588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:37.950588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:12:37.950588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:12:37.950588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:12:37.950588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:12:37.950588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:12:37.950588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:12:37.950588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:12:37.950588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:12:37.950588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:12:37.950588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:12:37.950588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:12:37.950588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:12:37.951258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:12:37.951258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:12:37.951258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:12:37.951258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:12:37.951258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:12:37.951258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:37.951258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:12:37.951258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:37.951258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:12:37.951258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:37.951258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:12:37.951258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:37.951258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:12:37.951258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:37.951258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:12:37.951258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:12:37.951258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:37.951258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:12:37.951258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:37.952158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:12:37.952158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:12:37.952158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:37.952158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:12:37.952158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:37.952158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:12:38.423477 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:12:38.424153 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:12:38.424475 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:12:38.425449 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:12:38.425926 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:12:38.426171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:38.427236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:12:38.427236 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:38.427236 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:12:38.427236 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:12:38.427236 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:38.427236 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:12:38.427236 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:38.427236 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:12:38.430026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:38.430026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:12:38.430026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:12:38.430026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:12:38.430026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:12:38.430026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:12:38.430026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:12:38.430026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:38.430026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:12:38.430026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:38.430026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:12:38.430026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:38.430638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:12:38.430638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:38.430638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:12:38.430638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:38.430638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:12:38.430638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:12:38.430638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:38.430638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:12:38.430638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:38.430638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:12:38.430638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:12:38.430638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:38.430638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:12:38.431203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:38.431203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:12:38.431203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:38.431203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:12:38.431203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:12:38.431203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:12:38.431203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:12:38.431203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:12:38.431203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:12:38.431203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:12:38.431203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:12:38.431203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:12:38.431203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:12:38.431203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:12:38.431203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:12:38.431829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:12:38.431829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:12:38.431829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:12:38.431829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:12:38.431829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:12:38.431829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:38.431829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:12:38.431829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:38.431829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:12:38.431829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:38.431829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:12:38.431829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:38.431829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:12:38.431829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:38.431829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:12:38.431829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:12:38.431829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:38.431829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:12:38.431829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:38.432989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:12:38.432989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:12:38.432989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:38.432989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:12:38.432989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:38.432989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:12:38.436180 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:12:38.436641 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:12:38.436825 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:12:38.437028 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:12:38.437384 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:12:38.437619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:38.438166 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:12:38.438166 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:38.438166 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:12:38.438166 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:12:38.438166 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:38.438166 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:12:38.438166 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:38.438166 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:12:38.438928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:38.438928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:12:38.438928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:12:38.438928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:12:38.438928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:38.438928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:12:38.438928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:38.438928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:12:38.438928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:38.438928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:12:38.438928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:38.438928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:12:38.438928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:38.438928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:12:38.438928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:12:38.439397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:38.439397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:12:38.439397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:38.439397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:12:38.439397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:12:38.439397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:38.439397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:12:38.439397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:38.439397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:12:38.439947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:38.439947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:12:38.439947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:12:38.439947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:12:38.439947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:12:38.439947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:12:38.439947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:12:38.439947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:12:38.439947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:12:38.439947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:12:38.439947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:38.439947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:12:38.440336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:38.440336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:12:38.440336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:38.440336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:12:38.440336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:38.440336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:12:38.440336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:38.440336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:12:38.440336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:12:38.440336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:38.440336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:12:38.440336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:38.440336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:12:38.440336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:12:38.440965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:38.440965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:12:38.440965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:38.440965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:12:38.441320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:38.441320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:12:38.441320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:12:38.441320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:12:38.441320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:12:38.441320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:12:38.441320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:12:38.441320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:12:38.441320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:12:38.441320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:12:38.441320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:12:38.441320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:12:38.441320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:12:38.441320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:12:38.441320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:12:38.441320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:12:38.441951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:12:38.441951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:12:38.441951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:12:38.441951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:12:38.441951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:12:38.441951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:38.441951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:12:38.441951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:38.441951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:12:38.441951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:38.441951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:12:38.441951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:38.441951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:12:38.441951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:38.441951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:12:38.441951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:12:38.441951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:38.441951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:12:38.442473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:38.442473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:12:38.442473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:12:38.442473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:38.442473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:12:38.442473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:38.442473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:12:38.495026 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:12:38.495535 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:12:38.495706 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:12:38.497168 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:12:38.497168 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:12:38.497168 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:38.497536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:12:38.497536 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:38.497536 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:12:38.497536 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:12:38.497536 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:38.497536 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:12:38.497536 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:38.497536 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:12:38.498400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:38.498400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:12:38.498400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:12:38.498400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:12:38.498400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:38.498400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:12:38.498400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:38.498400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:12:38.498400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:38.498400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:12:38.498400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:38.498400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:12:38.498400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:38.498400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:12:38.498400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:12:38.498858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:38.498858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:12:38.498858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:38.498858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:12:38.498858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:12:38.498858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:38.498858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:12:38.498858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:38.498858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:12:38.499367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:38.499367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:12:38.499367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:12:38.499367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:12:38.499367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:12:38.499367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:12:38.499367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:12:38.499367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:12:38.499367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:12:38.499367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:38.499367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:12:38.499685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:38.499685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:12:38.499685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:38.499685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:12:38.499685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:38.499685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:12:38.499685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:38.499685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:12:38.499685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:12:38.499685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:38.499685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:12:38.499685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:38.499685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:12:38.499685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:12:38.500044 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:38.500044 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:12:38.500044 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:38.500044 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:12:38.500644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:38.500644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:12:38.500644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:12:38.500644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:12:38.500644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:12:38.500644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:12:38.500644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:12:38.500644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:12:38.500644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:12:38.500644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:12:38.500644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:12:38.500644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:12:38.500644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:12:38.500644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:12:38.500644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:12:38.500644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:12:38.500644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:12:38.501126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:12:38.501126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:12:38.501126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:12:38.501126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:38.501126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:12:38.501126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:38.501126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:12:38.501126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:38.501126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:12:38.501126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:38.501126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:12:38.501126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:38.501126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:12:38.501126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:12:38.501126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:38.501126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:12:38.501126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:38.501126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:12:38.501126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:12:38.501654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:38.501654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:12:38.501654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:38.501654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:12:38.506485 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:12:38.507163 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:12:38.507387 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:12:38.507710 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:12:38.508303 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:12:38.508711 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:12:38.509024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:38.509972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:12:38.509972 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:38.509972 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:12:38.509972 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:12:38.509972 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:38.509972 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:12:38.509972 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:38.509972 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:12:38.511130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:38.511130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:12:38.511130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:12:38.511130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:12:38.511130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:12:38.511130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:12:38.511130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:12:38.511130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:38.511130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:12:38.511130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:38.511130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:12:38.511130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:38.511504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:12:38.511504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:38.511504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:12:38.511504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:38.511504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:12:38.511504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:12:38.511504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:38.511504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:12:38.511504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:38.511504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:12:38.511504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:12:38.511504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:38.511504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:12:38.511891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:38.511891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:12:38.512115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:38.512115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:12:38.512115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:12:38.512115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:12:38.512115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:12:38.512115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:12:38.512115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:12:38.512115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:12:38.512115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:12:38.512115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:12:38.512115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:12:38.512115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:12:38.512115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:12:38.512115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:12:38.512115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:12:38.512115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:12:38.512560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:12:38.512560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:12:38.512560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:38.512560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:12:38.512560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:38.512560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:12:38.512560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:38.512560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:12:38.512560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:38.512560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:12:38.512560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:38.512560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:12:38.512560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:12:38.512560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:38.512560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:12:38.512560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:38.512560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:12:38.512560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:12:38.512560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:38.513087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:12:38.513087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:38.513087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:12:42.913330 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:12:42.914334 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:12:42.914334 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:12:42.915695 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:12:42.918197 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG dbcounter [-] [106314] Writing DB stats neutron:SELECT=240 {{(pid=106314) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 09 09:12:42.953545 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:12:42.953898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:42.954265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:12:42.954265 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:42.954265 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:12:42.954265 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:12:42.954265 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:42.954265 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:12:42.954265 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:42.954265 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:12:42.954990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:42.954990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:12:42.954990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:12:42.954990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:12:42.954990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:12:42.954990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:12:42.954990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:12:42.954990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:42.954990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:12:42.954990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:42.954990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:12:42.954990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:42.955598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:12:42.955598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:42.955598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:12:42.955598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:42.955598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:12:42.955598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:12:42.955598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:42.955598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:12:42.955598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:42.955598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:12:42.955598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:12:42.955598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:42.955598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:12:42.956251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:42.956251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:12:42.956251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:42.956251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:12:42.956251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:12:42.956251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:12:42.956251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:12:42.956251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:12:42.956251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:12:42.956251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:12:42.956251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:12:42.956251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:12:42.956251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:12:42.956251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:12:42.956251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:12:42.956947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:12:42.956947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:12:42.956947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:12:42.956947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:12:42.956947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:12:42.956947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:42.956947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:12:42.956947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:42.956947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:12:42.956947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:42.956947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:12:42.956947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:42.956947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:12:42.956947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:42.956947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:12:42.956947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:12:42.956947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:42.956947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:12:42.956947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:42.957795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:12:42.957795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:12:42.957795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:42.957795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:12:42.957795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:42.957795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:12:43.424272 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:12:43.425366 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:12:43.425637 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:12:43.426984 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:12:43.427308 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:12:43.427735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:43.428601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:12:43.428601 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:43.428601 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:12:43.428601 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:12:43.428601 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:43.428601 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:12:43.428601 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:43.428601 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:12:43.429562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:43.429562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:12:43.429562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:12:43.429562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:12:43.429562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:12:43.429562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:12:43.429562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:12:43.429562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:43.429562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:12:43.429562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:43.429562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:12:43.429562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:43.430165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:12:43.430165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:43.430165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:12:43.430165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:43.430165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:12:43.430165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:12:43.430165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:43.430165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:12:43.430165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:43.430165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:12:43.430165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:12:43.430165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:43.430165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:12:43.430679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:43.430679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:12:43.430679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:43.430679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:12:43.430679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:12:43.430679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:12:43.430679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:12:43.430679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:12:43.430679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:12:43.430679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:12:43.430679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:12:43.430679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:12:43.430679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:12:43.430679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:12:43.430679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:12:43.431303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:12:43.431303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:12:43.431303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:12:43.431303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:12:43.431303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:12:43.431303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:43.431303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:12:43.431303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:43.431303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:12:43.431303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:43.431303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:12:43.431303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:43.431303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:12:43.431303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:43.431303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:12:43.431303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:12:43.431303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:43.431303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:12:43.431303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:43.432002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:12:43.432002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:12:43.432002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:43.432002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:12:43.432002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:43.432002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:12:43.436671 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:12:43.437023 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:12:43.437122 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:12:43.437260 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:12:43.437524 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:12:43.437729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:43.438195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:12:43.438195 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:43.438195 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:12:43.438195 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:12:43.438195 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:43.438195 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:12:43.438195 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:43.438195 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:12:43.438928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:43.438928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:12:43.438928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:12:43.438928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:12:43.438928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:43.438928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:12:43.438928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:43.438928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:12:43.438928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:43.438928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:12:43.438928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:43.438928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:12:43.438928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:43.438928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:12:43.438928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:12:43.439376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:43.439376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:12:43.439376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:43.439376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:12:43.439376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:12:43.439376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:43.439376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:12:43.439376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:43.439376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:12:43.439841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:43.439841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:12:43.439841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:12:43.439841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:12:43.439841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:12:43.439841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:12:43.439841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:12:43.439841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:12:43.439841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:12:43.439841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:12:43.439841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:43.439841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:12:43.440928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:43.440928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:12:43.440928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:43.440928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:12:43.440928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:43.440928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:12:43.440928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:43.440928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:12:43.440928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:12:43.440928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:43.440928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:12:43.440928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:43.440928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:12:43.440928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:12:43.441959 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:43.441959 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:12:43.441959 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:43.441959 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:12:43.441959 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:43.441959 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:12:43.441959 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:12:43.441959 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:12:43.441959 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:12:43.441959 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:12:43.441959 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:12:43.441959 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:12:43.441959 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:12:43.441959 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:12:43.442306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:12:43.442306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:12:43.442306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:12:43.442306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:12:43.442306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:12:43.442306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:12:43.442306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:12:43.442306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:12:43.442306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:12:43.442306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:12:43.442306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:12:43.442306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:43.442306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:12:43.442306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:43.442306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:12:43.442306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:43.442306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:12:43.442306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:43.442306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:12:43.442798 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:43.442798 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:12:43.442798 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:12:43.442798 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:43.442798 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:12:43.442798 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:43.442798 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:12:43.442798 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:12:43.442798 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:43.442798 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:12:43.442798 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:43.442798 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:12:43.495776 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:12:43.496174 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:12:43.496299 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:12:43.496380 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:12:43.497283 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:12:43.497571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:43.498278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:12:43.498278 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:43.498278 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:12:43.498278 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:12:43.498278 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:43.498278 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:12:43.498278 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:43.498278 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:12:43.499229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:43.499229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:12:43.499229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:12:43.499229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:12:43.499229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:43.499229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:12:43.499229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:43.499229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:12:43.499229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:43.499229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:12:43.499229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:43.499229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:12:43.499229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:43.499229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:12:43.499229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:12:43.499907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:43.499907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:12:43.499907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:43.499907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:12:43.499907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:12:43.499907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:43.499907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:12:43.499907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:43.499907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:12:43.500800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:43.500800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:12:43.500800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:12:43.500800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:12:43.500800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:12:43.500800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:12:43.500800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:12:43.500800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:12:43.500800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:12:43.500800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:43.500800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:12:43.501941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:43.501941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:12:43.501941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:43.501941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:12:43.501941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:43.501941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:12:43.501941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:43.501941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:12:43.501941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:12:43.501941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:43.501941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:12:43.501941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:43.501941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:12:43.501941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:12:43.503393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:43.503393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:12:43.503393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:43.503393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:12:43.503393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:43.503393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:12:43.503393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:12:43.503393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:12:43.503393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:12:43.503393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:12:43.503393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:12:43.503393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:12:43.503393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:12:43.503393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:12:43.504068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:12:43.504068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:12:43.504068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:12:43.504068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:12:43.504068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:12:43.504068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:12:43.504068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:12:43.504068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:12:43.504068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:12:43.504068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:12:43.504068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:43.504068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:12:43.504068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:43.504068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:12:43.504068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:43.504068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:12:43.504068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:43.504068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:12:43.504068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:43.504526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:12:43.504526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:12:43.504526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:43.504526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:12:43.504526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:43.504526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:12:43.504526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:12:43.504526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:43.504526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:12:43.504526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:43.504526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:12:43.507071 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:12:43.507399 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:12:43.507532 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:12:43.507731 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:12:43.508180 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:12:43.508447 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:12:43.508615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:43.509260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:12:43.509260 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:43.509260 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:12:43.509260 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:12:43.509260 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:43.509260 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:12:43.509260 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:43.509260 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:12:43.510137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:43.510137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:12:43.510137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:12:43.510137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:12:43.510137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:12:43.510137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:12:43.510137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:12:43.510137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:43.510137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:12:43.510137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:43.510137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:12:43.510137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:43.510637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:12:43.510637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:43.510637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:12:43.510637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:43.510637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:12:43.510637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:12:43.510637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:43.510637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:12:43.510637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:43.510637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:12:43.510637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:12:43.510637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:43.510637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:12:43.510987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:43.510987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:12:43.511320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:43.511320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:12:43.511320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:12:43.511320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:12:43.511320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:12:43.511320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:12:43.511320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:12:43.511320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:12:43.511320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:12:43.511320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:12:43.511320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:12:43.511320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:12:43.511320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:12:43.511320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:12:43.511320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:12:43.511320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:12:43.512218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:12:43.512218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:12:43.512218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:43.512218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:12:43.512218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:43.512218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:12:43.512218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:43.512218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:12:43.512218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:43.512218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:12:43.512218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:43.512218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:12:43.512218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:12:43.512218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:43.512218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:12:43.512218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:43.512218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:12:43.512218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:12:43.512218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:43.512672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:12:43.512672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:43.512672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:12:47.914172 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:12:47.914962 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:12:47.914962 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:12:47.916155 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:12:47.951640 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:12:47.952169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:47.953263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:12:47.953263 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:47.953263 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:12:47.953263 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:12:47.953263 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:47.953263 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:12:47.953263 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:47.953263 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:12:47.954314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:47.954314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:12:47.954314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:12:47.954314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:12:47.954314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:12:47.954314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:12:47.954314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:12:47.954314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:47.954314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:12:47.954314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:47.954314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:12:47.954314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:47.954735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:12:47.954735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:47.954735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:12:47.954735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:47.954735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:12:47.954735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:12:47.954735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:47.954735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:12:47.954735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:47.954735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:12:47.954735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:12:47.954735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:47.954735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:12:47.955142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:47.955142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:12:47.955503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:47.955503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:12:47.955503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:12:47.955503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:12:47.955503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:12:47.955503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:12:47.955503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:12:47.955503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:12:47.955503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:12:47.955503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:12:47.955503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:12:47.955503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:12:47.955503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:12:47.955503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:12:47.955503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:12:47.955503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:12:47.956068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:12:47.956068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:12:47.956068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:47.956068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:12:47.956068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:47.956068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:12:47.956068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:47.956068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:12:47.956068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:47.956068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:12:47.956068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:47.956068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:12:47.956068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:12:47.956068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:47.956068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:12:47.956068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:47.956068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:12:47.956068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:12:47.956068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:47.956604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:12:47.956604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:47.956604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:12:48.425187 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:12:48.425933 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:12:48.426118 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:12:48.427314 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:12:48.427857 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:12:48.427970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:48.428904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:12:48.428904 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:48.428904 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:12:48.428904 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:12:48.428904 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:48.428904 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:12:48.428904 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:48.428904 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:12:48.430407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:48.430407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:12:48.430407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:12:48.430407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:12:48.430407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:12:48.430407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:12:48.430407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:12:48.430407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:48.430407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:12:48.430407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:48.430407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:12:48.430407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:48.431079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:12:48.431079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:48.431079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:12:48.431079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:48.431079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:12:48.431079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:12:48.431079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:48.431079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:12:48.431079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:48.431079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:12:48.431079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:12:48.431079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:48.431079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:12:48.431780 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:48.431780 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:12:48.432122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:48.432122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:12:48.432122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:12:48.432122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:12:48.432122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:12:48.432122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:12:48.432122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:12:48.432122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:12:48.432122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:12:48.432122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:12:48.432122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:12:48.432122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:12:48.432122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:12:48.432122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:12:48.432122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:12:48.432122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:12:48.432949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:12:48.432949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:12:48.432949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:48.432949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:12:48.432949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:48.432949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:12:48.432949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:48.432949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:12:48.432949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:48.432949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:12:48.432949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:48.432949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:12:48.432949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:12:48.432949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:48.432949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:12:48.432949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:48.432949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:12:48.432949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:12:48.432949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:48.433863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:12:48.433863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:48.433863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:12:48.437280 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:12:48.437610 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:12:48.437712 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:12:48.437848 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:12:48.438318 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:12:48.438604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:48.439276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:12:48.439276 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:48.439276 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:12:48.439276 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:12:48.439276 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:48.439276 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:12:48.439276 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:48.439276 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:12:48.440161 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:48.440161 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:12:48.440161 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:12:48.440161 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:12:48.440161 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:48.440161 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:12:48.440161 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:48.440161 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:12:48.440161 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:48.440161 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:12:48.440161 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:48.440161 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:12:48.440161 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:48.440161 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:12:48.440161 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:12:48.440609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:48.440609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:12:48.440609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:48.440609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:12:48.440609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:12:48.440609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:48.440609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:12:48.440609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:48.440609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:12:48.441054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:48.441054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:12:48.441054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:12:48.441054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:12:48.441054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:12:48.441054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:12:48.441054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:12:48.441054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:12:48.441054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:12:48.441054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:12:48.441054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:48.441054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:12:48.441452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:48.441452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:12:48.441452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:48.441452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:12:48.441452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:48.441452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:12:48.441452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:48.441452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:12:48.441452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:12:48.441452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:48.441452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:12:48.441452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:48.441452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:12:48.441452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:12:48.441877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:48.441877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:12:48.441877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:48.441877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:12:48.442192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:48.442192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:12:48.442192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:12:48.442192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:12:48.442192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:12:48.442192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:12:48.442192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:12:48.442192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:12:48.442192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:12:48.442192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:12:48.442192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:12:48.442192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:12:48.442192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:12:48.442192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:12:48.442192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:12:48.442192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:12:48.442684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:12:48.442684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:12:48.442684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:12:48.442684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:12:48.442684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:12:48.442684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:48.442684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:12:48.442684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:48.442684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:12:48.442684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:48.442684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:12:48.442684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:48.442684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:12:48.442684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:48.442684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:12:48.442684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:12:48.442684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:48.442684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:12:48.443193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:48.443193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:12:48.443193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:12:48.443193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:48.443193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:12:48.443193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:48.443193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:12:48.496551 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:12:48.497172 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:12:48.497440 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:12:48.497768 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:12:48.498379 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:12:48.498757 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:48.499817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:12:48.499817 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:48.499817 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:12:48.499817 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:12:48.499817 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:48.499817 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:12:48.499817 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:48.499817 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:12:48.500869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:48.500869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:12:48.500869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:12:48.500869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:12:48.500869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:48.500869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:12:48.500869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:48.500869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:12:48.500869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:48.500869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:12:48.500869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:48.500869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:12:48.500869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:48.500869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:12:48.500869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:12:48.501271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:48.501271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:12:48.501271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:48.501271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:12:48.501271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:12:48.501271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:48.501271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:12:48.501271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:48.501271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:12:48.501725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:48.501725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:12:48.501725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:12:48.501725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:12:48.501725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:12:48.501725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:12:48.501725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:12:48.501725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:12:48.501725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:12:48.501725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:48.501725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:12:48.502103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:48.502103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:12:48.502103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:48.502103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:12:48.502103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:48.502103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:12:48.502103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:48.502103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:12:48.502103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:12:48.502103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:48.502103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:12:48.502103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:48.502103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:12:48.502103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:12:48.502475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:48.502475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:12:48.502475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:48.502475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:12:48.502866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:48.502866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:12:48.502866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:12:48.502866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:12:48.502866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:12:48.502866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:12:48.502866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:12:48.502866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:12:48.502866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:12:48.502866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:12:48.502866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:12:48.502866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:12:48.502866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:12:48.502866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:12:48.502866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:12:48.502866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:12:48.502866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:12:48.503391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:12:48.503391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:12:48.503391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:12:48.503391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:48.503391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:12:48.503391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:48.503391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:12:48.503391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:48.503391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:12:48.503391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:48.503391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:12:48.503391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:48.503391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:12:48.503391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:12:48.503391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:48.503391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:12:48.503391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:48.503391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:12:48.503391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:12:48.503911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:48.503911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:12:48.503911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:48.503911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:12:48.507567 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:12:48.507882 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:12:48.507992 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:12:48.508148 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:12:48.508497 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:12:48.508761 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:12:48.508965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:48.509571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:12:48.509571 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:48.509571 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:12:48.509571 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:12:48.509571 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:48.509571 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:12:48.509571 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:48.509571 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:12:48.510320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:48.510320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:12:48.510320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:12:48.510320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:12:48.510320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:12:48.510320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:12:48.510320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:12:48.510320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:48.510320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:12:48.510320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:48.510320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:12:48.510320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:48.510642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:12:48.510642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:48.510642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:12:48.510642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:48.510642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:12:48.510642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:12:48.510642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:48.510642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:12:48.510642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:48.510642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:12:48.510642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:12:48.510642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:48.510642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:12:48.511043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:48.511043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:12:48.511277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:48.511277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:12:48.511277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:12:48.511277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:12:48.511277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:12:48.511277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:12:48.511277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:12:48.511277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:12:48.511277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:12:48.511277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:12:48.511277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:12:48.511277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:12:48.511277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:12:48.511277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:12:48.511277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:12:48.511277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:12:48.511773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:12:48.511773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:12:48.511773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:48.511773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:12:48.511773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:48.511773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:12:48.511773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:48.511773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:12:48.511773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:48.511773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:12:48.511773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:48.511773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:12:48.511773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:12:48.511773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:48.511773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:12:48.511773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:48.511773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:12:48.511773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:12:48.511773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:48.512324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:12:48.512324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:48.512324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:12:52.915152 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:12:52.916236 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:12:52.916236 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:12:52.917596 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:12:52.950108 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:12:52.950545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:52.951500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:12:52.951500 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:52.951500 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:12:52.951500 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:12:52.951500 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:52.951500 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:12:52.951500 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:52.951500 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:12:52.952271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:52.952271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:12:52.952271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:12:52.952271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:12:52.952271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:12:52.952271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:12:52.952271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:12:52.952271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:52.952271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:12:52.952271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:52.952271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:12:52.952271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:52.953190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:12:52.953190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:52.953190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:12:52.953190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:52.953190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:12:52.953190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:12:52.953190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:52.953190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:12:52.953190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:52.953190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:12:52.953190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:12:52.953190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:52.953190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:12:52.954085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:52.954085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:12:52.954085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:52.954085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:12:52.954085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:12:52.954085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:12:52.954085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:12:52.954085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:12:52.954085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:12:52.954085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:12:52.954085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:12:52.954085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:12:52.954085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:12:52.954085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:12:52.954085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:12:52.955168 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:12:52.955168 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:12:52.955168 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:12:52.955168 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:12:52.955168 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:12:52.955168 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:52.955168 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:12:52.955168 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:52.955168 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:12:52.955168 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:52.955168 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:12:52.955168 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:52.955168 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:12:52.955168 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:52.955168 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:12:52.955168 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:12:52.955168 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:52.955168 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:12:52.955168 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:52.956503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:12:52.956503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:12:52.956503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:52.956503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:12:52.956503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:52.956503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:12:53.426001 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:12:53.426279 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:12:53.426456 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:12:53.427156 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:12:53.427345 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:12:53.427489 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:53.428064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:12:53.428064 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:53.428064 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:12:53.428064 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:12:53.428064 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:53.428064 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:12:53.428064 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:53.428064 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:12:53.429004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:53.429004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:12:53.429004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:12:53.429004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:12:53.429004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:12:53.429004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:12:53.429004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:12:53.429004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:53.429004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:12:53.429004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:53.429004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:12:53.429004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:53.430443 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:12:53.430443 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:53.430443 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:12:53.430443 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:53.430443 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:12:53.430443 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:12:53.430443 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:53.430443 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:12:53.430443 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:53.430443 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:12:53.430443 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:12:53.430443 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:53.430443 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:12:53.431746 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:53.431746 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:12:53.431746 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:53.431746 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:12:53.431746 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:12:53.431746 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:12:53.431746 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:12:53.431746 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:12:53.431746 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:12:53.431746 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:12:53.431746 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:12:53.431746 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:12:53.431746 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:12:53.431746 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:12:53.431746 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:12:53.433323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:12:53.433323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:12:53.433323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:12:53.433323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:12:53.433323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:12:53.433323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:53.433323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:12:53.433323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:53.433323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:12:53.433323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:53.433323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:12:53.433323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:53.433323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:12:53.433323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:53.433323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:12:53.433323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:12:53.433323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:53.433323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:12:53.433323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:53.435242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:12:53.435242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:12:53.435242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:53.435242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:12:53.435242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:53.435242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:12:53.437886 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:12:53.438109 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:12:53.438236 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:12:53.438369 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:12:53.438610 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:12:53.438785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:53.439220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:12:53.439220 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:53.439220 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:12:53.439220 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:12:53.439220 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:53.439220 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:12:53.439220 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:53.439220 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:12:53.440120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:53.440120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:12:53.440120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:12:53.440120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:12:53.440120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:53.440120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:12:53.440120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:53.440120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:12:53.440120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:53.440120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:12:53.440120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:53.440120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:12:53.440120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:53.440120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:12:53.440120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:12:53.442024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:53.442024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:12:53.442024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:53.442024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:12:53.442024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:12:53.442024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:53.442024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:12:53.442024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:53.442024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:12:53.442024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:53.442024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:12:53.442024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:12:53.442024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:12:53.442024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:12:53.443661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:12:53.443661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:12:53.443661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:12:53.443661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:12:53.443661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:12:53.443661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:53.443661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:12:53.443661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:53.443661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:12:53.443661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:53.443661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:12:53.443661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:53.443661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:12:53.445157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:53.445157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:12:53.445157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:12:53.445157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:53.445157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:12:53.445157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:53.445157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:12:53.445157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:12:53.445157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:53.445157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:12:53.445157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:53.445157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:12:53.446612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:53.446612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:12:53.446612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:12:53.446612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:12:53.446612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:12:53.446612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:12:53.446612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:12:53.446612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:12:53.446612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:12:53.446612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:12:53.446612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:12:53.446612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:12:53.446612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:12:53.446612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:12:53.446612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:12:53.446612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:12:53.448320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:12:53.448320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:12:53.448320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:12:53.448320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:12:53.448320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:12:53.448320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:53.448320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:12:53.448320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:53.448320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:12:53.448320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:53.448320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:12:53.448320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:53.448320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:12:53.448320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:53.448320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:12:53.448320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:12:53.448320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:53.448320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:12:53.449808 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:53.449808 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:12:53.449808 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:12:53.449808 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:53.449808 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:12:53.449808 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:53.449808 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:12:53.497125 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:12:53.497371 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:12:53.497371 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:12:53.497673 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:12:53.498056 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:12:53.498413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:53.499132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:12:53.499132 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:53.499132 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:12:53.499132 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:12:53.499132 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:53.499132 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:12:53.499132 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:53.499132 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:12:53.500395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:53.500395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:12:53.500395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:12:53.500395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:12:53.500395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:53.500395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:12:53.500395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:53.500395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:12:53.500395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:53.500395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:12:53.500395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:53.500395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:12:53.500395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:53.500395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:12:53.500395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:12:53.502088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:53.502088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:12:53.502088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:53.502088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:12:53.502088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:12:53.502088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:53.502088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:12:53.502088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:53.502088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:12:53.502088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:53.502088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:12:53.502088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:12:53.502088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:12:53.502088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:12:53.503736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:12:53.503736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:12:53.503736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:12:53.503736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:12:53.503736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:53.503736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:12:53.503736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:53.503736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:12:53.503736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:53.503736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:12:53.503736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:53.503736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:12:53.503736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:53.504344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:12:53.504344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:12:53.504344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:53.504344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:12:53.504344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:53.504344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:12:53.504344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:12:53.504344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:53.504344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:12:53.504344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:53.504344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:12:53.504344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:53.505493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:12:53.505493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:12:53.505493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:12:53.505493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:12:53.505493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:12:53.505493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:12:53.505493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:12:53.505493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:12:53.505493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:12:53.505493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:12:53.505493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:12:53.505493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:12:53.505493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:12:53.505493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:12:53.505493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:12:53.505493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:12:53.505493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:12:53.505493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:12:53.505493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:12:53.507196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:53.507196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:12:53.507196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:53.507196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:12:53.507196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:53.507196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:12:53.507196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:53.507196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:12:53.507196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:53.507196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:12:53.507196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:12:53.507196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:53.507196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:12:53.507196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:53.507196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:12:53.507196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:12:53.507196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:53.507196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:12:53.507196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:53.507196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:12:53.508067 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:12:53.508298 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:12:53.508380 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:12:53.508506 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:12:53.508953 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:12:53.509290 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:12:53.509509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:53.513125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:12:53.513125 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:53.513125 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:12:53.513125 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:12:53.513125 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:53.513125 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:12:53.513125 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:53.513125 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:12:53.513718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:53.513718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:12:53.513718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:12:53.513718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:12:53.513718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:12:53.513718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:12:53.513718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:12:53.513718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:53.513718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:12:53.513718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:53.513718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:12:53.513718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:53.513964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:12:53.513964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:53.513964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:12:53.513964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:53.513964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:12:53.513964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:12:53.513964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:53.513964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:12:53.513964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:53.513964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:12:53.513964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:12:53.513964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:53.513964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:12:53.514186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:53.514186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:12:53.514473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:53.514473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:12:53.514473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:12:53.514473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:12:53.514473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:12:53.514473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:12:53.514473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:12:53.514473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:12:53.514473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:12:53.514473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:12:53.514473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:12:53.514473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:12:53.514473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:12:53.514473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:12:53.514473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:12:53.514473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:12:53.515200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:12:53.515200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:12:53.515200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:53.515200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:12:53.515200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:53.515200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:12:53.515200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:53.515200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:12:53.515200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:53.515200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:12:53.515200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:53.515200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:12:53.515200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:12:53.515200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:53.515200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:12:53.515200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:53.515200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:12:53.515200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:12:53.515200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:53.515940 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:12:53.515940 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:53.515940 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:12:57.915942 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:12:57.916895 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:12:57.916895 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:12:57.918689 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:12:57.948312 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:12:57.948671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:57.949544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:12:57.949544 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:57.949544 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:12:57.949544 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:12:57.949544 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:57.949544 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:12:57.949544 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:57.949544 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:12:57.950821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.033 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:57.950821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:12:57.950821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:12:57.950821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:12:57.950821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:12:57.950821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:12:57.950821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:12:57.950821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:57.950821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:12:57.950821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:57.950821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:12:57.950821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:57.951148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:12:57.951148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:57.951148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:12:57.951148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:57.951148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:12:57.951148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:12:57.951148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:57.951148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:12:57.951148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:57.951148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:12:57.951148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:12:57.951148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:57.951148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:12:57.951467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:57.951467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:12:57.951895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:57.951895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:12:57.951895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:12:57.951895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:12:57.951895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:12:57.951895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:12:57.951895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:12:57.951895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:12:57.951895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:12:57.951895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:12:57.951895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:12:57.951895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:12:57.951895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:12:57.951895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:12:57.951895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:12:57.951895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:12:57.952299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:12:57.952299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:12:57.952299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:57.952299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:12:57.952299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:57.952299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:12:57.952299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:57.952299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:12:57.952299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:57.952299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:12:57.952299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:57.952299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:12:57.952299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:12:57.952299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:57.952299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:12:57.952299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:57.952299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:12:57.952299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:12:57.952299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:57.952786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:12:57.952786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:57.952786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:12:58.426664 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:12:58.427140 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:12:58.428063 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:12:58.430279 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:12:58.430527 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:12:58.430847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:58.431577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:12:58.431577 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:58.431577 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:12:58.431577 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:12:58.431577 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:58.431577 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:12:58.431577 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:58.431577 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:12:58.433069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:58.433069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:12:58.433069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:12:58.433069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:12:58.433069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:12:58.433069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:12:58.433069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:12:58.433069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:58.433069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:12:58.433069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:58.433069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:12:58.433069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:58.433596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:12:58.433596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:58.433596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:12:58.433596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:58.433596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:12:58.433596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:12:58.433596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:58.433596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:12:58.433596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:58.433596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:12:58.433596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:12:58.433596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:58.433596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:12:58.434158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:58.434158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:12:58.434701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:58.434701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:12:58.434701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:12:58.434701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:12:58.434701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:12:58.434701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:12:58.434701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:12:58.434701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:12:58.434701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:12:58.434701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:12:58.434701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:12:58.434701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:12:58.434701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:12:58.434701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:12:58.434701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:12:58.434701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:12:58.435404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:12:58.435404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:12:58.435404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:58.435404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:12:58.435404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:58.435404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:12:58.435404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:58.435404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:12:58.435404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:58.435404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:12:58.435404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:58.435404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:12:58.435404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:12:58.435404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:58.435404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:12:58.435404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:58.435404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:12:58.435404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:12:58.435404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:58.436179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:12:58.436179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:58.436179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:12:58.438261 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:12:58.438636 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:12:58.438922 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:12:58.438922 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:12:58.439304 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:12:58.439471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:58.439969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:12:58.439969 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:58.439969 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:12:58.439969 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:12:58.439969 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:58.439969 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:12:58.439969 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:58.439969 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:12:58.440700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:58.440700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:12:58.440700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:12:58.440700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:12:58.440700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:58.440700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:12:58.440700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:58.440700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:12:58.440700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:58.440700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:12:58.440700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:58.440700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:12:58.440700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:58.440700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:12:58.440700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:12:58.441556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:58.441556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:12:58.441556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:58.441556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:12:58.441556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:12:58.441556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:58.441556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:12:58.441556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:58.441556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:12:58.441874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:58.441874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:12:58.441874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:12:58.441874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:12:58.441874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:12:58.441874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:12:58.441874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:12:58.441874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:12:58.441874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:12:58.441874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:12:58.441874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:58.441874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:12:58.442186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:58.442186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:12:58.442186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:58.442186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:12:58.442186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:58.442186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:12:58.442186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:58.442186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:12:58.442186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:12:58.442186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:58.442186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:12:58.442186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:58.442186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:12:58.442186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:12:58.442547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:58.442547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:12:58.442547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:58.442547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:12:58.443663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:58.443663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:12:58.443663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:12:58.443663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:12:58.443663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:12:58.443663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:12:58.443663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:12:58.443663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:12:58.443663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:12:58.443663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:12:58.443663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:12:58.443663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:12:58.443663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:12:58.443663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:12:58.443663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:12:58.443663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:12:58.444559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:12:58.444559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:12:58.444559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:12:58.444559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:12:58.444559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:12:58.444559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:58.444559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:12:58.444559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:58.444559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:12:58.444559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:58.444559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:12:58.444559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:58.444559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:12:58.444559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:58.444559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:12:58.444559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:12:58.444559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:58.444559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:12:58.445331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:58.445331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:12:58.445331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:12:58.445331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:58.445331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:12:58.445331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:58.445331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:12:58.497541 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:12:58.497997 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:12:58.498223 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:12:58.498469 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:12:58.499201 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:12:58.499694 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:58.500509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:12:58.500509 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:58.500509 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:12:58.500509 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:12:58.500509 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:58.500509 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:12:58.500509 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:58.500509 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:12:58.501675 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:58.501675 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:12:58.501675 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:12:58.501675 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:12:58.501675 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:58.501675 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:12:58.501675 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:58.501675 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:12:58.501675 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:58.501675 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:12:58.501675 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:58.501675 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:12:58.501675 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:58.501675 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:12:58.501675 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:12:58.502167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:58.502167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:12:58.502167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:58.502167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:12:58.502167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:12:58.502167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:58.502167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:12:58.502167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:58.502167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:12:58.503151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:58.503151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:12:58.503151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:12:58.503151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:12:58.503151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:12:58.503151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:12:58.503151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:12:58.503151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:12:58.503151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:12:58.503151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:58.503151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:12:58.503532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:58.503532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:12:58.503532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:58.503532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:12:58.503532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:58.503532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:12:58.503532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:58.503532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:12:58.503532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:12:58.503532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:58.503532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:12:58.503532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:58.503532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:12:58.503532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:12:58.504222 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:58.504222 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:12:58.504222 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:58.504222 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:12:58.505097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:58.505097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:12:58.505097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:12:58.505097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:12:58.505097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:12:58.505097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:12:58.505097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:12:58.505097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:12:58.505097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:12:58.505097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:12:58.505097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:12:58.505097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:12:58.505097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:12:58.505097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:12:58.505097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:12:58.505097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:12:58.505097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:12:58.506276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:12:58.506276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:12:58.506276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:12:58.506276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:58.506276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:12:58.506276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:58.506276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:12:58.506276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:58.506276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:12:58.506276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:58.506276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:12:58.506276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:58.506276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:12:58.506276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:12:58.506276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:58.506276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:12:58.506276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:58.506276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:12:58.506276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:12:58.507294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:58.507294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:12:58.507294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:58.507294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:12:58.508422 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:12:58.508610 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:12:58.508734 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:12:58.508896 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:12:58.509065 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:12:58.509276 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:12:58.509435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:58.509829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:12:58.509829 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:58.509829 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:12:58.509829 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:12:58.509829 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:58.509829 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:12:58.509829 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:58.509829 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:12:58.510538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:58.510538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:12:58.510538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:12:58.510538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:12:58.510538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:12:58.510538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:12:58.510538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:12:58.510538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:58.510538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:12:58.510538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:58.510538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:12:58.510538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:58.511571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:12:58.511571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:58.511571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:12:58.511571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:58.511571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:12:58.511571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:12:58.511571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:58.511571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:12:58.511571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:58.511571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:12:58.511571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:12:58.511571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:58.511571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:12:58.511909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:58.511909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:12:58.511909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:58.511909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:12:58.511909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:12:58.511909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:12:58.511909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:12:58.511909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:12:58.511909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:12:58.511909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:12:58.511909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:12:58.511909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:12:58.511909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:12:58.511909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:12:58.511909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:12:58.512246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:12:58.512246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:12:58.512246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:12:58.512246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:12:58.512246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:12:58.512246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:58.512246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:12:58.512246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:12:58.512246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:12:58.512246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:12:58.512246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:12:58.512246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:12:58.512246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:12:58.512246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:12:58.512246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:12:58.512246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:12:58.512246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:12:58.512246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:12:58.512246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:12:58.512698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:12:58.512698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:12:58.512698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:12:58.512698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:12:58.512698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:12:58.512698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:13:02.916728 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:13:02.917930 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:13:02.917930 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:13:02.919255 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:13:02.956675 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:13:02.957479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:02.958208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:13:02.958208 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:02.958208 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:13:02.958208 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:13:02.958208 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:02.958208 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:13:02.958208 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:02.958208 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:13:02.958868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:02.958868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:13:02.958868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:13:02.958868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:13:02.958868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:13:02.958868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:13:02.958868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:13:02.958868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:02.958868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:13:02.958868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:02.958868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:13:02.958868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:02.959250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:13:02.959250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:02.959250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:13:02.959250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:02.959250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:13:02.959250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:13:02.959250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:02.959250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:13:02.959250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:02.959250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:13:02.959250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:13:02.959250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:02.959250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:13:02.959898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:02.959898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:13:02.959898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:02.959898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:13:02.959898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:13:02.959898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:13:02.959898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:13:02.959898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:13:02.959898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:13:02.959898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:13:02.959898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:13:02.959898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:13:02.959898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:13:02.959898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:13:02.959898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:13:02.960407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:13:02.960407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:13:02.960407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:13:02.960407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:13:02.960407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:13:02.960407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:02.960407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:13:02.960407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:02.960407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:13:02.960407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:02.960407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:13:02.960407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:02.960407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:13:02.960407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:02.960407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:13:02.960407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:13:02.960407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:02.960407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:13:02.960407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:02.961000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:13:02.961000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:13:02.961000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:02.961000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:13:02.961000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:02.961000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:13:03.427590 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:13:03.428205 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:13:03.428416 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:13:03.429509 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:13:03.429862 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:13:03.430113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:03.431185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:13:03.431185 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:03.431185 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:13:03.431185 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:13:03.431185 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:03.431185 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:13:03.431185 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:03.431185 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:13:03.432855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:03.432855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:13:03.432855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:13:03.432855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:13:03.432855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:13:03.432855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:13:03.432855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:13:03.432855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:03.432855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:13:03.432855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:03.432855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:13:03.432855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:03.433446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:13:03.433446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:03.433446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:13:03.433446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:03.433446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:13:03.433446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:13:03.433446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:03.433446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:13:03.433446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:03.433446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:13:03.433446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:13:03.433446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:03.433446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:13:03.434047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:03.434047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:13:03.434643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:03.434643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:13:03.434643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:13:03.434643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:13:03.434643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:13:03.434643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:13:03.434643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:13:03.434643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:13:03.434643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:13:03.434643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:13:03.434643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:13:03.434643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:13:03.434643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:13:03.434643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:13:03.434643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:13:03.434643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:13:03.435489 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:13:03.435489 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:13:03.435489 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:03.435489 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:13:03.435489 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:03.435489 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:13:03.435489 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:03.435489 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:13:03.435489 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:03.435489 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:13:03.435489 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:03.435489 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:13:03.435489 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:13:03.435489 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:03.435489 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:13:03.435489 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:03.435489 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:13:03.435489 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:13:03.435489 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:03.436204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:13:03.436204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:03.436204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:13:03.438662 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:13:03.439213 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:13:03.439448 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:13:03.439686 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:13:03.440126 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:13:03.440427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:03.441235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:13:03.441235 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:03.441235 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:13:03.441235 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:13:03.441235 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:03.441235 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:13:03.441235 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:03.441235 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:13:03.441976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:03.441976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:13:03.441976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:13:03.441976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:13:03.441976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:03.441976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:13:03.441976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:03.441976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:13:03.441976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:03.441976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:13:03.441976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:03.441976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:13:03.441976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:03.441976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:13:03.441976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:13:03.442418 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:03.442418 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:13:03.442418 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:03.442418 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:13:03.442418 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:13:03.442418 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:03.442418 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:13:03.442418 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:03.442418 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:13:03.442808 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:03.442808 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:13:03.442808 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:13:03.442808 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:13:03.442808 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:13:03.442808 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:13:03.442808 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:13:03.442808 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:13:03.442808 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:13:03.442808 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:13:03.442808 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:03.442808 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:13:03.443159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:03.443159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:13:03.443159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:03.443159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:13:03.443159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:03.443159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:13:03.443159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:03.443159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:13:03.443159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:13:03.443159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:03.443159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:13:03.443159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:03.443159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:13:03.443159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:13:03.443555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:03.443555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:13:03.443555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:03.443555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:13:03.443881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:03.443881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:13:03.443881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:13:03.443881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:13:03.443881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:13:03.443881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:13:03.443881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:13:03.443881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:13:03.443881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:13:03.443881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:13:03.443881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:13:03.443881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:13:03.443881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:13:03.443881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:13:03.443881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:13:03.443881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:13:03.444365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:13:03.444365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:13:03.444365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:13:03.444365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:13:03.444365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:13:03.444365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:03.444365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:13:03.444365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:03.444365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:13:03.444365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:03.444365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:13:03.444365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:03.444365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:13:03.444365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:03.444365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:13:03.444365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:13:03.444365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:03.444365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:13:03.444863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:03.444863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:13:03.444863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:13:03.444863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:03.444863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:13:03.444863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:03.444863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:13:03.498553 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:13:03.498965 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:13:03.499235 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:13:03.499434 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:13:03.500148 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:13:03.500502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:03.501330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:13:03.501330 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:03.501330 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:13:03.501330 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:13:03.501330 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:03.501330 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:13:03.501330 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:03.501330 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:13:03.502626 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:03.502626 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:13:03.502626 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:13:03.502626 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:13:03.502626 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:03.502626 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:13:03.502626 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:03.502626 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:13:03.502626 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:03.502626 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:13:03.502626 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:03.502626 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:13:03.502626 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:03.502626 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:13:03.502626 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:13:03.503388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:03.503388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:13:03.503388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:03.503388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:13:03.503388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:13:03.503388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:03.503388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:13:03.503388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:03.503388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:13:03.504240 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:03.504240 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:13:03.504240 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:13:03.504240 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:13:03.504240 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:13:03.504240 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:13:03.504240 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:13:03.504240 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:13:03.504240 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:13:03.504240 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:03.504240 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:13:03.504949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:03.504949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:13:03.504949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:03.504949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:13:03.504949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:03.504949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:13:03.504949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:03.504949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:13:03.504949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:13:03.504949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:03.504949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:13:03.504949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:03.504949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:13:03.504949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:13:03.505660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:03.505660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:13:03.505660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:03.505660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:13:03.506142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:03.506142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:13:03.506142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:13:03.506142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:13:03.506142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:13:03.506142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:13:03.506142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:13:03.506142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:13:03.506142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:13:03.506142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:13:03.506142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:13:03.506142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:13:03.506142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:13:03.506142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:13:03.506142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:13:03.506142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:13:03.506142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:13:03.507052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:13:03.507052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:13:03.507052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:13:03.507052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:03.507052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:13:03.507052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:03.507052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:13:03.507052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:03.507052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:13:03.507052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:03.507052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:13:03.507052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:03.507052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:13:03.507052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:13:03.507052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:03.507052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:13:03.507052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:03.507052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:13:03.507052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:13:03.508114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:03.508114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:13:03.508114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:03.508114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:13:03.509187 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:13:03.509564 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:13:03.509717 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:13:03.509958 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:13:03.510387 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:13:03.510760 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:13:03.511015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:03.511663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:13:03.511663 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:03.511663 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:13:03.511663 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:13:03.511663 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:03.511663 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:13:03.511663 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:03.511663 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:13:03.512524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:03.512524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:13:03.512524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:13:03.512524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:13:03.512524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:13:03.512524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:13:03.512524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:13:03.512524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:03.512524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:13:03.512524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:03.512524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:13:03.512524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:03.512970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:13:03.512970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:03.512970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:13:03.512970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:03.512970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:13:03.512970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:13:03.512970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:03.512970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:13:03.512970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:03.512970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:13:03.512970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:13:03.512970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:03.512970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:13:03.513307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:03.513307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:13:03.513521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:03.513521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:13:03.513521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:13:03.513521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:13:03.513521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:13:03.513521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:13:03.513521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:13:03.513521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:13:03.513521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:13:03.513521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:13:03.513521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:13:03.513521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:13:03.513521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:13:03.513521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:13:03.513521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:13:03.513521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:13:03.514061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:13:03.514061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:13:03.514061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:03.514061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:13:03.514061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:03.514061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:13:03.514061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:03.514061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:13:03.514061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:03.514061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:13:03.514061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:03.514061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:13:03.514061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:13:03.514061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:03.514061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:13:03.514061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:03.514061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:13:03.514061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:13:03.514061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:03.514543 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:13:03.514543 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:03.514543 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:13:07.917398 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:13:07.918414 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:13:07.918414 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:13:07.919473 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:13:07.959870 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:13:07.959870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:07.960225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:13:07.960225 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:07.960225 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:13:07.960225 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:13:07.960225 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:07.960225 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:13:07.960225 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:07.960225 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:13:07.960835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:07.960835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:13:07.960835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:13:07.960835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:13:07.960835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:13:07.960835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:13:07.960835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:13:07.960835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:07.960835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:13:07.960835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:07.960835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:13:07.960835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:07.961440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:13:07.961440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:07.961440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:13:07.961440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:07.961440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:13:07.961440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:13:07.961440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:07.961440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:13:07.961440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:07.961440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:13:07.961440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:13:07.961440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:07.961440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:13:07.962278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:07.962278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:13:07.962278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:07.962278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:13:07.962278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:13:07.962278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:13:07.962278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:13:07.962278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:13:07.962278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:13:07.962278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:13:07.962278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:13:07.962278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:13:07.962278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:13:07.962278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:13:07.962278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:13:07.963651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:13:07.963651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:13:07.963651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:13:07.963651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:13:07.963651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:13:07.963651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:07.963651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:13:07.963651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:07.963651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:13:07.963651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:07.963651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:13:07.963651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:07.963651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:13:07.963651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:07.963651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:13:07.963651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:13:07.963651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:07.963651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:13:07.963651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:07.965661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:13:07.965661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:13:07.965661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:07.965661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:13:07.965661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:07.965661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:13:08.428244 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:13:08.428944 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:13:08.429115 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:13:08.431360 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:13:08.431360 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:13:08.431360 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:08.431739 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:13:08.431739 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:08.431739 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:13:08.431739 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:13:08.431739 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:08.431739 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:13:08.431739 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:08.431739 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:13:08.433251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:08.433251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:13:08.433251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:13:08.433251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:13:08.433251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:13:08.433251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:13:08.433251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:13:08.433251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:08.433251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:13:08.433251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:08.433251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:13:08.433251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:08.433634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:13:08.433634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:08.433634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:13:08.433634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:08.433634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:13:08.433634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:13:08.433634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:08.433634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:13:08.433634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:08.433634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:13:08.433634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:13:08.433634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:08.433634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:13:08.434089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:08.434089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:13:08.435028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:08.435028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:13:08.435028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:13:08.435028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:13:08.435028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:13:08.435028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:13:08.435028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:13:08.435028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:13:08.435028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:13:08.435028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:13:08.435028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:13:08.435028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:13:08.435028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:13:08.435028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:13:08.435028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:13:08.435028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:13:08.436128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:13:08.436128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:13:08.436128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:08.436128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:13:08.436128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:08.436128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:13:08.436128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:08.436128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:13:08.436128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:08.436128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:13:08.436128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:08.436128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:13:08.436128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:13:08.436128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:08.436128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:13:08.436128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:08.436128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:13:08.436128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:13:08.436128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:08.437152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:13:08.437152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:08.437152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:13:08.439116 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:13:08.439639 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:13:08.439931 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:13:08.440194 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:13:08.440782 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:13:08.440945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:08.441662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:13:08.441662 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:08.441662 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:13:08.441662 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:13:08.441662 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:08.441662 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:13:08.441662 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:08.441662 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:13:08.442628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:08.442628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:13:08.442628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:13:08.442628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:13:08.442628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:08.442628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:13:08.442628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:08.442628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:13:08.442628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:08.442628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:13:08.442628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:08.442628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:13:08.442628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:08.442628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:13:08.442628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:13:08.443308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:08.443308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:13:08.443308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:08.443308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:13:08.443308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:13:08.443308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:08.443308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:13:08.443308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:08.443308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:13:08.443670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:08.443670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:13:08.443670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:13:08.443670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:13:08.443670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:13:08.443670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:13:08.443670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:13:08.443670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:13:08.443670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:13:08.443670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:13:08.443670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:08.443670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:13:08.444249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:08.444249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:13:08.444249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:08.444249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:13:08.444249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:08.444249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:13:08.444249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:08.444249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:13:08.444249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:13:08.444249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:08.444249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:13:08.444249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:08.444249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:13:08.444249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:13:08.444898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:08.444898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:13:08.444898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:08.444898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:13:08.444898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:08.444898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:13:08.444898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:13:08.444898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:13:08.444898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:13:08.444898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:13:08.444898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:13:08.444898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:13:08.444898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:13:08.444898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:13:08.445539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:13:08.445539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:13:08.445539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:13:08.445539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:13:08.445539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:13:08.445539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:13:08.445539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:13:08.445539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:13:08.445539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:13:08.445539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:13:08.445539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:13:08.445539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:08.445539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:13:08.445539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:08.445539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:13:08.445539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:08.445539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:13:08.445539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:08.445539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:13:08.446295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:08.446295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:13:08.446295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:13:08.446295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:08.446295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:13:08.446295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:08.446295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:13:08.446295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:13:08.446295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:08.446295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:13:08.446295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:08.446295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:13:08.499276 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:13:08.499957 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:13:08.500181 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:13:08.500427 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:13:08.501402 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:13:08.501883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:08.502697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:13:08.502697 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:08.502697 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:13:08.502697 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:13:08.502697 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:08.502697 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:13:08.502697 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:08.502697 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:13:08.503721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:08.503721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:13:08.503721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:13:08.503721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:13:08.503721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:08.503721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:13:08.503721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:08.503721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:13:08.503721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:08.503721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:13:08.503721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:08.503721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:13:08.503721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:08.503721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:13:08.503721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:13:08.504173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:08.504173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:13:08.504173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:08.504173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:13:08.504173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:13:08.504173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:08.504173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:13:08.504173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:08.504173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:13:08.504641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:08.504641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:13:08.504641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:13:08.504641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:13:08.504641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:13:08.504641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:13:08.504641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:13:08.504641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:13:08.504641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:13:08.504641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:08.504641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:13:08.505160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:08.505160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:13:08.505160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:08.505160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:13:08.505160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:08.505160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:13:08.505160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:08.505160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:13:08.505160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:13:08.505160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:08.505160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:13:08.505160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:08.505160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:13:08.505160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:13:08.505542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:08.505542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:13:08.505542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:08.505542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:13:08.505813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:08.505813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:13:08.505813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:13:08.505813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:13:08.505813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:13:08.505813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:13:08.505813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:13:08.505813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:13:08.505813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:13:08.505813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:13:08.505813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:13:08.505813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:13:08.505813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:13:08.505813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:13:08.505813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:13:08.505813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:13:08.505813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:13:08.506353 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:13:08.506353 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:13:08.506353 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:13:08.506353 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:08.506353 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:13:08.506353 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:08.506353 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:13:08.506353 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:08.506353 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:13:08.506353 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:08.506353 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:13:08.506353 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:08.506353 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:13:08.506353 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:13:08.506353 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:08.506353 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:13:08.506353 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:08.506353 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:13:08.506353 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:13:08.506882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:08.506882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:13:08.506882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:08.506882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:13:08.509696 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:13:08.510073 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:13:08.510188 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:13:08.510347 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:13:08.510940 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:13:08.511154 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:13:08.511318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:08.511953 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:13:08.511953 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:08.511953 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:13:08.511953 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:13:08.511953 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:08.511953 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:13:08.511953 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:08.511953 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:13:08.512958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:08.512958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:13:08.512958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:13:08.512958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:13:08.512958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:13:08.512958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:13:08.512958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:13:08.512958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:08.512958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:13:08.512958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:08.512958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:13:08.512958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:08.513668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:13:08.513668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:08.513668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:13:08.513668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:08.513668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:13:08.513668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:13:08.513668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:08.513668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:13:08.513668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:08.513668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:13:08.513668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:13:08.513668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:08.513668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:13:08.514057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:08.514057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:13:08.514057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:08.514057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:13:08.514057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:13:08.514057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:13:08.514057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:13:08.514057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:13:08.514057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:13:08.514057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:13:08.514057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:13:08.514057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:13:08.514057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:13:08.514057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:13:08.514057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:13:08.514490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:13:08.514490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:13:08.514490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:13:08.514490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:13:08.514490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:13:08.514490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:08.514490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:13:08.514490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:08.514490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:13:08.514490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:08.514490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:13:08.514490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:08.514490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:13:08.514490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:08.514490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:13:08.514490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:13:08.514490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:08.514490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:13:08.514490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:08.515173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:13:08.515173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:13:08.515173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:08.515173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:13:08.515173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:08.515173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:13:12.918155 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:13:12.918946 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:13:12.918946 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:13:12.920249 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:13:12.956115 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:13:12.956568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:12.957645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:13:12.957645 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:12.957645 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:13:12.957645 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:13:12.957645 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:12.957645 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:13:12.957645 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:12.957645 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:13:12.958499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:12.958499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:13:12.958499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:13:12.958499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:13:12.958499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:13:12.958499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:13:12.958499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:13:12.958499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:12.958499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:13:12.958499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:12.958499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:13:12.958499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:12.959084 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:13:12.959084 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:12.959084 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:13:12.959084 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:12.959084 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:13:12.959084 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:13:12.959084 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:12.959084 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:13:12.959084 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:12.959084 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:13:12.959084 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:13:12.959084 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:12.959084 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:13:12.959721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:12.959721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:13:12.959721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:12.959721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:13:12.959721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:13:12.959721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:13:12.959721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:13:12.959721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:13:12.959721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:13:12.959721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:13:12.959721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:13:12.959721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:13:12.959721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:13:12.959721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:13:12.959721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:13:12.960382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:13:12.960382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:13:12.960382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:13:12.960382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:13:12.960382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:13:12.960382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:12.960382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:13:12.960382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:12.960382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:13:12.960382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:12.960382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:13:12.960382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:12.960382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:13:12.960382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:12.960382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:13:12.960382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:13:12.960382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:12.960382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:13:12.960382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:12.961543 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:13:12.961543 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:13:12.961543 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:12.961543 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:13:12.961543 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:12.961543 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:13:13.429048 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:13:13.429759 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:13:13.430176 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:13:13.431665 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:13:13.432067 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:13:13.432391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:13.434119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:13:13.434119 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:13.434119 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:13:13.434119 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:13:13.434119 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:13.434119 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:13:13.434119 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:13.434119 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:13:13.435761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:13.435761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:13:13.435761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:13:13.435761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:13:13.435761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:13:13.435761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:13:13.435761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:13:13.435761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:13.435761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:13:13.435761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:13.435761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:13:13.435761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:13.436285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:13:13.436285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:13.436285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:13:13.436285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:13.436285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:13:13.436285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:13:13.436285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:13.436285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:13:13.436285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:13.436285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:13:13.436285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:13:13.436285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:13.436285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:13:13.436906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:13.436906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:13:13.437415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:13.437415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:13:13.437415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:13:13.437415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:13:13.437415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:13:13.437415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:13:13.437415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:13:13.437415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:13:13.437415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:13:13.437415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:13:13.437415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:13:13.437415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:13:13.437415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:13:13.437415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:13:13.437415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:13:13.437415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:13:13.439213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:13:13.439213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:13:13.439213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:13.439213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:13:13.439213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:13.439213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:13:13.439213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:13.439213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:13:13.439213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:13.439213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:13:13.439213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:13.439213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:13:13.439213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:13:13.439213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:13.439213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:13:13.439213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:13.439213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:13:13.439213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:13:13.439213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:13.440879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:13:13.440879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:13.440879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:13:13.440879 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:13:13.440879 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:13:13.440879 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:13:13.440879 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:13:13.440879 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:13:13.441256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:13.441905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:13:13.441905 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:13.441905 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:13:13.441905 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:13:13.441905 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:13.441905 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:13:13.441905 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:13.441905 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:13:13.442613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:13.442613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:13:13.442613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:13:13.442613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:13:13.442613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:13.442613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:13:13.442613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:13.442613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:13:13.442613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:13.442613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:13:13.442613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:13.442613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:13:13.442613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:13.442613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:13:13.442613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:13:13.443021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:13.443021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:13:13.443021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:13.443021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:13:13.443021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:13:13.443021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:13.443021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:13:13.443021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:13.443021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:13:13.443515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:13.443515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:13:13.443515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:13:13.443515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:13:13.443515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:13:13.443515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:13:13.443515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:13:13.443515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:13:13.443515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:13:13.443515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:13:13.443515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:13.443515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:13:13.443924 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:13.443924 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:13:13.443924 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:13.443924 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:13:13.443924 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:13.443924 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:13:13.443924 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:13.443924 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:13:13.443924 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:13:13.443924 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:13.443924 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:13:13.443924 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:13.443924 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:13:13.443924 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:13:13.444281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:13.444281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:13:13.444281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:13.444281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:13:13.444550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:13.444550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:13:13.444550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:13:13.444550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:13:13.444550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:13:13.444550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:13:13.444550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:13:13.444550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:13:13.444550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:13:13.444550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:13:13.444550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:13:13.444550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:13:13.444550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:13:13.444550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:13:13.444550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:13:13.444550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:13:13.445060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:13:13.445060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:13:13.445060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:13:13.445060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:13:13.445060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:13:13.445060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:13.445060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:13:13.445060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:13.445060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:13:13.445060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:13.445060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:13:13.445060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:13.445060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:13:13.445060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:13.445060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:13:13.445060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:13:13.445060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:13.445060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:13:13.445562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:13.445562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:13:13.445562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:13:13.445562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:13.445562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:13:13.445562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:13.445562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:13:13.499795 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:13:13.500380 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:13:13.500527 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:13:13.500938 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:13:13.501784 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:13:13.502713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:13.503684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:13:13.503684 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:13.503684 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:13:13.503684 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:13:13.503684 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:13.503684 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:13:13.503684 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:13.503684 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:13:13.505317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:13.505317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:13:13.505317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:13:13.505317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:13:13.505317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:13.505317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:13:13.505317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:13.505317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:13:13.505317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:13.505317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:13:13.505317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:13.505317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:13:13.505317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:13.505317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:13:13.505317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:13:13.505734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:13.505734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:13:13.505734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:13.505734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:13:13.505734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:13:13.505734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:13.505734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:13:13.505734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:13.505734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:13:13.507455 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:13.507455 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:13:13.507455 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:13:13.507455 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:13:13.507455 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:13:13.507455 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:13:13.507455 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:13:13.507455 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:13:13.507455 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:13:13.507455 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:13.507455 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:13:13.507991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:13.507991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:13:13.507991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:13.507991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:13:13.507991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:13.507991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:13:13.507991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:13.507991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:13:13.507991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:13:13.507991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:13.507991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:13:13.507991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:13.507991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:13:13.507991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:13:13.508537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:13.508537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:13:13.508537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:13.508537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:13:13.510067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:13.510067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:13:13.510067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:13:13.510067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:13:13.510067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:13:13.510067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:13:13.510067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:13:13.510067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:13:13.510067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:13:13.510067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:13:13.510067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:13:13.510067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:13:13.510067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:13:13.510067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:13:13.510067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:13:13.510067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:13:13.510067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:13:13.510557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:13:13.510557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:13:13.510557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:13:13.510557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:13.510557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:13:13.510557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:13.510557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:13:13.510557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:13.510557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:13:13.510557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:13.510557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:13:13.510557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:13.510557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:13:13.510557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:13:13.510557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:13.510557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:13:13.510557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:13.510557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:13:13.510557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:13:13.511062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:13.511062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:13:13.511062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:13.511062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:13:13.511062 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:13:13.511062 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:13:13.511226 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:13:13.511259 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:13:13.511650 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:13:13.511907 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:13:13.512059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:13.512565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:13:13.512565 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:13.512565 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:13:13.512565 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:13:13.512565 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:13.512565 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:13:13.512565 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:13.512565 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:13:13.513486 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:13.513486 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:13:13.513486 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:13:13.513486 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:13:13.513486 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:13:13.513486 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:13:13.513486 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:13:13.513486 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:13.513486 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:13:13.513486 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:13.513486 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:13:13.513486 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:13.513853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:13:13.513853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:13.513853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:13:13.513853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:13.513853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:13:13.513853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:13:13.513853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:13.513853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:13:13.513853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:13.513853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:13:13.513853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:13:13.513853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:13.513853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:13:13.514169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:13.514169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:13:13.514484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:13.514484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:13:13.514484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:13:13.514484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:13:13.514484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:13:13.514484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:13:13.514484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:13:13.514484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:13:13.514484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:13:13.514484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:13:13.514484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:13:13.514484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:13:13.514484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:13:13.514484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:13:13.514484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:13:13.514484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:13:13.514909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:13:13.514909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:13:13.514909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:13.514909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:13:13.514909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:13.514909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:13:13.514909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:13.514909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:13:13.514909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:13.514909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:13:13.514909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:13.514909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:13:13.514909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:13:13.514909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:13.514909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:13:13.514909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:13.514909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:13:13.514909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:13:13.514909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:13.515433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:13:13.515433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:13.515433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:13:17.918813 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:13:17.919894 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:13:17.919894 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:13:17.921478 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:13:17.957156 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:13:17.957722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:17.958662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:13:17.958662 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:17.958662 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:13:17.958662 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:13:17.958662 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:17.958662 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:13:17.958662 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:17.958662 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:13:17.959907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:17.959907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:13:17.959907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:13:17.959907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:13:17.959907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:13:17.959907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:13:17.959907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:13:17.959907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:17.959907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:13:17.959907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:17.959907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:13:17.959907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:17.960310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:13:17.960310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:17.960310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:13:17.960310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:17.960310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:13:17.960310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:13:17.960310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:17.960310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:13:17.960310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:17.960310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:13:17.960310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:13:17.960310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:17.960310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:13:17.960774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:17.960774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:13:17.960774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:17.960774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:13:17.960774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:13:17.960774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:13:17.960774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:13:17.960774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:13:17.960774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:13:17.960774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:13:17.960774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:13:17.960774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:13:17.960774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:13:17.960774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:13:17.960774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:13:17.961266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:13:17.961266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:13:17.961266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:13:17.961266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:13:17.961266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:13:17.961266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:17.961266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:13:17.961266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:17.961266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:13:17.961266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:17.961266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:13:17.961266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:17.961266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:13:17.961266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:17.961266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:13:17.961266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:13:17.961266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:17.961266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:13:17.961266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:17.961903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:13:17.961903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:13:17.961903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:17.961903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:13:17.961903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:17.961903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:13:18.429634 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:13:18.430193 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:13:18.430355 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:13:18.431476 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:13:18.431945 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:13:18.432286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:18.433242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:13:18.433242 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:18.433242 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:13:18.433242 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:13:18.433242 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:18.433242 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:13:18.433242 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:18.433242 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:13:18.434347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:18.434347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:13:18.434347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:13:18.434347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:13:18.434347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:13:18.434347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:13:18.434347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:13:18.434347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:18.434347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:13:18.434347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:18.434347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:13:18.434347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:18.434717 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:13:18.434717 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:18.434717 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:13:18.434717 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:18.434717 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:13:18.434717 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:13:18.434717 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:18.434717 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:13:18.434717 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:18.434717 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:13:18.434717 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:13:18.434717 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:18.434717 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:13:18.435049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:18.435049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:13:18.435441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:18.435441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:13:18.435441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:13:18.435441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:13:18.435441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:13:18.435441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:13:18.435441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:13:18.435441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:13:18.435441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:13:18.435441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:13:18.435441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:13:18.435441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:13:18.435441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:13:18.435441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:13:18.435441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:13:18.435441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:13:18.435967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:13:18.435967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:13:18.435967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:18.435967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:13:18.435967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:18.435967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:13:18.435967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:18.435967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:13:18.435967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:18.435967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:13:18.435967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:18.435967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:13:18.435967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:13:18.435967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:18.435967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:13:18.435967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:18.435967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:13:18.435967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:13:18.435967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:18.436494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:13:18.436494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:18.436494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:13:18.440141 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:13:18.440414 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:13:18.440512 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:13:18.440648 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:13:18.441190 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:13:18.441545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:18.442659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:13:18.442659 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:18.442659 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:13:18.442659 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:13:18.442659 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:18.442659 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:13:18.442659 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:18.442659 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:13:18.444146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:18.444146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:13:18.444146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:13:18.444146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:13:18.444146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:18.444146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:13:18.444146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:18.444146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:13:18.444146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:18.444146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:13:18.444146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:18.444146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:13:18.444146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:18.444146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:13:18.444146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:13:18.444668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:18.444668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:13:18.444668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:18.444668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:13:18.444668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:13:18.444668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:18.444668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:13:18.444668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:18.444668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:13:18.445713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:18.445713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:13:18.445713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:13:18.445713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:13:18.445713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:13:18.445713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:13:18.445713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:13:18.445713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:13:18.445713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:13:18.445713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:13:18.445713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:18.445713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:13:18.446111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:18.446111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:13:18.446111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:18.446111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:13:18.446111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:18.446111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:13:18.446111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:18.446111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:13:18.446111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:13:18.446111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:18.446111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:13:18.446111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:18.446111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:13:18.446111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:13:18.446542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:18.446542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:13:18.446542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:18.446542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:13:18.448123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:18.448123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:13:18.448123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:13:18.448123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:13:18.448123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:13:18.448123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:13:18.448123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:13:18.448123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:13:18.448123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:13:18.448123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:13:18.448123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:13:18.448123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:13:18.448123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:13:18.448123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:13:18.448123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:13:18.448123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:13:18.448623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:13:18.448623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:13:18.448623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:13:18.448623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:13:18.448623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:13:18.448623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:18.448623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:13:18.448623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:18.448623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:13:18.448623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:18.448623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:13:18.448623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:18.448623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:13:18.448623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:18.448623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:13:18.448623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:13:18.448623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:18.448623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:13:18.449197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:18.449197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:13:18.449197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:13:18.449197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:18.449197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:13:18.449197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:18.449197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:13:18.500462 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:13:18.500943 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:13:18.501139 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:13:18.501349 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:13:18.502315 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:13:18.502833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:18.503790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:13:18.503790 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:18.503790 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:13:18.503790 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:13:18.503790 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:18.503790 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:13:18.503790 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:18.503790 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:13:18.505132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:18.505132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:13:18.505132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:13:18.505132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:13:18.505132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:18.505132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:13:18.505132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:18.505132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:13:18.505132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:18.505132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:13:18.505132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:18.505132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:13:18.505132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:18.505132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:13:18.505132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:13:18.505449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:18.505449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:13:18.505449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:18.505449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:13:18.505449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:13:18.505449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:18.505449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:13:18.505449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:18.505449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:13:18.506763 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:18.506763 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:13:18.506763 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:13:18.506763 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:13:18.506763 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:13:18.506763 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:13:18.506763 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:13:18.506763 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:13:18.506763 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:13:18.506763 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:18.506763 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:13:18.507022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:18.507022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:13:18.507022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:18.507022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:13:18.507022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:18.507022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:13:18.507022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:18.507022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:13:18.507022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:13:18.507022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:18.507022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:13:18.507022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:18.507022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:13:18.507022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:13:18.507276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:18.507276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:13:18.507276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:18.507276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:13:18.508665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:18.508665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:13:18.508665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:13:18.508665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:13:18.508665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:13:18.508665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:13:18.508665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:13:18.508665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:13:18.508665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:13:18.508665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:13:18.508665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:13:18.508665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:13:18.508665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:13:18.508665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:13:18.508665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:13:18.508665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:13:18.508665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:13:18.509192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:13:18.509192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:13:18.509192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:13:18.509192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:18.509192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:13:18.509192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:18.509192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:13:18.509192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:18.509192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:13:18.509192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:18.509192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:13:18.509192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:18.509192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:13:18.509192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:13:18.509192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:18.509192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:13:18.509192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:18.509192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:13:18.509192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:13:18.509673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:18.509673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:13:18.509673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:18.509673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:13:18.510991 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:13:18.511371 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:13:18.511558 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:13:18.511832 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:13:18.512440 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:13:18.512809 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:13:18.513015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:18.513729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:13:18.513729 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:18.513729 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:13:18.513729 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:13:18.513729 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:18.513729 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:13:18.513729 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:18.513729 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:13:18.514375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:18.514375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:13:18.514375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:13:18.514375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:13:18.514375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:13:18.514375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:13:18.514375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:13:18.514375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:18.514375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:13:18.514375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:18.514375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:13:18.514375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:18.514861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:13:18.514861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:18.514861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:13:18.514861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:18.514861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:13:18.514861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:13:18.514861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:18.514861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:13:18.514861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:18.514861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:13:18.514861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:13:18.514861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:18.514861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:13:18.515178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:18.515178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:13:18.515349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:18.515349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:13:18.515349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:13:18.515349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:13:18.515349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:13:18.515349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:13:18.515349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:13:18.515349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:13:18.515349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:13:18.515349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:13:18.515349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:13:18.515349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:13:18.515349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:13:18.515349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:13:18.515349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:13:18.515349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:13:18.516019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:13:18.516019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:13:18.516019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:18.516019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:13:18.516019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:18.516019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:13:18.516019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:18.516019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:13:18.516019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:18.516019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:13:18.516019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:18.516019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:13:18.516019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:13:18.516019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:18.516019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:13:18.516019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:18.516019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:13:18.516019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:13:18.516019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:18.516780 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:13:18.516780 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:18.516780 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:13:22.919641 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:13:22.921163 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:13:22.921163 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:13:22.921903 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:13:22.962277 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:13:22.962682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:22.963722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:13:22.963722 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:22.963722 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:13:22.963722 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:13:22.963722 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:22.963722 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:13:22.963722 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:22.963722 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:13:22.964893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.043 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:22.964893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:13:22.964893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:13:22.964893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:13:22.964893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:13:22.964893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:13:22.964893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:13:22.964893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:22.964893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:13:22.964893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:22.964893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:13:22.964893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:22.965732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:13:22.965732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:22.965732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:13:22.965732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:22.965732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:13:22.965732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:13:22.965732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:22.965732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:13:22.965732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:22.965732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:13:22.965732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:13:22.965732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:22.965732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:13:22.966376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:22.966376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:13:22.966376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:22.966376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:13:22.966376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:13:22.966376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:13:22.966376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:13:22.966376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:13:22.966376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:13:22.966376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:13:22.966376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:13:22.966376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:13:22.966376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:13:22.966376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:13:22.966376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:13:22.967566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:13:22.967566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:13:22.967566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:13:22.967566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:13:22.967566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:13:22.967566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:22.967566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:13:22.967566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:22.967566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:13:22.967566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:22.967566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:13:22.967566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:22.967566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:13:22.967566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:22.967566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:13:22.967566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:13:22.967566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:22.967566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:13:22.967566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:22.968597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:13:22.968597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:13:22.968597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:22.968597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:13:22.968597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:22.968597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:13:23.430272 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:13:23.430853 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:13:23.430965 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:13:23.432179 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:13:23.432593 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:13:23.432899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:23.433974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:13:23.433974 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:23.433974 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:13:23.433974 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:13:23.433974 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:23.433974 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:13:23.433974 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:23.433974 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:13:23.435176 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:23.435176 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:13:23.435176 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:13:23.435176 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:13:23.435176 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:13:23.435176 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:13:23.435176 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:13:23.435176 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:23.435176 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:13:23.435176 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:23.435176 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:13:23.435176 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:23.435885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:13:23.435885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:23.435885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:13:23.435885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:23.435885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:13:23.435885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:13:23.435885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:23.435885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:13:23.435885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:23.435885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:13:23.435885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:13:23.435885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:23.435885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:13:23.436480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:23.436480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:13:23.436480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:23.436480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:13:23.436480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:13:23.436480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:13:23.436480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:13:23.436480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:13:23.436480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:13:23.436480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:13:23.436480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:13:23.436480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:13:23.436480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:13:23.436480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:13:23.436480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:13:23.437308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:13:23.437308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:13:23.437308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:13:23.437308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:13:23.437308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:13:23.437308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:23.437308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:13:23.437308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:23.437308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:13:23.437308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:23.437308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:13:23.437308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:23.437308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:13:23.437308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:23.437308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:13:23.437308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:13:23.437308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:23.437308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:13:23.437308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:23.438224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:13:23.438224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:13:23.438224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:23.438224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:13:23.438224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:23.438224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:13:23.440636 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:13:23.440903 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:13:23.441035 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:13:23.441176 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:13:23.441684 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:13:23.442027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:23.442799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:13:23.442799 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:23.442799 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:13:23.442799 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:13:23.442799 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:23.442799 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:13:23.442799 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:23.442799 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:13:23.443807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:23.443807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:13:23.443807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:13:23.443807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:13:23.443807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:23.443807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:13:23.443807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:23.443807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:13:23.443807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:23.443807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:13:23.443807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:23.443807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:13:23.443807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:23.443807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:13:23.443807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:13:23.444329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:23.444329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:13:23.444329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:23.444329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:13:23.444329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:13:23.444329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:23.444329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:13:23.444329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:23.444329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:13:23.444825 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:23.444825 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:13:23.444825 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:13:23.444825 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:13:23.444825 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:13:23.444825 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:13:23.444825 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:13:23.444825 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:13:23.444825 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:13:23.444825 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:13:23.444825 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:23.444825 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:13:23.445286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:23.445286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:13:23.445286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:23.445286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:13:23.445286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:23.445286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:13:23.445286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:23.445286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:13:23.445286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:13:23.445286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:23.445286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:13:23.445286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:23.445286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:13:23.445286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:13:23.445713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:23.445713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:13:23.445713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:23.445713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:13:23.445921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:23.445921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:13:23.445921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:13:23.445921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:13:23.445921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:13:23.445921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:13:23.445921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:13:23.445921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:13:23.445921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:13:23.445921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:13:23.445921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:13:23.445921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:13:23.445921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:13:23.445921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:13:23.445921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:13:23.445921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:13:23.446484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:13:23.446484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:13:23.446484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:13:23.446484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:13:23.446484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:13:23.446484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:23.446484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:13:23.446484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:23.446484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:13:23.446484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:23.446484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:13:23.446484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:23.446484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:13:23.446484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:23.446484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:13:23.446484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:13:23.446484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:23.446484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:13:23.447216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:23.447216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:13:23.447216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:13:23.447216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:23.447216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:13:23.447216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:23.447216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:13:23.501169 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:13:23.501446 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:13:23.501537 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:13:23.501698 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:13:23.502166 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:13:23.502500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:23.503021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:13:23.503021 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:23.503021 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:13:23.503021 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:13:23.503021 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:23.503021 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:13:23.503021 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:23.503021 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:13:23.503834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:23.503834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:13:23.503834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:13:23.503834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:13:23.503834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:23.503834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:13:23.503834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:23.503834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:13:23.503834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:23.503834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:13:23.503834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:23.503834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:13:23.503834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:23.503834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:13:23.503834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:13:23.504514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:23.504514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:13:23.504514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:23.504514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:13:23.504514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:13:23.504514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:23.504514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:13:23.504514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:23.504514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:13:23.504935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:23.504935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:13:23.504935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:13:23.504935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:13:23.504935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:13:23.504935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:13:23.504935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:13:23.504935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:13:23.504935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:13:23.504935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:23.504935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:13:23.505456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:23.505456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:13:23.505456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:23.505456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:13:23.505456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:23.505456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:13:23.505456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:23.505456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:13:23.505456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:13:23.505456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:23.505456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:13:23.505456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:23.505456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:13:23.505456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:13:23.506084 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:23.506084 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:13:23.506084 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:23.506084 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:13:23.506084 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:23.506084 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:13:23.506084 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:13:23.506084 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:13:23.506084 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:13:23.506084 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:13:23.506084 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:13:23.506084 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:13:23.506084 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:13:23.506084 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:13:23.506689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:13:23.506689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:13:23.506689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:13:23.506689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:13:23.506689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:13:23.506689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:13:23.506689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:13:23.506689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:13:23.506689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:13:23.506689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:13:23.506689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:23.506689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:13:23.506689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:23.506689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:13:23.506689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:23.506689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:13:23.506689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:23.506689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:13:23.506689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:23.507677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:13:23.507677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:13:23.507677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:23.507677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:13:23.507677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:23.507677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:13:23.507677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:13:23.507677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:23.507677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:13:23.507677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:23.507677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:13:23.511539 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:13:23.511870 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:13:23.511985 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:13:23.512261 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:13:23.512586 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:13:23.513012 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:13:23.513275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:23.513994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:13:23.513994 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:23.513994 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:13:23.513994 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:13:23.513994 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:23.513994 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:13:23.513994 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:23.513994 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:13:23.514794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:23.514794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:13:23.514794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:13:23.514794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:13:23.514794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:13:23.514794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:13:23.514794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:13:23.514794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:23.514794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:13:23.514794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:23.514794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:13:23.514794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:23.515175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:13:23.515175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:23.515175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:13:23.515175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:23.515175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:13:23.515175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:13:23.515175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:23.515175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:13:23.515175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:23.515175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:13:23.515175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:13:23.515175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:23.515175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:13:23.515565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:23.515565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:13:23.515843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:23.515843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:13:23.515843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:13:23.515843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:13:23.515843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:13:23.515843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:13:23.515843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:13:23.515843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:13:23.515843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:13:23.515843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:13:23.515843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:13:23.515843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:13:23.515843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:13:23.515843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:13:23.515843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:13:23.515843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:13:23.516311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:13:23.516311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:13:23.516311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:23.516311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:13:23.516311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:23.516311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:13:23.516311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:23.516311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:13:23.516311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:23.516311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:13:23.516311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:23.516311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:13:23.516311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:13:23.516311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:23.516311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:13:23.516311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:23.516311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:13:23.516311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:13:23.516311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:23.516879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:13:23.516879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:23.516879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:13:27.920199 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:13:27.920828 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:13:27.920905 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:13:27.921841 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:13:27.957436 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:13:27.958227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:27.959207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:13:27.959207 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:27.959207 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:13:27.959207 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:13:27.959207 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:27.959207 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:13:27.959207 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:27.959207 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:13:27.960496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:27.960496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:13:27.960496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:13:27.960496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:13:27.960496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:13:27.960496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:13:27.960496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:13:27.960496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:27.960496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:13:27.960496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:27.960496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:13:27.960496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:27.961108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:13:27.961108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:27.961108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:13:27.961108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:27.961108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:13:27.961108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:13:27.961108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:27.961108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:13:27.961108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:27.961108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:13:27.961108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:13:27.961108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:27.961108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:13:27.961740 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:27.961740 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:13:27.962094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:27.962094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:13:27.962094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:13:27.962094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:13:27.962094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:13:27.962094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:13:27.962094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:13:27.962094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:13:27.962094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:13:27.962094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:13:27.962094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:13:27.962094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:13:27.962094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:13:27.962094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:13:27.962094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:13:27.962094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:13:27.962856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:13:27.962856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:13:27.962856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:27.962856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:13:27.962856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:27.962856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:13:27.962856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:27.962856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:13:27.962856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:27.962856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:13:27.962856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:27.962856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:13:27.962856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:13:27.962856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:27.962856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:13:27.962856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:27.962856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:13:27.962856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:13:27.962856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:27.963509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:13:27.963509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:27.963509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:13:28.430964 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:13:28.431428 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:13:28.431640 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:13:28.433227 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:13:28.433631 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:13:28.433911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:28.435299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:13:28.435299 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:28.435299 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:13:28.435299 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:13:28.435299 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:28.435299 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:13:28.435299 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:28.435299 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:13:28.437184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:28.437184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:13:28.437184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:13:28.437184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:13:28.437184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:13:28.437184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:13:28.437184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:13:28.437184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:28.437184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:13:28.437184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:28.437184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:13:28.437184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:28.437793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:13:28.437793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:28.437793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:13:28.437793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:28.437793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:13:28.437793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:13:28.437793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:28.437793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:13:28.437793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:28.437793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:13:28.437793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:13:28.437793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:28.437793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:13:28.438355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:28.438355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:13:28.438929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:28.438929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:13:28.438929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:13:28.438929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:13:28.438929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:13:28.438929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:13:28.438929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:13:28.438929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:13:28.438929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:13:28.438929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:13:28.438929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:13:28.438929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:13:28.438929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:13:28.438929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:13:28.438929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:13:28.438929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:13:28.439612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:13:28.439612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:13:28.439612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:28.439612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:13:28.439612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:28.439612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:13:28.439612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:28.439612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:13:28.439612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:28.439612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:13:28.439612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:28.439612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:13:28.439612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:13:28.439612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:28.439612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:13:28.439612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:28.439612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:13:28.439612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:13:28.439612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:28.440402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:13:28.440402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:28.440402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:13:28.441013 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:13:28.441301 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:13:28.441420 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:13:28.441549 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:13:28.442138 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:13:28.442520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:28.443344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:13:28.443344 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:28.443344 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:13:28.443344 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:13:28.443344 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:28.443344 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:13:28.443344 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:28.443344 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:13:28.444119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:28.444119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:13:28.444119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:13:28.444119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:13:28.444119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:28.444119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:13:28.444119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:28.444119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:13:28.444119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:28.444119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:13:28.444119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:28.444119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:13:28.444119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:28.444119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:13:28.444119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:13:28.444546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:28.444546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:13:28.444546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:28.444546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:13:28.444546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:13:28.444546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:28.444546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:13:28.444546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:28.444546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:13:28.444944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:28.444944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:13:28.444944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:13:28.444944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:13:28.444944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:13:28.444944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:13:28.444944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:13:28.444944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:13:28.444944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:13:28.444944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:13:28.444944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:28.444944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:13:28.445295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:28.445295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:13:28.445295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:28.445295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:13:28.445295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:28.445295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:13:28.445295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:28.445295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:13:28.445295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:13:28.445295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:28.445295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:13:28.445295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:28.445295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:13:28.445295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:13:28.445645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:28.445645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:13:28.445645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:28.445645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:13:28.445939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:28.445939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:13:28.445939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:13:28.445939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:13:28.445939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:13:28.445939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:13:28.445939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:13:28.445939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:13:28.445939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:13:28.445939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:13:28.445939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:13:28.445939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:13:28.445939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:13:28.445939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:13:28.445939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:13:28.445939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:13:28.446380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:13:28.446380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:13:28.446380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:13:28.446380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:13:28.446380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:13:28.446380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:28.446380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:13:28.446380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:28.446380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:13:28.446380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:28.446380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:13:28.446380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:28.446380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:13:28.446380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:28.446380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:13:28.446380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:13:28.446380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:28.446380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:13:28.446954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:28.446954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:13:28.446954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:13:28.446954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:28.446954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:13:28.446954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:28.446954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:13:28.501632 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:13:28.501955 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:13:28.502199 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:13:28.502431 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:13:28.503400 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:13:28.503898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:28.504857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:13:28.504857 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:28.504857 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:13:28.504857 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:13:28.504857 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:28.504857 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:13:28.504857 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:28.504857 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:13:28.506313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:28.506313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:13:28.506313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:13:28.506313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:13:28.506313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:28.506313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:13:28.506313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:28.506313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:13:28.506313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:28.506313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:13:28.506313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:28.506313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:13:28.506313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:28.506313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:13:28.506313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:13:28.507044 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:28.507044 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:13:28.507044 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:28.507044 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:13:28.507044 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:13:28.507044 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:28.507044 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:13:28.507044 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:28.507044 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:13:28.507762 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:28.507762 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:13:28.507762 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:13:28.507762 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:13:28.507762 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:13:28.507762 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:13:28.507762 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:13:28.507762 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:13:28.507762 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:13:28.507762 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:28.507762 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:13:28.508305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:28.508305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:13:28.508305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:28.508305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:13:28.508305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:28.508305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:13:28.508305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:28.508305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:13:28.508305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:13:28.508305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:28.508305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:13:28.508305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:28.508305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:13:28.508305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:13:28.508945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:28.508945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:13:28.508945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:28.508945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:13:28.509456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:28.509456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:13:28.509456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:13:28.509456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:13:28.509456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:13:28.509456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:13:28.509456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:13:28.509456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:13:28.509456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:13:28.509456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:13:28.509456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:13:28.509456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:13:28.509456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:13:28.509456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:13:28.509456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:13:28.509456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:13:28.509456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:13:28.510345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:13:28.510345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:13:28.510345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:13:28.510345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:28.510345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:13:28.510345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:28.510345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:13:28.510345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:28.510345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:13:28.510345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:28.510345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:13:28.510345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:28.510345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:13:28.510345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:13:28.510345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:28.510345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:13:28.510345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:28.510345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:13:28.510345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:13:28.511041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:28.511041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:13:28.511041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:28.511041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:13:28.512071 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:13:28.512288 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:13:28.512398 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:13:28.512550 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:13:28.513087 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:13:28.513450 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:13:28.513680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:28.514488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:13:28.514488 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:28.514488 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:13:28.514488 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:13:28.514488 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:28.514488 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:13:28.514488 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:28.514488 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:13:28.515329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:28.515329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:13:28.515329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:13:28.515329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:13:28.515329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:13:28.515329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:13:28.515329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:13:28.515329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:28.515329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:13:28.515329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:28.515329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:13:28.515329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:28.515657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:13:28.515657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:28.515657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:13:28.515657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:28.515657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:13:28.515657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:13:28.515657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:28.515657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:13:28.515657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:28.515657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:13:28.515657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:13:28.515657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:28.515657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:13:28.516035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:28.516035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:13:28.516288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:28.516288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:13:28.516288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:13:28.516288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:13:28.516288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:13:28.516288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:13:28.516288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:13:28.516288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:13:28.516288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:13:28.516288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:13:28.516288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:13:28.516288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:13:28.516288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:13:28.516288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:13:28.516288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:13:28.516288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:13:28.516717 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:13:28.516717 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:13:28.516717 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:28.516717 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:13:28.516717 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:28.516717 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:13:28.516717 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:28.516717 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:13:28.516717 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:28.516717 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:13:28.516717 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:28.516717 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:13:28.516717 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:13:28.516717 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:28.516717 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:13:28.516717 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:28.516717 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:13:28.516717 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:13:28.516717 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:28.517221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:13:28.517221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:28.517221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:13:32.921240 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:13:32.922307 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:13:32.922307 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:13:32.922920 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:13:32.958533 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:13:32.959029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:32.960050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:13:32.960050 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:32.960050 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:13:32.960050 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:13:32.960050 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:32.960050 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:13:32.960050 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:32.960050 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:13:32.961588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:32.961588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:13:32.961588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:13:32.961588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:13:32.961588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:13:32.961588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:13:32.961588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:13:32.961588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:32.961588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:13:32.961588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:32.961588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:13:32.961588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:32.962709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:13:32.962709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:32.962709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:13:32.962709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:32.962709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:13:32.962709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:13:32.962709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:32.962709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:13:32.962709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:32.962709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:13:32.962709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:13:32.962709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:32.962709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:13:32.965696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:32.965696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:13:32.965696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:32.965696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:13:32.965696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:13:32.965696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:13:32.965696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:13:32.965696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:13:32.965696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:13:32.965696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:13:32.965696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:13:32.965696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:13:32.965696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:13:32.965696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:13:32.965696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:13:32.966662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:13:32.966662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:13:32.966662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:13:32.966662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:13:32.966662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:13:32.966662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:32.966662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:13:32.966662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:32.966662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:13:32.966662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:32.966662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:13:32.966662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:32.966662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:13:32.966662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:32.966662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:13:32.966662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:13:32.966662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:32.966662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:13:32.966662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:32.967478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:13:32.967478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:13:32.967478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:32.967478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:13:32.967478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:32.967478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:13:33.431856 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:13:33.432460 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:13:33.432737 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:13:33.433765 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:13:33.434128 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:13:33.434358 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:33.435758 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:13:33.435758 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:33.435758 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:13:33.435758 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:13:33.435758 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:33.435758 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:13:33.435758 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:33.435758 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:13:33.437422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:33.437422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:13:33.437422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:13:33.437422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:13:33.437422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:13:33.437422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:13:33.437422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:13:33.437422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:33.437422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:13:33.437422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:33.437422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:13:33.437422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:33.438137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:13:33.438137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:33.438137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:13:33.438137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:33.438137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:13:33.438137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:13:33.438137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:33.438137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:13:33.438137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:33.438137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:13:33.438137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:13:33.438137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:33.438137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:13:33.438809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:33.438809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:13:33.440326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:33.440326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:13:33.440326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:13:33.440326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:13:33.440326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:13:33.440326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:13:33.440326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:13:33.440326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:13:33.440326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:13:33.440326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:13:33.440326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:13:33.440326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:13:33.440326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:13:33.440326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:13:33.440326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:13:33.440326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:13:33.441242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:13:33.441242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:13:33.441242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:33.441242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:13:33.441242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:33.441242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:13:33.441242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:33.441242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:13:33.441242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:33.441242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:13:33.441242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:33.441242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:13:33.441242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:13:33.441242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:33.441242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:13:33.441242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:33.441242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:13:33.441242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:13:33.441242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:33.442102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:13:33.442102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:33.442102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:13:33.442646 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:13:33.443051 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:13:33.443162 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:13:33.443272 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:13:33.443730 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:13:33.444030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:33.445003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:13:33.445003 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:33.445003 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:13:33.445003 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:13:33.445003 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:33.445003 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:13:33.445003 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:33.445003 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:13:33.446056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:33.446056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:13:33.446056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:13:33.446056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:13:33.446056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:33.446056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:13:33.446056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:33.446056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:13:33.446056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:33.446056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:13:33.446056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:33.446056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:13:33.446056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:33.446056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:13:33.446056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:13:33.446553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:33.446553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:13:33.446553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:33.446553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:13:33.446553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:13:33.446553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:33.446553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:13:33.446553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:33.446553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:13:33.446927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:33.446927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:13:33.446927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:13:33.446927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:13:33.446927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:13:33.446927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:13:33.446927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:13:33.446927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:13:33.446927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:13:33.446927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:13:33.446927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:33.446927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:13:33.447292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:33.447292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:13:33.447292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:33.447292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:13:33.447292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:33.447292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:13:33.447292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:33.447292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:13:33.447292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:13:33.447292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:33.447292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:13:33.447292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:33.447292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:13:33.447292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:13:33.447664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:33.447664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:13:33.447664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:33.447664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:13:33.448594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:33.448594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:13:33.448594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:13:33.448594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:13:33.448594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:13:33.448594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:13:33.448594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:13:33.448594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:13:33.448594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:13:33.448594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:13:33.448594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:13:33.448594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:13:33.448594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:13:33.448594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:13:33.448594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:13:33.448594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:13:33.449498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:13:33.449498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:13:33.449498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:13:33.449498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:13:33.449498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:13:33.449498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:33.449498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:13:33.449498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:33.449498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:13:33.449498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:33.449498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:13:33.449498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:33.449498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:13:33.449498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:33.449498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:13:33.449498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:13:33.449498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:33.449498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:13:33.450121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:33.450121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:13:33.450121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:13:33.450121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:33.450121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:13:33.450121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:33.450121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:13:33.502313 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:13:33.502786 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:13:33.503100 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:13:33.503422 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:13:33.504231 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:13:33.504636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:33.505484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:13:33.505484 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:33.505484 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:13:33.505484 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:13:33.505484 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:33.505484 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:13:33.505484 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:33.505484 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:13:33.506512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:33.506512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:13:33.506512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:13:33.506512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:13:33.506512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:33.506512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:13:33.506512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:33.506512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:13:33.506512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:33.506512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:13:33.506512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:33.506512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:13:33.506512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:33.506512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:13:33.506512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:13:33.507163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:33.507163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:13:33.507163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:33.507163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:13:33.507163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:13:33.507163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:33.507163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:13:33.507163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:33.507163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:13:33.507776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:33.507776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:13:33.507776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:13:33.507776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:13:33.507776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:13:33.507776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:13:33.507776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:13:33.507776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:13:33.507776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:13:33.507776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:33.507776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:13:33.508291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:33.508291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:13:33.508291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:33.508291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:13:33.508291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:33.508291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:13:33.508291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:33.508291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:13:33.508291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:13:33.508291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:33.508291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:13:33.508291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:33.508291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:13:33.508291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:13:33.508861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:33.508861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:13:33.508861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:33.508861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:13:33.509809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:33.509809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:13:33.509809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:13:33.509809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:13:33.509809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:13:33.509809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:13:33.509809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:13:33.509809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:13:33.509809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:13:33.509809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:13:33.509809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:13:33.509809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:13:33.509809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:13:33.509809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:13:33.509809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:13:33.509809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:13:33.509809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:13:33.510495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:13:33.510495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:13:33.510495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:13:33.510495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:33.510495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:13:33.510495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:33.510495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:13:33.510495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:33.510495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:13:33.510495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:33.510495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:13:33.510495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:33.510495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:13:33.510495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:13:33.510495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:33.510495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:13:33.510495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:33.510495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:13:33.510495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:13:33.511065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:33.511065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:13:33.511065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:33.511065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:13:33.512451 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:13:33.514113 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:13:33.514262 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:13:33.514464 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:13:33.514989 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:13:33.515332 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:13:33.515552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:33.516305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:13:33.516305 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:33.516305 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:13:33.516305 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:13:33.516305 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:33.516305 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:13:33.516305 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:33.516305 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:13:33.517391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:33.517391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:13:33.517391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:13:33.517391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:13:33.517391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:13:33.517391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:13:33.517391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:13:33.517391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:33.517391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:13:33.517391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:33.517391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:13:33.517391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:33.517790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:13:33.517790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:33.517790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:13:33.517790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:33.517790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:13:33.517790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:13:33.517790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:33.517790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:13:33.517790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:33.517790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:13:33.517790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:13:33.517790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:33.517790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:13:33.518171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:33.518171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:13:33.518385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:33.518385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:13:33.518385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:13:33.518385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:13:33.518385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:13:33.518385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:13:33.518385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:13:33.518385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:13:33.518385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:13:33.518385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:13:33.518385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:13:33.518385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:13:33.518385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:13:33.518385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:13:33.518385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:13:33.518385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:13:33.518859 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:13:33.518859 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:13:33.518859 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:33.518859 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:13:33.518859 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:33.518859 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:13:33.518859 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:33.518859 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:13:33.518859 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:33.518859 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:13:33.518859 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:33.518859 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:13:33.518859 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:13:33.518859 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:33.518859 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:13:33.518859 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:33.518859 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:13:33.518859 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:13:33.518859 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:33.519382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:13:33.519382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:33.519382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:13:37.921871 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:13:37.922584 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:13:37.922674 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:13:37.924077 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:13:37.963884 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:13:37.963884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:37.964478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:13:37.964478 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:37.964478 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:13:37.964478 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:13:37.964478 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:37.964478 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:13:37.964478 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:37.964478 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:13:37.965255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:37.965255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:13:37.965255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:13:37.965255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:13:37.965255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:13:37.965255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:13:37.965255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:13:37.965255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:37.965255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:13:37.965255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:37.965255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:13:37.965255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:37.965842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:13:37.965842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:37.965842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:13:37.965842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:37.965842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:13:37.965842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:13:37.965842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:37.965842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:13:37.965842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:37.965842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:13:37.965842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:13:37.965842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:37.965842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:13:37.966408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:37.966408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:13:37.966408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:37.966408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:13:37.966408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:13:37.966408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:13:37.966408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:13:37.966408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:13:37.966408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:13:37.966408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:13:37.966408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:13:37.966408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:13:37.966408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:13:37.966408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:13:37.966408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:13:37.967132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:13:37.967132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:13:37.967132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:13:37.967132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:13:37.967132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:13:37.967132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:37.967132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:13:37.967132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:37.967132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:13:37.967132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:37.967132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:13:37.967132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:37.967132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:13:37.967132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:37.967132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:13:37.967132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:13:37.967132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:37.967132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:13:37.967132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:37.968170 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:13:37.968170 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:13:37.968170 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:37.968170 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:13:37.968170 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:37.968170 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:13:38.432544 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:13:38.433243 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:13:38.433496 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:13:38.434626 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:13:38.434850 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:13:38.434979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:38.435635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:13:38.435635 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:38.435635 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:13:38.435635 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:13:38.435635 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:38.435635 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:13:38.435635 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:38.435635 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:13:38.437702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:38.437702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:13:38.437702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:13:38.437702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:13:38.437702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:13:38.437702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:13:38.437702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:13:38.437702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:38.437702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:13:38.437702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:38.437702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:13:38.437702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:38.438264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:13:38.438264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:38.438264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:13:38.438264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:38.438264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:13:38.438264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:13:38.438264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:38.438264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:13:38.438264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:38.438264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:13:38.438264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:13:38.438264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:38.438264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:13:38.438686 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:38.438686 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:13:38.440122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:38.440122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:13:38.440122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:13:38.440122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:13:38.440122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:13:38.440122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:13:38.440122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:13:38.440122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:13:38.440122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:13:38.440122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:13:38.440122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:13:38.440122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:13:38.440122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:13:38.440122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:13:38.440122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:13:38.440122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:13:38.440647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:13:38.440647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:13:38.440647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:38.440647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:13:38.440647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:38.440647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:13:38.440647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:38.440647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:13:38.440647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:38.440647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:13:38.440647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:38.440647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:13:38.440647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:13:38.440647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:38.440647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:13:38.440647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:38.440647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:13:38.440647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:13:38.440647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:38.441148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:13:38.441148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:38.441148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:13:38.443322 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:13:38.443575 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:13:38.443693 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:13:38.443850 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:13:38.444149 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:13:38.444309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:38.444805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:13:38.444805 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:38.444805 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:13:38.444805 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:13:38.444805 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:38.444805 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:13:38.444805 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:38.444805 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:13:38.445806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:38.445806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:13:38.445806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:13:38.445806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:13:38.445806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:38.445806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:13:38.445806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:38.445806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:13:38.445806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:38.445806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:13:38.445806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:38.445806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:13:38.445806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:38.445806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:13:38.445806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:13:38.446219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:38.446219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:13:38.446219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:38.446219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:13:38.446219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:13:38.446219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:38.446219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:13:38.446219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:38.446219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:13:38.446642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:38.446642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:13:38.446642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:13:38.446642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:13:38.446642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:13:38.446642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:13:38.446642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:13:38.446642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:13:38.446642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:13:38.446642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:13:38.446642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:38.446642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:13:38.447000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:38.447000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:13:38.447000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:38.447000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:13:38.447000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:38.447000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:13:38.447000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:38.447000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:13:38.447000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:13:38.447000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:38.447000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:13:38.447000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:38.447000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:13:38.447000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:13:38.447373 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:38.447373 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:13:38.447373 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:38.447373 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:13:38.447762 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:38.447762 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:13:38.447762 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:13:38.447762 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:13:38.447762 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:13:38.447762 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:13:38.447762 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:13:38.447762 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:13:38.447762 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:13:38.447762 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:13:38.447762 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:13:38.447762 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:13:38.447762 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:13:38.447762 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:13:38.447762 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:13:38.447762 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:13:38.448175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:13:38.448175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:13:38.448175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:13:38.448175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:13:38.448175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:13:38.448175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:38.448175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:13:38.448175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:38.448175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:13:38.448175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:38.448175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:13:38.448175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:38.448175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:13:38.448175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:38.448175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:13:38.448175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:13:38.448175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:38.448175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:13:38.448617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:38.448617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:13:38.448617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:13:38.448617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:38.448617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:13:38.448617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:38.448617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:13:38.503319 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:13:38.504125 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:13:38.504381 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:13:38.504831 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:13:38.505497 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:13:38.505942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:38.506790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:13:38.506790 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:38.506790 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:13:38.506790 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:13:38.506790 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:38.506790 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:13:38.506790 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:38.506790 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:13:38.508215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:38.508215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:13:38.508215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:13:38.508215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:13:38.508215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:38.508215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:13:38.508215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:38.508215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:13:38.508215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:38.508215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:13:38.508215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:38.508215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:13:38.508215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:38.508215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:13:38.508215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:13:38.508925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:38.508925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:13:38.508925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:38.508925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:13:38.508925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:13:38.508925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:38.508925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:13:38.508925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:38.508925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:13:38.509261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:38.509261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:13:38.509261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:13:38.509261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:13:38.509261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:13:38.509261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:13:38.509261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:13:38.509261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:13:38.509261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:13:38.509261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:38.509261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:13:38.509717 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:38.509717 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:13:38.509717 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:38.509717 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:13:38.509717 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:38.509717 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:13:38.509717 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:38.509717 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:13:38.509717 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:13:38.509717 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:38.509717 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:13:38.509717 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:38.509717 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:13:38.509717 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:13:38.510262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:38.510262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:13:38.510262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:38.510262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:13:38.510262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:38.510262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:13:38.510262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:13:38.510262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:13:38.510262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:13:38.510262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:13:38.510262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:13:38.510262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:13:38.510262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:13:38.510262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:13:38.511015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:13:38.511015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:13:38.511015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:13:38.511015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:13:38.511015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:13:38.511015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:13:38.511015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:13:38.511015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:13:38.511015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:13:38.511015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:13:38.511015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:38.511015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:13:38.511015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:38.511015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:13:38.511015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:38.511015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:13:38.511015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:38.511015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:13:38.511015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:38.511720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:13:38.511720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:13:38.511720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:38.511720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:13:38.511720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:38.511720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:13:38.511720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:13:38.511720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:38.511720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:13:38.511720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:38.511720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:13:38.514545 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:13:38.514821 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:13:38.514924 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:13:38.515145 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:13:38.515725 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:13:38.516104 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:13:38.516334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:38.517094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:13:38.517094 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:38.517094 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:13:38.517094 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:13:38.517094 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:38.517094 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:13:38.517094 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:38.517094 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:13:38.517990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:38.517990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:13:38.517990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:13:38.517990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:13:38.517990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:13:38.517990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:13:38.517990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:13:38.517990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:38.517990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:13:38.517990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:38.517990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:13:38.517990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:38.518370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:13:38.518370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:38.518370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:13:38.518370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:38.518370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:13:38.518370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:13:38.518370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:38.518370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:13:38.518370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:38.518370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:13:38.518370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:13:38.518370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:38.518370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:13:38.518697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:38.518697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:13:38.518982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:38.518982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:13:38.518982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:13:38.518982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:13:38.518982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:13:38.518982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:13:38.518982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:13:38.518982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:13:38.518982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:13:38.518982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:13:38.518982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:13:38.518982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:13:38.518982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:13:38.518982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:13:38.518982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:13:38.518982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:13:38.519472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:13:38.519472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:13:38.519472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:38.519472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:13:38.519472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:38.519472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:13:38.519472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:38.519472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:13:38.519472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:38.519472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:13:38.519472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:38.519472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:13:38.519472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:13:38.519472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:38.519472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:13:38.519472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:38.519472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:13:38.519472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:13:38.519472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:38.520075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:13:38.520075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:38.520075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:13:42.922647 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:13:42.923922 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:13:42.923922 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:13:42.924929 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:13:42.926909 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG dbcounter [-] [106314] Writing DB stats neutron:SELECT=240 {{(pid=106314) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 09 09:13:42.962704 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:13:42.963141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:42.964050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:13:42.964050 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:42.964050 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:13:42.964050 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:13:42.964050 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:42.964050 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:13:42.964050 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:42.964050 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:13:42.964991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:42.964991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:13:42.964991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:13:42.964991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:13:42.964991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:13:42.964991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:13:42.964991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:13:42.964991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:42.964991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:13:42.964991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:42.964991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:13:42.964991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:42.965646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:13:42.965646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:42.965646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:13:42.965646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:42.965646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:13:42.965646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:13:42.965646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:42.965646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:13:42.965646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:42.965646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:13:42.965646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:13:42.965646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:42.965646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:13:42.966365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:42.966365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:13:42.966365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:42.966365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:13:42.966365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:13:42.966365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:13:42.966365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:13:42.966365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:13:42.966365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:13:42.966365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:13:42.966365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:13:42.966365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:13:42.966365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:13:42.966365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:13:42.966365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:13:42.967125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:13:42.967125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:13:42.967125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:13:42.967125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:13:42.967125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:13:42.967125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:42.967125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:13:42.967125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:42.967125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:13:42.967125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:42.967125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:13:42.967125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:42.967125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:13:42.967125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:42.967125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:13:42.967125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:13:42.967125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:42.967125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:13:42.967125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:42.968046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:13:42.968046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:13:42.968046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:42.968046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:13:42.968046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:42.968046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:13:43.433228 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:13:43.433902 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:13:43.433902 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:13:43.435130 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:13:43.435512 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:13:43.435774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:43.436664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:13:43.436664 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:43.436664 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:13:43.436664 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:13:43.436664 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:43.436664 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:13:43.436664 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:43.436664 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:13:43.437770 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:43.437770 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:13:43.437770 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:13:43.437770 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:13:43.437770 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:13:43.437770 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:13:43.437770 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:13:43.437770 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:43.437770 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:13:43.437770 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:43.437770 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:13:43.437770 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:43.438348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:13:43.438348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:43.438348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:13:43.438348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:43.438348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:13:43.438348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:13:43.438348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:43.438348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:13:43.438348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:43.438348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:13:43.438348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:13:43.438348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:43.438348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:13:43.439152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:43.439152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:13:43.439152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:43.439152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:13:43.439152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:13:43.439152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:13:43.439152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:13:43.439152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:13:43.439152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:13:43.439152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:13:43.439152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:13:43.439152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:13:43.439152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:13:43.439152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:13:43.439152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:13:43.440031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:13:43.440031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:13:43.440031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:13:43.440031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:13:43.440031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:13:43.440031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:43.440031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:13:43.440031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:43.440031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:13:43.440031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:43.440031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:13:43.440031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:43.440031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:13:43.440031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:43.440031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:13:43.440031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:13:43.440031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:43.440031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:13:43.440031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:43.441179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:13:43.441179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:13:43.441179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:43.441179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:13:43.441179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:43.441179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:13:43.443840 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:13:43.444181 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:13:43.444317 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:13:43.444438 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:13:43.444940 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:13:43.445346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:43.446038 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:13:43.446038 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:43.446038 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:13:43.446038 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:13:43.446038 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:43.446038 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:13:43.446038 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:43.446038 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:13:43.446913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:43.446913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:13:43.446913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:13:43.446913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:13:43.446913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:43.446913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:13:43.446913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:43.446913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:13:43.446913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:43.446913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:13:43.446913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:43.446913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:13:43.446913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:43.446913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:13:43.446913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:13:43.447399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:43.447399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:13:43.447399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:43.447399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:13:43.447399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:13:43.447399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:43.447399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:13:43.447399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:43.447399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:13:43.448015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:43.448015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:13:43.448015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:13:43.448015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:13:43.448015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:13:43.448015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:13:43.448015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:13:43.448015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:13:43.448015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:13:43.448015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:13:43.448015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:43.448015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:13:43.448896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:43.448896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:13:43.448896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:43.448896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:13:43.448896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:43.448896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:13:43.448896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:43.448896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:13:43.448896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:13:43.448896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:43.448896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:13:43.448896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:43.448896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:13:43.448896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:13:43.449502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:43.449502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:13:43.449502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:43.449502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:13:43.449502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:43.449502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:13:43.449502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:13:43.449502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:13:43.449502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:13:43.449502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:13:43.449502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:13:43.449502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:13:43.449502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:13:43.449502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:13:43.449921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:13:43.449921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:13:43.449921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:13:43.449921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:13:43.449921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:13:43.449921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:13:43.449921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:13:43.449921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:13:43.449921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:13:43.449921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:13:43.449921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:13:43.449921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:43.449921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:13:43.449921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:43.449921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:13:43.449921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:43.449921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:13:43.449921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:43.449921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:13:43.450673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:43.450673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:13:43.450673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:13:43.450673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:43.450673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:13:43.450673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:43.450673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:13:43.450673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:13:43.450673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:43.450673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:13:43.450673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:43.450673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:13:43.503803 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:13:43.504354 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:13:43.504544 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:13:43.504793 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:13:43.505458 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:13:43.505900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:43.506867 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:13:43.506867 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:43.506867 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:13:43.506867 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:13:43.506867 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:43.506867 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:13:43.506867 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:43.506867 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:13:43.508487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:43.508487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:13:43.508487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:13:43.508487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:13:43.508487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:43.508487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:13:43.508487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:43.508487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:13:43.508487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:43.508487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:13:43.508487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:43.508487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:13:43.508487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:43.508487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:13:43.508487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:13:43.509178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:43.509178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:13:43.509178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:43.509178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:13:43.509178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:13:43.509178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:43.509178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:13:43.509178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:43.509178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:13:43.510208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:43.510208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:13:43.510208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:13:43.510208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:13:43.510208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:13:43.510208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:13:43.510208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:13:43.510208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:13:43.510208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:13:43.510208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:43.510208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:13:43.510834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:43.510834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:13:43.510834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:43.510834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:13:43.510834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:43.510834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:13:43.510834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:43.510834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:13:43.510834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:13:43.510834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:43.510834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:13:43.510834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:43.510834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:13:43.510834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:13:43.511397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:43.511397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:13:43.511397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:43.511397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:13:43.512183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:43.512183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:13:43.512183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:13:43.512183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:13:43.512183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:13:43.512183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:13:43.512183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:13:43.512183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:13:43.512183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:13:43.512183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:13:43.512183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:13:43.512183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:13:43.512183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:13:43.512183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:13:43.512183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:13:43.512183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:13:43.512183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:13:43.513002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:13:43.513002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:13:43.513002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:13:43.513002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:43.513002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:13:43.513002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:43.513002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:13:43.513002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:43.513002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:13:43.513002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:43.513002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:13:43.513002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:43.513002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:13:43.513002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:13:43.513002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:43.513002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:13:43.513002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:43.513002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:13:43.513002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:13:43.513861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:43.513861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:13:43.513861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:43.513861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:13:43.514887 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:13:43.515301 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:13:43.515589 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:13:43.515934 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:13:43.516447 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:13:43.516955 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:13:43.517198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:43.518123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:13:43.518123 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:43.518123 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:13:43.518123 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:13:43.518123 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:43.518123 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:13:43.518123 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:43.518123 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:13:43.519770 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:43.519770 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:13:43.519770 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:13:43.519770 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:13:43.519770 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:13:43.519770 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:13:43.519770 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:13:43.519770 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:43.519770 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:13:43.519770 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:43.519770 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:13:43.519770 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:43.520098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:13:43.520098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:43.520098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:13:43.520098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:43.520098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:13:43.520098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:13:43.520098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:43.520098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:13:43.520098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:43.520098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:13:43.520098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:13:43.520098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:43.520098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:13:43.520433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:43.520433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:13:43.521590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:43.521590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:13:43.521590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:13:43.521590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:13:43.521590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:13:43.521590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:13:43.521590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:13:43.521590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:13:43.521590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:13:43.521590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:13:43.521590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:13:43.521590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:13:43.521590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:13:43.521590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:13:43.521590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:13:43.521590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:13:43.522189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:13:43.522189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:13:43.522189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:43.522189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:13:43.522189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:43.522189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:13:43.522189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:43.522189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:13:43.522189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:43.522189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:13:43.522189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:43.522189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:13:43.522189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:13:43.522189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:43.522189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:13:43.522189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:43.522189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:13:43.522189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:13:43.522189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:43.522735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:13:43.522735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:43.522735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:13:47.923686 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:13:47.924678 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:13:47.924678 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:13:47.925913 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:13:47.961059 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:13:47.961328 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:47.962095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:13:47.962095 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:47.962095 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:13:47.962095 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:13:47.962095 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:47.962095 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:13:47.962095 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:47.962095 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:13:47.963160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:47.963160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:13:47.963160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:13:47.963160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:13:47.963160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:13:47.963160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:13:47.963160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:13:47.963160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:47.963160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:13:47.963160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:47.963160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:13:47.963160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:47.963704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:13:47.963704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:47.963704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:13:47.963704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:47.963704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:13:47.963704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:13:47.963704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:47.963704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:13:47.963704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:47.963704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:13:47.963704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:13:47.963704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:47.963704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:13:47.964242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:47.964242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:13:47.964242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:47.964242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:13:47.964242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:13:47.964242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:13:47.964242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:13:47.964242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:13:47.964242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:13:47.964242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:13:47.964242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:13:47.964242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:13:47.964242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:13:47.964242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:13:47.964242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:13:47.964898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:13:47.964898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:13:47.964898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:13:47.964898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:13:47.964898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:13:47.964898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:47.964898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:13:47.964898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:47.964898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:13:47.964898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:47.964898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:13:47.964898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:47.964898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:13:47.964898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:47.964898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:13:47.964898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:13:47.964898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:47.964898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:13:47.964898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:47.965638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:13:47.965638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:13:47.965638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:47.965638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:13:47.965638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:47.965638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:13:48.434028 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:13:48.434692 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:13:48.434827 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:13:48.436033 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:13:48.436399 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:13:48.436775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:48.437666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:13:48.437666 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:48.437666 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:13:48.437666 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:13:48.437666 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:48.437666 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:13:48.437666 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:48.437666 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:13:48.439343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:48.439343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:13:48.439343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:13:48.439343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:13:48.439343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:13:48.439343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:13:48.439343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:13:48.439343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:48.439343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:13:48.439343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:48.439343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:13:48.439343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:48.439943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:13:48.439943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:48.439943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:13:48.439943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:48.439943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:13:48.439943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:13:48.439943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:48.439943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:13:48.439943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:48.439943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:13:48.439943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:13:48.439943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:48.439943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:13:48.440441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:48.440441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:13:48.441159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:48.441159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:13:48.441159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:13:48.441159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:13:48.441159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:13:48.441159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:13:48.441159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:13:48.441159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:13:48.441159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:13:48.441159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:13:48.441159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:13:48.441159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:13:48.441159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:13:48.441159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:13:48.441159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:13:48.441159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:13:48.442082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:13:48.442082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:13:48.442082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:48.442082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:13:48.442082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:48.442082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:13:48.442082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:48.442082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:13:48.442082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:48.442082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:13:48.442082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:48.442082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:13:48.442082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:13:48.442082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:48.442082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:13:48.442082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:48.442082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:13:48.442082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:13:48.442082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:48.442978 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:13:48.442978 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:48.442978 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:13:48.444386 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:13:48.444818 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:13:48.444977 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:13:48.445116 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:13:48.445574 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:13:48.445913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:48.446711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:13:48.446711 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:48.446711 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:13:48.446711 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:13:48.446711 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:48.446711 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:13:48.446711 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:48.446711 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:13:48.447535 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:48.447535 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:13:48.447535 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:13:48.447535 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:13:48.447535 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:48.447535 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:13:48.447535 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:48.447535 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:13:48.447535 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:48.447535 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:13:48.447535 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:48.447535 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:13:48.447535 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:48.447535 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:13:48.447535 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:13:48.448124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:48.448124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:13:48.448124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:48.448124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:13:48.448124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:13:48.448124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:48.448124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:13:48.448124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:48.448124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:13:48.448350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:48.448350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:13:48.448350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:13:48.448350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:13:48.448350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:13:48.448350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:13:48.448350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:13:48.448350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:13:48.448350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:13:48.448350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:13:48.448350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:48.448350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:13:48.448685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:48.448685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:13:48.448685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:48.448685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:13:48.448685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:48.448685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:13:48.448685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:48.448685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:13:48.448685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:13:48.448685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:48.448685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:13:48.448685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:48.448685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:13:48.448685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:13:48.449076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:48.449076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:13:48.449076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:48.449076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:13:48.449683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:48.449683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:13:48.449683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:13:48.449683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:13:48.449683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:13:48.449683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:13:48.449683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:13:48.449683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:13:48.449683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:13:48.449683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:13:48.449683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:13:48.449683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:13:48.449683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:13:48.449683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:13:48.449683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:13:48.449683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:13:48.450625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:13:48.450625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:13:48.450625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:13:48.450625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:13:48.450625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:13:48.450625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:48.450625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:13:48.450625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:48.450625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:13:48.450625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:48.450625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:13:48.450625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:48.450625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:13:48.450625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:48.450625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:13:48.450625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:13:48.450625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:48.450625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:13:48.451586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:48.451586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:13:48.451586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:13:48.451586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:48.451586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:13:48.451586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:48.451586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:13:48.504610 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:13:48.505166 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:13:48.505356 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:13:48.505585 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:13:48.506285 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:13:48.506781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:48.507680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:13:48.507680 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:48.507680 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:13:48.507680 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:13:48.507680 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:48.507680 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:13:48.507680 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:48.507680 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:13:48.509310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:48.509310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:13:48.509310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:13:48.509310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:13:48.509310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:48.509310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:13:48.509310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:48.509310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:13:48.509310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:48.509310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:13:48.509310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:48.509310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:13:48.509310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:48.509310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:13:48.509310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:13:48.510120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:48.510120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:13:48.510120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:48.510120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:13:48.510120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:13:48.510120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:48.510120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:13:48.510120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:48.510120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:13:48.511521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:48.511521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:13:48.511521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:13:48.511521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:13:48.511521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:13:48.511521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:13:48.511521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:13:48.511521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:13:48.511521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:13:48.511521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:48.511521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:13:48.511993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:48.511993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:13:48.511993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:48.511993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:13:48.511993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:48.511993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:13:48.511993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:48.511993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:13:48.511993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:13:48.511993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:48.511993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:13:48.511993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:48.511993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:13:48.511993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:13:48.512357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:48.512357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:13:48.512357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:48.512357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:13:48.513474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:48.513474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:13:48.513474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:13:48.513474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:13:48.513474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:13:48.513474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:13:48.513474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:13:48.513474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:13:48.513474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:13:48.513474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:13:48.513474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:13:48.513474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:13:48.513474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:13:48.513474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:13:48.513474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:13:48.513474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:13:48.513474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:13:48.513984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:13:48.513984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:13:48.513984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:13:48.513984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:48.513984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:13:48.513984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:48.513984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:13:48.513984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:48.513984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:13:48.513984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:48.513984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:13:48.513984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:48.513984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:13:48.513984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:13:48.513984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:48.513984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:13:48.513984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:48.513984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:13:48.513984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:13:48.514442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:48.514442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:13:48.514442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:48.514442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:13:48.515353 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:13:48.515738 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:13:48.515953 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:13:48.516202 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:13:48.517536 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:13:48.517950 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:13:48.518203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:48.519203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:13:48.519203 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:48.519203 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:13:48.519203 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:13:48.519203 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:48.519203 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:13:48.519203 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:48.519203 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:13:48.520772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:48.520772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:13:48.520772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:13:48.520772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:13:48.520772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:13:48.520772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:13:48.520772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:13:48.520772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:48.520772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:13:48.520772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:48.520772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:13:48.520772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:48.522048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:13:48.522048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:48.522048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:13:48.522048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:48.522048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:13:48.522048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:13:48.522048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:48.522048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:13:48.522048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:48.522048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:13:48.522048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:13:48.522048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:48.522048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:13:48.522566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:48.522566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:13:48.522566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:48.522566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:13:48.522566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:13:48.522566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:13:48.522566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:13:48.522566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:13:48.522566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:13:48.522566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:13:48.522566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:13:48.522566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:13:48.522566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:13:48.522566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:13:48.522566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:13:48.523225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:13:48.523225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:13:48.523225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:13:48.523225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:13:48.523225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:13:48.523225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:48.523225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:13:48.523225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:48.523225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:13:48.523225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:48.523225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:13:48.523225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:48.523225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:13:48.523225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:48.523225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:13:48.523225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:13:48.523225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:48.523225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:13:48.523225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:48.525467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:13:48.525467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:13:48.525467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:48.525467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:13:48.525467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:48.525467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:13:52.924262 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:13:52.925125 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:13:52.925125 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:13:52.926596 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:13:52.963217 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:13:52.963595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:52.964502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:13:52.964502 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:52.964502 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:13:52.964502 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:13:52.964502 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:52.964502 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:13:52.964502 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:52.964502 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:13:52.965320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:52.965320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:13:52.965320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:13:52.965320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:13:52.965320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:13:52.965320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:13:52.965320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:13:52.965320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:52.965320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:13:52.965320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:52.965320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:13:52.965320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:52.966080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:13:52.966080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:52.966080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:13:52.966080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:52.966080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:13:52.966080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:13:52.966080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:52.966080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:13:52.966080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:52.966080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:13:52.966080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:13:52.966080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:52.966080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:13:52.966815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:52.966815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:13:52.966815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:52.966815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:13:52.966815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:13:52.966815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:13:52.966815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:13:52.966815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:13:52.966815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:13:52.966815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:13:52.966815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:13:52.966815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:13:52.966815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:13:52.966815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:13:52.966815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:13:52.967585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:13:52.967585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:13:52.967585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:13:52.967585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:13:52.967585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:13:52.967585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:52.967585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:13:52.967585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:52.967585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:13:52.967585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:52.967585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:13:52.967585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:52.967585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:13:52.967585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:52.967585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:13:52.967585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:13:52.967585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:52.967585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:13:52.967585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:52.968555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:13:52.968555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:13:52.968555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:52.968555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:13:52.968555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:52.968555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:13:53.434858 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:13:53.457163 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:13:53.457163 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:13:53.457163 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:13:53.457163 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:13:53.457163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:53.457163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:13:53.457163 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:53.457163 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:13:53.457163 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:13:53.457738 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:53.457738 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:13:53.457738 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:53.457738 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:13:53.457738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:53.457738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:13:53.457738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:13:53.457738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:13:53.457738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:13:53.457738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:13:53.457738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:13:53.457738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:53.457738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:13:53.458416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:53.458416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:13:53.458416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:53.458416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:13:53.458416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:53.458416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:13:53.458416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:53.458416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:13:53.458416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:13:53.458416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:53.458416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:13:53.458416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:53.458416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:13:53.458416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:13:53.459056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:53.459056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:13:53.459056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:53.459056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:13:53.459056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:53.459056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:13:53.459056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:13:53.459056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:13:53.459056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:13:53.459056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:13:53.459056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:13:53.459056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:13:53.459056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:13:53.459056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:13:53.460031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:13:53.460031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:13:53.460031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:13:53.460031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:13:53.460031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:13:53.460031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:13:53.460031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:13:53.460031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:13:53.460031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:53.460031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:13:53.460031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:53.460031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:13:53.460031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:53.460031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:13:53.460031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:53.460031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:13:53.460031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:53.460031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:13:53.460031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:13:53.460803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:53.460803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:13:53.460803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:53.460803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:13:53.460803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:13:53.460803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:53.460803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:13:53.460803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:53.460803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:13:53.460803 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:13:53.460803 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:13:53.460803 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:13:53.461201 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:13:53.461201 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:13:53.461201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:53.461201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:13:53.461201 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:53.461201 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:13:53.461201 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:13:53.461201 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:53.461201 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:13:53.461201 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:53.461201 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:13:53.461201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:53.461201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:13:53.461584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:13:53.461584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:13:53.461584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:53.461584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:13:53.461584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:53.461584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:13:53.461584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:53.461584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:13:53.461584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:53.461584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:13:53.461584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:53.461584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:13:53.461584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:13:53.461584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:53.461584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:13:53.461584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:53.461584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:13:53.462234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:13:53.462234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:53.462234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:13:53.462234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:53.462234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:13:53.462234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:53.462234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:13:53.462234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:13:53.462234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:13:53.462234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:13:53.462234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:13:53.462234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:13:53.462234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:13:53.462656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:13:53.462656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:13:53.462656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:53.462656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:13:53.462656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:53.462656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:13:53.462656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:53.462656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:13:53.462656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:53.462656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:13:53.462656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:53.462656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:13:53.462656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:13:53.463059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:53.463059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:13:53.463059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:53.463059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:13:53.463059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:13:53.463059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:53.463059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:13:53.463059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:53.463059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:13:53.463059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:53.463059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:13:53.463059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:13:53.463482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:13:53.463482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:13:53.463482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:13:53.463482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:13:53.463482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:13:53.463482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:13:53.463482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:13:53.463482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:13:53.463482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:13:53.463482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:13:53.463482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:13:53.463482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:13:53.463482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:13:53.463482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:13:53.463482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:13:53.463482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:13:53.463482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:13:53.463482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:13:53.463482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:53.463482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:13:53.464058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:53.464058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:13:53.464058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:53.464058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:13:53.464058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:53.464058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:13:53.464058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:53.464058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:13:53.464058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:13:53.464058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:53.464058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:13:53.464058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:53.464058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:13:53.464058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:13:53.464058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:53.464058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:13:53.464058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:53.464058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:13:53.505208 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:13:53.506094 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:13:53.506365 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:13:53.506647 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:13:53.507541 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:13:53.508039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:53.509225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:13:53.509225 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:53.509225 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:13:53.509225 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:13:53.509225 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:53.509225 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:13:53.509225 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:53.509225 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:13:53.510284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:53.510284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:13:53.510284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:13:53.510284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:13:53.510284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:53.510284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:13:53.510284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:53.510284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:13:53.510284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:53.510284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:13:53.510284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:53.510284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:13:53.510284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:53.510284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:13:53.510284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:13:53.510724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:53.510724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:13:53.510724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:53.510724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:13:53.510724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:13:53.510724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:53.510724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:13:53.510724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:53.510724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:13:53.511209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:53.511209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:13:53.511209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:13:53.511209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:13:53.511209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:13:53.511209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:13:53.511209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:13:53.511209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:13:53.511209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:13:53.511209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:53.511209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:13:53.511606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:53.511606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:13:53.511606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:53.511606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:13:53.511606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:53.511606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:13:53.511606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:53.511606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:13:53.511606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:13:53.511606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:53.511606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:13:53.511606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:53.511606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:13:53.511606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:13:53.512064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:53.512064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:13:53.512064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:53.512064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:13:53.512913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:53.512913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:13:53.512913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:13:53.512913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:13:53.512913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:13:53.512913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:13:53.512913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:13:53.512913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:13:53.512913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:13:53.512913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:13:53.512913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:13:53.512913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:13:53.512913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:13:53.512913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:13:53.512913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:13:53.512913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:13:53.512913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:13:53.513849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:13:53.513849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:13:53.513849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:13:53.513849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:53.513849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:13:53.513849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:53.513849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:13:53.513849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:53.513849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:13:53.513849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:53.513849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:13:53.513849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:53.513849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:13:53.513849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:13:53.513849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:53.513849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:13:53.513849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:53.513849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:13:53.513849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:13:53.514998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:53.514998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:13:53.514998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:53.514998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:13:53.515879 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:13:53.516178 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:13:53.516283 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:13:53.516460 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:13:53.517088 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:13:53.517493 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:13:53.517726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:53.518560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:13:53.518560 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:53.518560 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:13:53.518560 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:13:53.518560 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:53.518560 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:13:53.518560 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:53.518560 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:13:53.519526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:53.519526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:13:53.519526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:13:53.519526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:13:53.519526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:13:53.519526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:13:53.519526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:13:53.519526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:53.519526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:13:53.519526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:53.519526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:13:53.519526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:53.520215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:13:53.520215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:53.520215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:13:53.520215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:53.520215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:13:53.520215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:13:53.520215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:53.520215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:13:53.520215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:53.520215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:13:53.520215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:13:53.520215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:53.520215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:13:53.520723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:53.520723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:13:53.520723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:53.520723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:13:53.520723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:13:53.520723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:13:53.520723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:13:53.520723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:13:53.520723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:13:53.520723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:13:53.520723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:13:53.520723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:13:53.520723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:13:53.520723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:13:53.520723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:13:53.521369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:13:53.521369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:13:53.521369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:13:53.521369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:13:53.521369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:13:53.521369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:53.521369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:13:53.521369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:53.521369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:13:53.521369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:53.521369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:13:53.521369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:53.521369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:13:53.521369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:53.521369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:13:53.521369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:13:53.521369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:53.521369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:13:53.521369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:53.522707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:13:53.522707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:13:53.522707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:53.522707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:13:53.522707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:53.522707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:13:57.925092 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:13:57.925919 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:13:57.926010 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:13:57.927404 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:13:57.964494 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:13:57.964823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:57.965568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:13:57.965568 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:57.965568 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:13:57.965568 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:13:57.965568 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:57.965568 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:13:57.965568 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:57.965568 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:13:57.966357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:57.966357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:13:57.966357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:13:57.966357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:13:57.966357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:13:57.966357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:13:57.966357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:13:57.966357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:57.966357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:13:57.966357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:57.966357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:13:57.966357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:57.967129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:13:57.967129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:57.967129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:13:57.967129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:57.967129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:13:57.967129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:13:57.967129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:57.967129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:13:57.967129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:57.967129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:13:57.967129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:13:57.967129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:57.967129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:13:57.967942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:57.967942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:13:57.967942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:57.967942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:13:57.967942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:13:57.967942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:13:57.967942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:13:57.967942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:13:57.967942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:13:57.967942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:13:57.967942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:13:57.967942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:13:57.967942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:13:57.967942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:13:57.967942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:13:57.968896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:13:57.968896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:13:57.968896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:13:57.968896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:13:57.968896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:13:57.968896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:57.968896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:13:57.968896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:57.968896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:13:57.968896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:57.968896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:13:57.968896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:57.968896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:13:57.968896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:57.968896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:13:57.968896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:13:57.968896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:57.968896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:13:57.968896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:57.970061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:13:57.970061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:13:57.970061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:57.970061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:13:57.970061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:57.970061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:13:58.435704 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:13:58.436144 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:13:58.436273 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:13:58.437497 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:13:58.437903 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:13:58.438172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:58.439055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:13:58.439055 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:58.439055 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:13:58.439055 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:13:58.439055 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:58.439055 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:13:58.439055 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:58.439055 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:13:58.440101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:58.440101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:13:58.440101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:13:58.440101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:13:58.440101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:13:58.440101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:13:58.440101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:13:58.440101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:58.440101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:13:58.440101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:58.440101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:13:58.440101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:58.440682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:13:58.440682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:58.440682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:13:58.440682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:58.440682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:13:58.440682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:13:58.440682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:58.440682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:13:58.440682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:58.440682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:13:58.440682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:13:58.440682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:58.440682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:13:58.441306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:58.441306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:13:58.441409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:58.441409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:13:58.441409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:13:58.441409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:13:58.441409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:13:58.441409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:13:58.441409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:13:58.441409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:13:58.441409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:13:58.441409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:13:58.441409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:13:58.441409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:13:58.441409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:13:58.441409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:13:58.441409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:13:58.441409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:13:58.442079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:13:58.442079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:13:58.442079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:58.442079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:13:58.442079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:58.442079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:13:58.442079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:58.442079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:13:58.442079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:58.442079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:13:58.442079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:58.442079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:13:58.442079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:13:58.442079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:58.442079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:13:58.442079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:58.442079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:13:58.442079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:13:58.442079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:58.442889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:13:58.442889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:58.442889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:13:58.445435 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:13:58.445662 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:13:58.445806 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:13:58.445913 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:13:58.446297 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:13:58.446520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:58.447109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:13:58.447109 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:58.447109 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:13:58.447109 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:13:58.447109 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:58.447109 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:13:58.447109 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:58.447109 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:13:58.447854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:58.447854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:13:58.447854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:13:58.447854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:13:58.447854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:58.447854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:13:58.447854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:58.447854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:13:58.447854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:58.447854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:13:58.447854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:58.447854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:13:58.447854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:58.447854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:13:58.447854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:13:58.448279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:58.448279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:13:58.448279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:58.448279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:13:58.448279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:13:58.448279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:58.448279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:13:58.448279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:58.448279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:13:58.448653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:58.448653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:13:58.448653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:13:58.448653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:13:58.448653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:13:58.448653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:13:58.448653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:13:58.448653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:13:58.448653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:13:58.448653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:13:58.448653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:58.448653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:13:58.449041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:58.449041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:13:58.449041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:58.449041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:13:58.449041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:58.449041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:13:58.449041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:58.449041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:13:58.449041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:13:58.449041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:58.449041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:13:58.449041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:58.449041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:13:58.449041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:13:58.449471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:58.449471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:13:58.449471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:58.449471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:13:58.449670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:58.449670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:13:58.449670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:13:58.449670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:13:58.449670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:13:58.449670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:13:58.449670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:13:58.449670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:13:58.449670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:13:58.449670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:13:58.449670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:13:58.449670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:13:58.449670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:13:58.449670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:13:58.449670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:13:58.449670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:13:58.450151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:13:58.450151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:13:58.450151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:13:58.450151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:13:58.450151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:13:58.450151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:58.450151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:13:58.450151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:58.450151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:13:58.450151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:58.450151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:13:58.450151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:58.450151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:13:58.450151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:58.450151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:13:58.450151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:13:58.450151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:58.450151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:13:58.450654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:58.450654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:13:58.450654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:13:58.450654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:58.450654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:13:58.450654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:58.450654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:13:58.505684 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:13:58.505988 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:13:58.506118 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:13:58.506310 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:13:58.507195 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:13:58.507633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:58.508572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:13:58.508572 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:58.508572 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:13:58.508572 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:13:58.508572 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:58.508572 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:13:58.508572 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:58.508572 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:13:58.509623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:58.509623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:13:58.509623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:13:58.509623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:13:58.509623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:58.509623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:13:58.509623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:58.509623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:13:58.509623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:58.509623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:13:58.509623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:58.509623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:13:58.509623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:58.509623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:13:58.509623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:13:58.509937 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:58.509937 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:13:58.509937 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:58.509937 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:13:58.509937 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:13:58.509937 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:58.509937 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:13:58.509937 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:58.509937 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:13:58.510950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:58.510950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:13:58.510950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:13:58.510950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:13:58.510950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:13:58.510950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:13:58.510950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:13:58.510950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:13:58.510950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:13:58.510950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:58.510950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:13:58.511210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:58.511210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:13:58.511210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:58.511210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:13:58.511210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:58.511210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:13:58.511210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:58.511210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:13:58.511210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:13:58.511210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:58.511210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:13:58.511210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:58.511210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:13:58.511210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:13:58.511492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:58.511492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:13:58.511492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:58.511492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:13:58.512593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:58.512593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:13:58.512593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:13:58.512593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:13:58.512593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:13:58.512593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:13:58.512593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:13:58.512593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:13:58.512593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:13:58.512593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:13:58.512593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:13:58.512593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:13:58.512593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:13:58.512593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:13:58.512593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:13:58.512593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:13:58.512593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:13:58.512984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:13:58.512984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:13:58.512984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:13:58.512984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:58.512984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:13:58.512984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:58.512984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:13:58.512984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:58.512984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:13:58.512984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:58.512984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:13:58.512984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:58.512984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:13:58.512984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:13:58.512984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:58.512984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:13:58.512984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:58.512984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:13:58.512984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:13:58.513356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:58.513356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:13:58.513356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:58.513356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:13:58.516245 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:13:58.516578 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:13:58.516691 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:13:58.516941 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:13:58.517512 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:13:58.517994 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:13:58.518253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:58.519124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:13:58.519124 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:58.519124 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:13:58.519124 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:13:58.519124 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:58.519124 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:13:58.519124 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:58.519124 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:13:58.520290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:58.520290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:13:58.520290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:13:58.520290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:13:58.520290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:13:58.520290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:13:58.520290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:13:58.520290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:58.520290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:13:58.520290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:58.520290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:13:58.520290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:58.520680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:13:58.520680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:58.520680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:13:58.520680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:58.520680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:13:58.520680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:13:58.520680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:58.520680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:13:58.520680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:58.520680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:13:58.520680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:13:58.520680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:58.520680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:13:58.521130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:58.521130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:13:58.521130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:58.521130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:13:58.521130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:13:58.521130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:13:58.521130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:13:58.521130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:13:58.521130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:13:58.521130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:13:58.521130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:13:58.521130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:13:58.521130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:13:58.521130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:13:58.521130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:13:58.521629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:13:58.521629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:13:58.521629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:13:58.521629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:13:58.521629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:13:58.521629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:58.521629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:13:58.521629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:13:58.521629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:13:58.521629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:13:58.521629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:13:58.521629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:13:58.521629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:13:58.521629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:13:58.521629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:13:58.521629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:13:58.521629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:13:58.521629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:13:58.521629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:13:58.522291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:13:58.522291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:13:58.522291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:13:58.522291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:13:58.522291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:13:58.522291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:14:02.925871 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:14:02.926666 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:14:02.926752 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:14:02.928057 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:14:02.973489 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:14:02.973851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:02.974590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:14:02.974590 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:02.974590 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:14:02.974590 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:14:02.974590 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:02.974590 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:14:02.974590 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:02.974590 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:14:02.975688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.048 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:02.975688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:14:02.975688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:14:02.975688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:14:02.975688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:14:02.975688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:14:02.975688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:14:02.975688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:02.975688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:14:02.975688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:02.975688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:14:02.975688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:02.976086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:14:02.976086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:02.976086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:14:02.976086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:02.976086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:14:02.976086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:14:02.976086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:02.976086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:14:02.976086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:02.976086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:14:02.976086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:14:02.976086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:02.976086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:14:02.976470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:02.976470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:14:02.976740 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:02.976740 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:14:02.976740 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:14:02.976740 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:14:02.976740 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:14:02.976740 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:14:02.976740 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:14:02.976740 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:14:02.976740 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:14:02.976740 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:14:02.976740 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:14:02.976740 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:14:02.976740 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:14:02.976740 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:14:02.976740 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:14:02.976740 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:14:02.977205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:14:02.977205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:14:02.977205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:02.977205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:14:02.977205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:02.977205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:14:02.977205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:02.977205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:14:02.977205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:02.977205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:14:02.977205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:02.977205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:14:02.977205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:14:02.977205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:02.977205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:14:02.977205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:02.977205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:14:02.977205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:14:02.977205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:02.977750 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:14:02.977750 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:02.977750 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:14:03.436505 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:14:03.437067 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:14:03.437168 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:14:03.438129 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:14:03.438375 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:14:03.438539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:03.439290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:14:03.439290 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:03.439290 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:14:03.439290 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:14:03.439290 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:03.439290 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:14:03.439290 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:03.439290 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:14:03.440786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:03.440786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:14:03.440786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:14:03.440786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:14:03.440786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:14:03.440786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:14:03.440786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:14:03.440786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:03.440786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:14:03.440786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:03.440786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:14:03.440786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:03.442262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:14:03.442262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:03.442262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:14:03.442262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:03.442262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:14:03.442262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:14:03.442262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:03.442262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:14:03.442262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:03.442262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:14:03.442262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:14:03.442262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:03.442262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:14:03.442708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:03.442708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:14:03.442708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:03.442708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:14:03.442708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:14:03.442708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:14:03.442708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:14:03.442708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:14:03.442708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:14:03.442708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:14:03.442708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:14:03.442708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:14:03.442708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:14:03.442708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:14:03.442708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:14:03.443180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:14:03.443180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:14:03.443180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:14:03.443180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:14:03.443180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:14:03.443180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:03.443180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:14:03.443180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:03.443180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:14:03.443180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:03.443180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:14:03.443180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:03.443180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:14:03.443180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:03.443180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:14:03.443180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:14:03.443180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:03.443180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:14:03.443180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:03.443691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:14:03.443691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:14:03.443691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:03.443691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:14:03.443691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:03.443691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:14:03.445805 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:14:03.446273 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:14:03.446448 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:14:03.446567 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:14:03.447016 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:14:03.447248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:03.447846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:14:03.447846 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:03.447846 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:14:03.447846 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:14:03.447846 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:03.447846 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:14:03.447846 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:03.447846 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:14:03.448604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:03.448604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:14:03.448604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:14:03.448604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:14:03.448604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:03.448604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:14:03.448604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:03.448604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:14:03.448604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:03.448604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:14:03.448604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:03.448604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:14:03.448604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:03.448604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:14:03.448604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:14:03.449189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:03.449189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:14:03.449189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:03.449189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:14:03.449189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:14:03.449189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:03.449189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:14:03.449189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:03.449189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:14:03.449551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:03.449551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:14:03.449551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:14:03.449551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:14:03.449551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:14:03.449551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:14:03.449551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:14:03.449551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:14:03.449551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:14:03.449551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:14:03.449551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:03.449551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:14:03.450099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:03.450099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:14:03.450099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:03.450099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:14:03.450099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:03.450099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:14:03.450099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:03.450099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:14:03.450099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:14:03.450099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:03.450099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:14:03.450099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:03.450099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:14:03.450099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:14:03.450550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:03.450550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:14:03.450550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:03.450550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:14:03.450550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:03.450550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:14:03.450550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:14:03.450550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:14:03.450550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:14:03.450550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:14:03.450550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:14:03.450550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:14:03.450550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:14:03.450550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:14:03.451098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:14:03.451098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:14:03.451098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:14:03.451098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:14:03.451098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:14:03.451098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:14:03.451098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:14:03.451098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:14:03.451098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:14:03.451098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:14:03.451098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:14:03.451098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:03.451098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:14:03.451098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:03.451098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:14:03.451098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:03.451098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:14:03.451098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:03.451098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:14:03.451796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:03.451796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:14:03.451796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:14:03.451796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:03.451796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:14:03.451796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:03.451796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:14:03.451796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:14:03.451796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:03.451796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:14:03.451796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:03.451796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:14:03.506207 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:14:03.506709 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:14:03.506980 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:14:03.507194 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:14:03.508283 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:14:03.508810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:03.509930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:14:03.509930 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:03.509930 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:14:03.509930 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:14:03.509930 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:03.509930 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:14:03.509930 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:03.509930 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:14:03.511325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:03.511325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:14:03.511325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:14:03.511325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:14:03.511325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:03.511325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:14:03.511325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:03.511325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:14:03.511325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:03.511325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:14:03.511325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:03.511325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:14:03.511325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:03.511325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:14:03.511325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:14:03.512009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:03.512009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:14:03.512009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:03.512009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:14:03.512009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:14:03.512009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:03.512009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:14:03.512009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:03.512009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:14:03.512735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:03.512735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:14:03.512735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:14:03.512735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:14:03.512735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:14:03.512735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:14:03.512735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:14:03.512735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:14:03.512735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:14:03.512735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:03.512735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:14:03.513274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:03.513274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:14:03.513274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:03.513274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:14:03.513274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:03.513274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:14:03.513274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:03.513274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:14:03.513274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:14:03.513274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:03.513274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:14:03.513274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:03.513274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:14:03.513274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:14:03.513909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:03.513909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:14:03.513909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:03.513909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:14:03.514602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:03.514602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:14:03.514602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:14:03.514602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:14:03.514602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:14:03.514602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:14:03.514602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:14:03.514602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:14:03.514602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:14:03.514602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:14:03.514602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:14:03.514602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:14:03.514602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:14:03.514602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:14:03.514602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:14:03.514602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:14:03.514602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:14:03.515466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:14:03.515466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:14:03.515466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:14:03.515466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:03.515466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:14:03.515466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:03.515466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:14:03.515466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:03.515466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:14:03.515466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:03.515466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:14:03.515466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:03.515466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:14:03.515466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:14:03.515466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:03.515466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:14:03.515466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:03.515466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:14:03.515466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:14:03.516023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:03.516023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:14:03.516023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:03.516023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:14:03.516693 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:14:03.517099 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:14:03.517291 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:14:03.517541 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:14:03.517958 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:14:03.518193 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:14:03.518354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:03.519146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:14:03.519146 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:03.519146 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:14:03.519146 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:14:03.519146 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:03.519146 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:14:03.519146 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:03.519146 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:14:03.520167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:03.520167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:14:03.520167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:14:03.520167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:14:03.520167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:14:03.520167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:14:03.520167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:14:03.520167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:03.520167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:14:03.520167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:03.520167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:14:03.520167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:03.520481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:14:03.520481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:03.520481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:14:03.520481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:03.520481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:14:03.520481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:14:03.520481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:03.520481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:14:03.520481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:03.520481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:14:03.520481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:14:03.520481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:03.520481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:14:03.520875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:03.520875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:14:03.521150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:03.521150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:14:03.521150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:14:03.521150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:14:03.521150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:14:03.521150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:14:03.521150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:14:03.521150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:14:03.521150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:14:03.521150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:14:03.521150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:14:03.521150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:14:03.521150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:14:03.521150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:14:03.521150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:14:03.521150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:14:03.521590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:14:03.521590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:14:03.521590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:03.521590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:14:03.521590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:03.521590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:14:03.521590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:03.521590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:14:03.521590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:03.521590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:14:03.521590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:03.521590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:14:03.521590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:14:03.521590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:03.521590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:14:03.521590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:03.521590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:14:03.521590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:14:03.521590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:03.522113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:14:03.522113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:03.522113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:14:07.927132 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:14:07.928083 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:14:07.928083 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:14:07.929454 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:14:07.966026 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:14:07.966405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:07.967263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:14:07.967263 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:07.967263 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:14:07.967263 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:14:07.967263 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:07.967263 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:14:07.967263 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:07.967263 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:14:07.968280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:07.968280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:14:07.968280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:14:07.968280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:14:07.968280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:14:07.968280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:14:07.968280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:14:07.968280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:07.968280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:14:07.968280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:07.968280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:14:07.968280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:07.968722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:14:07.968722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:07.968722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:14:07.968722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:07.968722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:14:07.968722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:14:07.968722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:07.968722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:14:07.968722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:07.968722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:14:07.968722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:14:07.968722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:07.968722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:14:07.969117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:07.969117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:14:07.969313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:07.969313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:14:07.969313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:14:07.969313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:14:07.969313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:14:07.969313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:14:07.969313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:14:07.969313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:14:07.969313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:14:07.969313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:14:07.969313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:14:07.969313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:14:07.969313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:14:07.969313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:14:07.969313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:14:07.969313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:14:07.969897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:14:07.969897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:14:07.969897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:07.969897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:14:07.969897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:07.969897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:14:07.969897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:07.969897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:14:07.969897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:07.969897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:14:07.969897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:07.969897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:14:07.969897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:14:07.969897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:07.969897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:14:07.969897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:07.969897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:14:07.969897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:14:07.969897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:07.970435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:14:07.970435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:07.970435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:14:08.437176 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:14:08.437764 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:14:08.437965 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:14:08.438982 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:14:08.439311 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:14:08.439472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:08.440215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:14:08.440215 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:08.440215 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:14:08.440215 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:14:08.440215 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:08.440215 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:14:08.440215 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:08.440215 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:14:08.441719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:08.441719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:14:08.441719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:14:08.441719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:14:08.441719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:14:08.441719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:14:08.441719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:14:08.441719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:08.441719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:14:08.441719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:08.441719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:14:08.441719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:08.442529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:14:08.442529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:08.442529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:14:08.442529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:08.442529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:14:08.442529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:14:08.442529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:08.442529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:14:08.442529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:08.442529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:14:08.442529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:14:08.442529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:08.442529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:14:08.443190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:08.443190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:14:08.443190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:08.443190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:14:08.443190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:14:08.443190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:14:08.443190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:14:08.443190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:14:08.443190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:14:08.443190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:14:08.443190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:14:08.443190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:14:08.443190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:14:08.443190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:14:08.443190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:14:08.443953 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:14:08.443953 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:14:08.443953 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:14:08.443953 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:14:08.443953 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:14:08.443953 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:08.443953 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:14:08.443953 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:08.443953 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:14:08.443953 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:08.443953 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:14:08.443953 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:08.443953 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:14:08.443953 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:08.443953 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:14:08.443953 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:14:08.443953 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:08.443953 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:14:08.443953 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:08.444849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:14:08.444849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:14:08.444849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:08.444849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:14:08.444849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:08.444849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:14:08.446357 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:14:08.446647 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:14:08.446778 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:14:08.447058 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:14:08.447479 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:14:08.447698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:08.448807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:14:08.448807 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:08.448807 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:14:08.448807 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:14:08.448807 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:08.448807 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:14:08.448807 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:08.448807 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:14:08.449221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:08.449221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:14:08.449221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:14:08.449221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:14:08.449221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:08.449221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:14:08.449221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:08.449221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:14:08.449221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:08.449221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:14:08.449221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:08.449221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:14:08.449221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:08.449221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:14:08.449221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:14:08.449901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:08.449901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:14:08.449901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:08.449901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:14:08.449901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:14:08.449901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:08.449901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:14:08.449901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:08.449901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:14:08.450165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:08.450165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:14:08.450165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:14:08.450165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:14:08.450165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:14:08.450165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:14:08.450165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:14:08.450165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:14:08.450165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:14:08.450165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:14:08.450165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:08.450165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:14:08.450510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:08.450510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:14:08.450510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:08.450510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:14:08.450510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:08.450510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:14:08.450510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:08.450510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:14:08.450510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:14:08.450510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:08.450510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:14:08.450510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:08.450510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:14:08.450510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:14:08.450899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:08.450899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:14:08.450899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:08.450899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:14:08.451011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:08.451011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:14:08.451011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:14:08.451011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:14:08.451011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:14:08.451011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:14:08.451011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:14:08.451011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:14:08.451011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:14:08.451011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:14:08.451011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:14:08.451011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:14:08.451011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:14:08.451011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:14:08.451011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:14:08.451011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:14:08.451431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:14:08.451431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:14:08.451431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:14:08.451431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:14:08.451431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:14:08.451431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:08.451431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:14:08.451431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:08.451431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:14:08.451431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:08.451431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:14:08.451431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:08.451431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:14:08.451431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:08.451431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:14:08.451431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:14:08.451431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:08.451431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:14:08.452607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:08.452607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:14:08.452607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:14:08.452607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:08.452607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:14:08.452607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:08.452607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:14:08.506885 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:14:08.507390 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:14:08.507694 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:14:08.508014 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:14:08.508941 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:14:08.509963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:08.510347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:14:08.510347 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:08.510347 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:14:08.510347 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:14:08.510347 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:08.510347 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:14:08.510347 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:08.510347 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:14:08.512031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:08.512031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:14:08.512031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:14:08.512031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:14:08.512031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:08.512031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:14:08.512031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:08.512031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:14:08.512031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:08.512031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:14:08.512031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:08.512031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:14:08.512031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:08.512031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:14:08.512031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:14:08.513060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:08.513060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:14:08.513060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:08.513060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:14:08.513060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:14:08.513060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:08.513060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:14:08.513060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:08.513060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:14:08.514130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:08.514130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:14:08.514130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:14:08.514130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:14:08.514130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:14:08.514130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:14:08.514130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:14:08.514130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:14:08.514130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:14:08.514130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:08.514130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:14:08.514861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:08.514861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:14:08.514861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:08.514861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:14:08.514861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:08.514861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:14:08.514861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:08.514861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:14:08.514861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:14:08.514861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:08.514861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:14:08.514861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:08.514861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:14:08.514861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:14:08.515459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:08.515459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:14:08.515459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:08.515459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:14:08.515872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:08.515872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:14:08.515872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:14:08.515872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:14:08.515872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:14:08.515872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:14:08.515872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:14:08.515872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:14:08.515872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:14:08.515872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:14:08.515872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:14:08.515872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:14:08.515872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:14:08.515872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:14:08.515872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:14:08.515872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:14:08.515872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:14:08.516422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:14:08.516422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:14:08.516422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:14:08.516422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:08.516422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:14:08.516422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:08.516422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:14:08.516422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:08.516422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:14:08.516422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:08.516422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:14:08.516422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:08.516422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:14:08.516422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:14:08.516422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:08.516422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:14:08.516422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:08.516422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:14:08.516422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:14:08.516960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:08.516960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:14:08.516960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:08.516960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:14:08.517126 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:14:08.517422 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:14:08.517573 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:14:08.517763 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:14:08.518652 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:14:08.519077 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:14:08.519293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:08.519992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:14:08.519992 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:08.519992 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:14:08.519992 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:14:08.519992 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:08.519992 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:14:08.519992 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:08.519992 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:14:08.520624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:08.520624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:14:08.520624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:14:08.520624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:14:08.520624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:14:08.520624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:14:08.520624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:14:08.520624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:08.520624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:14:08.520624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:08.520624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:14:08.520624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:08.521088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:14:08.521088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:08.521088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:14:08.521088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:08.521088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:14:08.521088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:14:08.521088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:08.521088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:14:08.521088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:08.521088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:14:08.521088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:14:08.521088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:08.521088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:14:08.521515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:08.521515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:14:08.521515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:08.521515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:14:08.521515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:14:08.521515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:14:08.521515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:14:08.521515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:14:08.521515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:14:08.521515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:14:08.521515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:14:08.521515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:14:08.521515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:14:08.521515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:14:08.521515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:14:08.522057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:14:08.522057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:14:08.522057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:14:08.522057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:14:08.522057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:14:08.522057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:08.522057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:14:08.522057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:08.522057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:14:08.522057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:08.522057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:14:08.522057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:08.522057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:14:08.522057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:08.522057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:14:08.522057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:14:08.522057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:08.522057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:14:08.522057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:08.522647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:14:08.522647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:14:08.522647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:08.522647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:14:08.522647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:08.522647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:14:12.927907 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:14:12.928888 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:14:12.928888 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:14:12.929951 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:14:12.977039 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:14:12.977265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:12.978096 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:14:12.978096 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:12.978096 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:14:12.978096 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:14:12.978096 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:12.978096 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:14:12.978096 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:12.978096 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:14:12.979374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.049 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:12.979374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:14:12.979374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:14:12.979374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:14:12.979374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:14:12.979374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:14:12.979374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:14:12.979374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:12.979374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:14:12.979374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:12.979374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:14:12.979374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:12.979873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:14:12.979873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:12.979873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:14:12.979873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:12.979873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:14:12.979873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:14:12.979873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:12.979873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:14:12.979873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:12.979873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:14:12.979873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:14:12.979873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:12.979873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:14:12.980257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:12.980257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:14:12.980515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:12.980515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:14:12.980515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:14:12.980515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:14:12.980515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:14:12.980515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:14:12.980515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:14:12.980515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:14:12.980515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:14:12.980515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:14:12.980515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:14:12.980515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:14:12.980515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:14:12.980515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:14:12.980515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:14:12.980515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:14:12.981278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:14:12.981278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:14:12.981278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:12.981278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:14:12.981278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:12.981278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:14:12.981278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:12.981278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:14:12.981278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:12.981278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:14:12.981278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:12.981278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:14:12.981278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:14:12.981278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:12.981278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:14:12.981278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:12.981278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:14:12.981278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:14:12.981278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:12.982416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:14:12.982416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:12.982416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:14:13.437998 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:14:13.438684 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:14:13.438864 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:14:13.440069 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:14:13.440485 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:14:13.440870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:13.441963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:14:13.441963 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:13.441963 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:14:13.441963 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:14:13.441963 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:13.441963 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:14:13.441963 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:13.441963 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:14:13.442923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:13.442923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:14:13.442923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:14:13.442923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:14:13.442923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:14:13.442923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:14:13.442923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:14:13.442923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:13.442923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:14:13.442923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:13.442923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:14:13.442923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:13.443627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:14:13.443627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:13.443627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:14:13.443627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:13.443627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:14:13.443627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:14:13.443627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:13.443627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:14:13.443627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:13.443627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:14:13.443627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:14:13.443627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:13.443627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:14:13.444542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:13.444542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:14:13.444542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:13.444542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:14:13.444542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:14:13.444542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:14:13.444542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:14:13.444542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:14:13.444542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:14:13.444542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:14:13.444542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:14:13.444542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:14:13.444542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:14:13.444542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:14:13.444542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:14:13.445904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:14:13.445904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:14:13.445904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:14:13.445904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:14:13.445904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:14:13.445904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:13.445904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:14:13.445904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:13.445904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:14:13.445904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:13.445904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:14:13.445904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:13.445904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:14:13.445904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:13.445904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:14:13.445904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:14:13.445904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:13.445904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:14:13.445904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:13.446502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:14:13.446502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:14:13.446502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:13.446502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:14:13.446502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:13.446502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:14:13.446856 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:14:13.447169 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:14:13.447382 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:14:13.447447 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:14:13.448106 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:14:13.448339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:13.448975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:14:13.448975 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:13.448975 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:14:13.448975 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:14:13.448975 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:13.448975 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:14:13.448975 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:13.448975 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:14:13.449764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:13.449764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:14:13.449764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:14:13.449764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:14:13.449764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:13.449764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:14:13.449764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:13.449764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:14:13.449764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:13.449764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:14:13.449764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:13.449764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:14:13.449764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:13.449764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:14:13.449764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:14:13.450245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:13.450245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:14:13.450245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:13.450245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:14:13.450245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:14:13.450245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:13.450245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:14:13.450245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:13.450245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:14:13.450653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:13.450653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:14:13.450653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:14:13.450653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:14:13.450653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:14:13.450653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:14:13.450653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:14:13.450653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:14:13.450653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:14:13.450653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:14:13.450653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:13.450653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:14:13.451154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:13.451154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:14:13.451154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:13.451154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:14:13.451154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:13.451154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:14:13.451154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:13.451154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:14:13.451154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:14:13.451154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:13.451154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:14:13.451154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:13.451154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:14:13.451154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:14:13.451539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:13.451539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:14:13.451539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:13.451539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:14:13.451830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:13.451830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:14:13.451830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:14:13.451830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:14:13.451830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:14:13.451830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:14:13.451830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:14:13.451830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:14:13.451830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:14:13.451830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:14:13.451830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:14:13.451830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:14:13.451830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:14:13.451830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:14:13.451830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:14:13.451830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:14:13.452348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:14:13.452348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:14:13.452348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:14:13.452348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:14:13.452348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:14:13.452348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:13.452348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:14:13.452348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:13.452348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:14:13.452348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:13.452348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:14:13.452348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:13.452348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:14:13.452348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:13.452348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:14:13.452348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:14:13.452348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:13.452348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:14:13.452887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:13.452887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:14:13.452887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:14:13.452887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:13.452887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:14:13.452887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:13.452887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:14:13.507297 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:14:13.507822 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:14:13.507934 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:14:13.508093 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:14:13.508965 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:14:13.509448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:13.510301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:14:13.510301 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:13.510301 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:14:13.510301 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:14:13.510301 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:13.510301 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:14:13.510301 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:13.510301 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:14:13.511440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:13.511440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:14:13.511440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:14:13.511440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:14:13.511440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:13.511440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:14:13.511440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:13.511440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:14:13.511440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:13.511440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:14:13.511440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:13.511440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:14:13.511440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:13.511440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:14:13.511440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:14:13.512327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:13.512327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:14:13.512327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:13.512327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:14:13.512327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:14:13.512327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:13.512327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:14:13.512327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:13.512327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:14:13.512327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:13.512327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:14:13.512327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:14:13.512327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:14:13.512327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:14:13.513103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:14:13.513103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:14:13.513103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:14:13.513103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:14:13.513103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:13.513103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:14:13.513103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:13.513103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:14:13.513103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:13.513103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:14:13.513103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:13.513103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:14:13.513103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:13.513795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:14:13.513795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:14:13.513795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:13.513795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:14:13.513795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:13.513795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:14:13.513795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:14:13.513795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:13.513795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:14:13.513795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:13.513795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:14:13.513795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:13.514354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:14:13.514354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:14:13.514354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:14:13.514354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:14:13.514354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:14:13.514354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:14:13.514354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:14:13.514354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:14:13.514354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:14:13.514354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:14:13.514354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:14:13.514354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:14:13.514354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:14:13.514354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:14:13.514354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:14:13.514354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:14:13.514354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:14:13.514354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:14:13.514354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:14:13.515198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:13.515198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:14:13.515198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:13.515198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:14:13.515198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:13.515198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:14:13.515198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:13.515198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:14:13.515198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:13.515198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:14:13.515198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:14:13.515198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:13.515198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:14:13.515198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:13.515198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:14:13.515198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:14:13.515198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:13.515198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:14:13.515198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:13.515198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:14:13.517712 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:14:13.518014 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:14:13.518151 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:14:13.518369 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:14:13.518864 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:14:13.519220 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:14:13.519475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:13.520220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:14:13.520220 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:13.520220 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:14:13.520220 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:14:13.520220 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:13.520220 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:14:13.520220 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:13.520220 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:14:13.521154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:13.521154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:14:13.521154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:14:13.521154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:14:13.521154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:14:13.521154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:14:13.521154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:14:13.521154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:13.521154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:14:13.521154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:13.521154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:14:13.521154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:13.521767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:14:13.521767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:13.521767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:14:13.521767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:13.521767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:14:13.521767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:14:13.521767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:13.521767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:14:13.521767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:13.521767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:14:13.521767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:14:13.521767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:13.521767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:14:13.522290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:13.522290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:14:13.522290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:13.522290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:14:13.522290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:14:13.522290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:14:13.522290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:14:13.522290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:14:13.522290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:14:13.522290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:14:13.522290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:14:13.522290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:14:13.522290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:14:13.522290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:14:13.522290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:14:13.522870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:14:13.522870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:14:13.522870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:14:13.522870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:14:13.522870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:14:13.522870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:13.522870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:14:13.522870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:13.522870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:14:13.522870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:13.522870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:14:13.522870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:13.522870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:14:13.522870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:13.522870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:14:13.522870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:14:13.522870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:13.522870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:14:13.522870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:13.524656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:14:13.524656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:14:13.524656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:13.524656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:14:13.524656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:13.524656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:14:17.928585 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:14:17.929430 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:14:17.929430 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:14:17.930673 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:14:17.966348 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:14:17.966703 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:17.967705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:14:17.967705 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:17.967705 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:14:17.967705 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:14:17.967705 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:17.967705 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:14:17.967705 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:17.967705 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:14:17.968484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:17.968484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:14:17.968484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:14:17.968484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:14:17.968484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:14:17.968484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:14:17.968484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:14:17.968484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:17.968484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:14:17.968484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:17.968484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:14:17.968484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:17.969283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:14:17.969283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:17.969283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:14:17.969283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:17.969283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:14:17.969283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:14:17.969283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:17.969283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:14:17.969283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:17.969283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:14:17.969283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:14:17.969283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:17.969283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:14:17.970079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:17.970079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:14:17.970079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:17.970079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:14:17.970079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:14:17.970079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:14:17.970079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:14:17.970079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:14:17.970079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:14:17.970079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:14:17.970079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:14:17.970079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:14:17.970079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:14:17.970079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:14:17.970079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:14:17.971795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:14:17.971795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:14:17.971795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:14:17.971795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:14:17.971795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:14:17.971795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:17.971795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:14:17.971795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:17.971795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:14:17.971795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:17.971795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:14:17.971795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:17.971795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:14:17.971795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:17.971795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:14:17.971795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:14:17.971795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:17.971795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:14:17.971795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:17.972439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:14:17.972439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:14:17.972439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:17.972439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:14:17.972439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:17.972439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:14:18.438716 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:14:18.439450 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:14:18.439651 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:14:18.440910 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:14:18.441274 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:14:18.441531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:18.442436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:14:18.442436 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:18.442436 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:14:18.442436 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:14:18.442436 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:18.442436 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:14:18.442436 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:18.442436 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:14:18.444074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:18.444074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:14:18.444074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:14:18.444074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:14:18.444074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:14:18.444074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:14:18.444074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:14:18.444074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:18.444074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:14:18.444074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:18.444074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:14:18.444074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:18.444438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:14:18.444438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:18.444438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:14:18.444438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:18.444438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:14:18.444438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:14:18.444438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:18.444438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:14:18.444438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:18.444438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:14:18.444438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:14:18.444438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:18.444438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:14:18.444849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:18.444849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:14:18.446070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:18.446070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:14:18.446070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:14:18.446070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:14:18.446070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:14:18.446070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:14:18.446070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:14:18.446070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:14:18.446070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:14:18.446070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:14:18.446070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:14:18.446070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:14:18.446070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:14:18.446070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:14:18.446070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:14:18.446070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:14:18.446519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:14:18.446519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:14:18.446519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:18.446519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:14:18.446519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:18.446519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:14:18.446519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:18.446519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:14:18.446519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:18.446519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:14:18.446519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:18.446519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:14:18.446519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:14:18.446519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:18.446519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:14:18.446519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:18.446519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:14:18.446519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:14:18.446519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:18.447074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:14:18.447074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:18.447074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:14:18.447922 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:14:18.448352 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:14:18.448500 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:14:18.448628 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:14:18.449188 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:14:18.449542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:18.450329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:14:18.450329 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:18.450329 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:14:18.450329 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:14:18.450329 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:18.450329 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:14:18.450329 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:18.450329 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:14:18.451208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:18.451208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:14:18.451208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:14:18.451208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:14:18.451208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:18.451208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:14:18.451208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:18.451208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:14:18.451208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:18.451208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:14:18.451208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:18.451208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:14:18.451208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:18.451208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:14:18.451208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:14:18.451948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:18.451948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:14:18.451948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:18.451948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:14:18.451948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:14:18.451948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:18.451948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:14:18.451948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:18.451948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:14:18.452378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:18.452378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:14:18.452378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:14:18.452378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:14:18.452378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:14:18.452378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:14:18.452378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:14:18.452378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:14:18.452378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:14:18.452378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:14:18.452378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:18.452378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:14:18.453018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:18.453018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:14:18.453018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:18.453018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:14:18.453018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:18.453018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:14:18.453018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:18.453018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:14:18.453018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:14:18.453018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:18.453018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:14:18.453018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:18.453018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:14:18.453018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:14:18.453664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:18.453664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:14:18.453664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:18.453664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:14:18.453664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:18.453664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:14:18.453664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:14:18.453664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:14:18.453664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:14:18.453664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:14:18.453664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:14:18.453664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:14:18.453664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:14:18.453664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:14:18.454354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:14:18.454354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:14:18.454354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:14:18.454354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:14:18.454354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:14:18.454354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:14:18.454354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:14:18.454354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:14:18.454354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:14:18.454354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:14:18.454354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:14:18.454354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:18.454354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:14:18.454354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:18.454354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:14:18.454354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:18.454354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:14:18.454354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:18.454354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:14:18.455259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:18.455259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:14:18.455259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:14:18.455259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:18.455259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:14:18.455259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:18.455259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:14:18.455259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:14:18.455259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:18.455259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:14:18.455259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:18.455259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:14:18.507977 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:14:18.508335 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:14:18.508604 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:14:18.508982 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:14:18.509771 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:14:18.510192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:18.511049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:14:18.511049 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:18.511049 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:14:18.511049 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:14:18.511049 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:18.511049 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:14:18.511049 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:18.511049 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:14:18.512507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:18.512507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:14:18.512507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:14:18.512507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:14:18.512507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:18.512507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:14:18.512507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:18.512507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:14:18.512507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:18.512507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:14:18.512507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:18.512507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:14:18.512507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:18.512507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:14:18.512507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:14:18.513215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:18.513215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:14:18.513215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:18.513215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:14:18.513215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:14:18.513215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:18.513215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:14:18.513215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:18.513215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:14:18.514337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:18.514337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:14:18.514337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:14:18.514337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:14:18.514337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:14:18.514337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:14:18.514337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:14:18.514337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:14:18.514337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:14:18.514337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:18.514337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:14:18.514888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:18.514888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:14:18.514888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:18.514888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:14:18.514888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:18.514888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:14:18.514888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:18.514888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:14:18.514888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:14:18.514888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:18.514888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:14:18.514888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:18.514888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:14:18.514888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:14:18.515529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:18.515529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:14:18.515529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:18.515529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:14:18.516527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:18.516527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:14:18.516527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:14:18.516527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:14:18.516527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:14:18.516527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:14:18.516527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:14:18.516527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:14:18.516527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:14:18.516527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:14:18.516527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:14:18.516527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:14:18.516527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:14:18.516527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:14:18.516527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:14:18.516527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:14:18.516527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:14:18.517399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:14:18.517399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:14:18.517399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:14:18.517399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:18.517399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:14:18.517399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:18.517399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:14:18.517399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:18.517399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:14:18.517399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:18.517399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:14:18.517399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:18.517399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:14:18.517399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:14:18.517399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:18.517399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:14:18.517399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:18.517399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:14:18.517399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:14:18.518162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:18.518162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:14:18.518162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:18.518162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:14:18.518162 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:14:18.518524 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:14:18.518676 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:14:18.518935 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:14:18.519522 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:14:18.519989 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:14:18.520216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:18.521076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:14:18.521076 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:18.521076 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:14:18.521076 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:14:18.521076 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:18.521076 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:14:18.521076 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:18.521076 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:14:18.522137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:18.522137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:14:18.522137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:14:18.522137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:14:18.522137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:14:18.522137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:14:18.522137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:14:18.522137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:18.522137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:14:18.522137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:18.522137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:14:18.522137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:18.522458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:14:18.522458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:18.522458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:14:18.522458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:18.522458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:14:18.522458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:14:18.522458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:18.522458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:14:18.522458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:18.522458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:14:18.522458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:14:18.522458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:18.522458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:14:18.522835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:18.522835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:14:18.523449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:18.523449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:14:18.523449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:14:18.523449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:14:18.523449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:14:18.523449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:14:18.523449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:14:18.523449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:14:18.523449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:14:18.523449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:14:18.523449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:14:18.523449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:14:18.523449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:14:18.523449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:14:18.523449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:14:18.523449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:14:18.523921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:14:18.523921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:14:18.523921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:18.523921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:14:18.523921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:18.523921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:14:18.523921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:18.523921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:14:18.523921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:18.523921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:14:18.523921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:18.523921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:14:18.523921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:14:18.523921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:18.523921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:14:18.523921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:18.523921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:14:18.523921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:14:18.523921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:18.524661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:14:18.524661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:18.524661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:14:22.929105 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:14:22.929773 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:14:22.929773 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:14:22.930464 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:14:22.967035 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:14:22.967466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:22.968314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:14:22.968314 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:22.968314 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:14:22.968314 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:14:22.968314 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:22.968314 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:14:22.968314 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:22.968314 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:14:22.969018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:22.969018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:14:22.969018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:14:22.969018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:14:22.969018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:14:22.969018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:14:22.969018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:14:22.969018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:22.969018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:14:22.969018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:22.969018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:14:22.969018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:22.969864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:14:22.969864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:22.969864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:14:22.969864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:22.969864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:14:22.969864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:14:22.969864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:22.969864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:14:22.969864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:22.969864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:14:22.969864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:14:22.969864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:22.969864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:14:22.970454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:22.970454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:14:22.970454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:22.970454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:14:22.970454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:14:22.970454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:14:22.970454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:14:22.970454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:14:22.970454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:14:22.970454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:14:22.970454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:14:22.970454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:14:22.970454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:14:22.970454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:14:22.970454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:14:22.970974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:14:22.970974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:14:22.970974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:14:22.970974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:14:22.970974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:14:22.970974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:22.970974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:14:22.970974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:22.970974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:14:22.970974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:22.970974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:14:22.970974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:22.970974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:14:22.970974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:22.970974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:14:22.970974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:14:22.970974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:22.970974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:14:22.970974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:22.971643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:14:22.971643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:14:22.971643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:22.971643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:14:22.971643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:22.971643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:14:23.439645 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:14:23.440438 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:14:23.440438 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:14:23.441521 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:14:23.441914 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:14:23.442241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:23.443079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:14:23.443079 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:23.443079 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:14:23.443079 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:14:23.443079 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:23.443079 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:14:23.443079 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:23.443079 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:14:23.444151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:23.444151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:14:23.444151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:14:23.444151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:14:23.444151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:14:23.444151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:14:23.444151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:14:23.444151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:23.444151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:14:23.444151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:23.444151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:14:23.444151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:23.444583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:14:23.444583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:23.444583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:14:23.444583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:23.444583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:14:23.444583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:14:23.444583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:23.444583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:14:23.444583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:23.444583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:14:23.444583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:14:23.444583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:23.444583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:14:23.445086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:23.445086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:14:23.445627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:23.445627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:14:23.445627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:14:23.445627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:14:23.445627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:14:23.445627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:14:23.445627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:14:23.445627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:14:23.445627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:14:23.445627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:14:23.445627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:14:23.445627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:14:23.445627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:14:23.445627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:14:23.445627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:14:23.445627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:14:23.446347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:14:23.446347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:14:23.446347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:23.446347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:14:23.446347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:23.446347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:14:23.446347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:23.446347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:14:23.446347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:23.446347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:14:23.446347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:23.446347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:14:23.446347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:14:23.446347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:23.446347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:14:23.446347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:23.446347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:14:23.446347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:14:23.446347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:23.446876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:14:23.446876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:23.446876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:14:23.448334 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:14:23.448609 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:14:23.448721 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:14:23.448860 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:14:23.449502 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:14:23.449908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:23.450795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:14:23.450795 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:23.450795 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:14:23.450795 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:14:23.450795 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:23.450795 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:14:23.450795 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:23.450795 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:14:23.451569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:23.451569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:14:23.451569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:14:23.451569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:14:23.451569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:23.451569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:14:23.451569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:23.451569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:14:23.451569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:23.451569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:14:23.451569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:23.451569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:14:23.451569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:23.451569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:14:23.451569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:14:23.452107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:23.452107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:14:23.452107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:23.452107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:14:23.452107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:14:23.452107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:23.452107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:14:23.452107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:23.452107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:14:23.452462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:23.452462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:14:23.452462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:14:23.452462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:14:23.452462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:14:23.452462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:14:23.452462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:14:23.452462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:14:23.452462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:14:23.452462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:14:23.452462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:23.452462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:14:23.452946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:23.452946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:14:23.452946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:23.452946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:14:23.452946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:23.452946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:14:23.452946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:23.452946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:14:23.452946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:14:23.452946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:23.452946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:14:23.452946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:23.452946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:14:23.452946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:14:23.453330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:23.453330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:14:23.453330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:23.453330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:14:23.453526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:23.453526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:14:23.453526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:14:23.453526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:14:23.453526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:14:23.453526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:14:23.453526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:14:23.453526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:14:23.453526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:14:23.453526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:14:23.453526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:14:23.453526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:14:23.453526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:14:23.453526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:14:23.453526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:14:23.453526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:14:23.454129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:14:23.454129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:14:23.454129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:14:23.454129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:14:23.454129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:14:23.454129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:23.454129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:14:23.454129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:23.454129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:14:23.454129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:23.454129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:14:23.454129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:23.454129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:14:23.454129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:23.454129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:14:23.454129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:14:23.454129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:23.454129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:14:23.454819 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:23.454819 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:14:23.454819 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:14:23.454819 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:23.454819 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:14:23.454819 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:23.454819 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:14:23.508671 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:14:23.508991 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:14:23.509107 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:14:23.509311 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:14:23.510060 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:14:23.510527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:23.511405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:14:23.511405 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:23.511405 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:14:23.511405 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:14:23.511405 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:23.511405 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:14:23.511405 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:23.511405 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:14:23.513304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:23.513304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:14:23.513304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:14:23.513304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:14:23.513304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:23.513304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:14:23.513304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:23.513304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:14:23.513304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:23.513304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:14:23.513304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:23.513304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:14:23.513304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:23.513304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:14:23.513304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:14:23.514392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:23.514392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:14:23.514392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:23.514392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:14:23.514392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:14:23.514392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:23.514392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:14:23.514392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:23.514392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:14:23.514392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:23.514392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:14:23.514392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:14:23.514392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:14:23.514392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:14:23.515048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:14:23.515048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:14:23.515048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:14:23.515048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:14:23.515048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:23.515048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:14:23.515048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:23.515048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:14:23.515048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:23.515048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:14:23.515048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:23.515048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:14:23.515048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:23.515677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:14:23.515677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:14:23.515677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:23.515677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:14:23.515677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:23.515677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:14:23.515677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:14:23.515677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:23.515677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:14:23.515677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:23.515677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:14:23.515677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:23.516377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:14:23.516377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:14:23.516377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:14:23.516377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:14:23.516377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:14:23.516377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:14:23.516377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:14:23.516377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:14:23.516377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:14:23.516377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:14:23.516377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:14:23.516377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:14:23.516377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:14:23.516377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:14:23.516377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:14:23.516377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:14:23.516377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:14:23.516377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:14:23.516377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:14:23.518214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:23.518214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:14:23.518214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:23.518214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:14:23.518214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:23.518214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:14:23.518214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:23.518214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:14:23.518214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:23.518214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:14:23.518214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:14:23.518214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:23.518214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:14:23.518214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:23.518214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:14:23.518214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:14:23.518214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:23.518214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:14:23.518214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:23.518214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:14:23.519043 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:14:23.519043 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:14:23.519043 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:14:23.521027 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:14:23.521027 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:14:23.521027 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:14:23.521027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:23.521714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:14:23.521714 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:23.521714 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:14:23.521714 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:14:23.521714 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:23.521714 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:14:23.521714 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:23.521714 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:14:23.522815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:23.522815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:14:23.522815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:14:23.522815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:14:23.522815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:14:23.522815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:14:23.522815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:14:23.522815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:23.522815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:14:23.522815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:23.522815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:14:23.522815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:23.523153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:14:23.523153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:23.523153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:14:23.523153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:23.523153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:14:23.523153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:14:23.523153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:23.523153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:14:23.523153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:23.523153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:14:23.523153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:14:23.523153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:23.523153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:14:23.523381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:23.523381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:14:23.523789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:23.523789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:14:23.523789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:14:23.523789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:14:23.523789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:14:23.523789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:14:23.523789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:14:23.523789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:14:23.523789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:14:23.523789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:14:23.523789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:14:23.523789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:14:23.523789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:14:23.523789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:14:23.523789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:14:23.523789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:14:23.524080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:14:23.524080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:14:23.524080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:23.524080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:14:23.524080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:23.524080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:14:23.524080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:23.524080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:14:23.524080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:23.524080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:14:23.524080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:23.524080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:14:23.524080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:14:23.524080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:23.524080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:14:23.524080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:23.524080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:14:23.524080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:14:23.524080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:23.525121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:14:23.525121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:23.525121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:14:27.929965 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:14:27.931079 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:14:27.931079 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:14:27.932463 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:14:27.967889 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:14:27.967889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:27.968714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:14:27.968714 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:27.968714 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:14:27.968714 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:14:27.968714 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:27.968714 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:14:27.968714 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:27.968714 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:14:27.969498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:27.969498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:14:27.969498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:14:27.969498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:14:27.969498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:14:27.969498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:14:27.969498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:14:27.969498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:27.969498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:14:27.969498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:27.969498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:14:27.969498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:27.970303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:14:27.970303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:27.970303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:14:27.970303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:27.970303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:14:27.970303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:14:27.970303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:27.970303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:14:27.970303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:27.970303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:14:27.970303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:14:27.970303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:27.970303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:14:27.970969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:27.970969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:14:27.970969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:27.970969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:14:27.970969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:14:27.970969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:14:27.970969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:14:27.970969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:14:27.970969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:14:27.970969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:14:27.970969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:14:27.970969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:14:27.970969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:14:27.970969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:14:27.970969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:14:27.971916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:14:27.971916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:14:27.971916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:14:27.971916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:14:27.971916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:14:27.971916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:27.971916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:14:27.971916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:27.971916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:14:27.971916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:27.971916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:14:27.971916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:27.971916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:14:27.971916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:27.971916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:14:27.971916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:14:27.971916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:27.971916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:14:27.971916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:27.974138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:14:27.974138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:14:27.974138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:27.974138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:14:27.974138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:27.974138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:14:28.440413 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:14:28.440910 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:14:28.441140 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:14:28.442377 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:14:28.442785 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:14:28.443096 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:28.444117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:14:28.444117 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:28.444117 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:14:28.444117 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:14:28.444117 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:28.444117 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:14:28.444117 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:28.444117 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:14:28.445709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:28.445709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:14:28.445709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:14:28.445709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:14:28.445709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:14:28.445709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:14:28.445709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:14:28.445709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:28.445709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:14:28.445709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:28.445709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:14:28.445709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:28.446454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:14:28.446454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:28.446454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:14:28.446454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:28.446454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:14:28.446454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:14:28.446454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:28.446454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:14:28.446454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:28.446454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:14:28.446454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:14:28.446454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:28.446454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:14:28.447442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:28.447442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:14:28.447660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:28.447660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:14:28.447660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:14:28.447660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:14:28.447660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:14:28.447660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:14:28.447660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:14:28.447660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:14:28.447660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:14:28.447660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:14:28.447660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:14:28.447660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:14:28.447660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:14:28.447660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:14:28.447660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:14:28.447660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:14:28.449072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:14:28.449072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:14:28.449072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:28.449072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:14:28.449072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:28.449072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:14:28.449072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:28.449072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:14:28.449072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:28.449072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:14:28.449072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:28.449072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:14:28.449072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:14:28.449072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:28.449072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:14:28.449072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:28.449072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:14:28.449072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:14:28.449072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:28.449971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:14:28.449971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:28.449971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:14:28.449971 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:14:28.449971 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:14:28.449971 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:14:28.449971 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:14:28.450347 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:14:28.450347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:28.450829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:14:28.450829 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:28.450829 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:14:28.450829 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:14:28.450829 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:28.450829 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:14:28.450829 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:28.450829 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:14:28.451564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:28.451564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:14:28.451564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:14:28.451564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:14:28.451564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:28.451564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:14:28.451564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:28.451564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:14:28.451564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:28.451564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:14:28.451564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:28.451564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:14:28.451564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:28.451564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:14:28.451564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:14:28.452286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:28.452286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:14:28.452286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:28.452286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:14:28.452286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:14:28.452286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:28.452286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:14:28.452286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:28.452286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:14:28.452689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:28.452689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:14:28.452689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:14:28.452689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:14:28.452689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:14:28.452689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:14:28.452689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:14:28.452689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:14:28.452689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:14:28.452689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:14:28.452689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:28.452689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:14:28.453396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:28.453396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:14:28.453396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:28.453396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:14:28.453396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:28.453396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:14:28.453396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:28.453396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:14:28.453396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:14:28.453396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:28.453396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:14:28.453396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:28.453396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:14:28.453396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:14:28.454169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:28.454169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:14:28.454169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:28.454169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:14:28.454169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:28.454169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:14:28.454169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:14:28.454169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:14:28.454169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:14:28.454169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:14:28.454169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:14:28.454169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:14:28.454169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:14:28.454169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:14:28.454631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:14:28.454631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:14:28.454631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:14:28.454631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:14:28.454631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:14:28.454631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:14:28.454631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:14:28.454631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:14:28.454631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:14:28.454631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:14:28.454631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:14:28.454631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:28.454631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:14:28.454631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:28.454631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:14:28.454631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:28.454631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:14:28.454631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:28.454631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:14:28.455126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:28.455126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:14:28.455126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:14:28.455126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:28.455126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:14:28.455126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:28.455126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:14:28.455126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:14:28.455126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:28.455126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:14:28.455126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:28.455126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:14:28.509254 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:14:28.509830 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:14:28.510074 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:14:28.510367 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:14:28.511309 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:14:28.511763 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:28.513020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:14:28.513020 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:28.513020 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:14:28.513020 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:14:28.513020 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:28.513020 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:14:28.513020 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:28.513020 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:14:28.514292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:28.514292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:14:28.514292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:14:28.514292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:14:28.514292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:28.514292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:14:28.514292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:28.514292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:14:28.514292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:28.514292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:14:28.514292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:28.514292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:14:28.514292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:28.514292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:14:28.514292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:14:28.515175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:28.515175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:14:28.515175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:28.515175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:14:28.515175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:14:28.515175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:28.515175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:14:28.515175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:28.515175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:14:28.515842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:28.515842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:14:28.515842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:14:28.515842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:14:28.515842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:14:28.515842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:14:28.515842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:14:28.515842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:14:28.515842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:14:28.515842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:28.515842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:14:28.516629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:28.516629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:14:28.516629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:28.516629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:14:28.516629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:28.516629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:14:28.516629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:28.516629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:14:28.516629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:14:28.516629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:28.516629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:14:28.516629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:28.516629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:14:28.516629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:14:28.517551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:28.517551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:14:28.517551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:28.517551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:14:28.517865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:28.517865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:14:28.517865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:14:28.517865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:14:28.517865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:14:28.517865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:14:28.517865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:14:28.517865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:14:28.517865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:14:28.517865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:14:28.517865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:14:28.517865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:14:28.517865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:14:28.517865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:14:28.517865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:14:28.517865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:14:28.517865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:14:28.519111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:14:28.519111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:14:28.519111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:14:28.519111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:28.519111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:14:28.519111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:28.519111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:14:28.519111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:28.519111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:14:28.519111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:28.519111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:14:28.519111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:28.519111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:14:28.519111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:14:28.519111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:28.519111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:14:28.519111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:28.519111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:14:28.519111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:14:28.520669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:28.520669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:14:28.520669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:28.520669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:14:28.520669 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:14:28.520669 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:14:28.520669 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:14:28.520669 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:14:28.521309 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:14:28.521309 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:14:28.521309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:28.521309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:14:28.521309 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:28.521309 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:14:28.521309 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:14:28.521309 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:28.521309 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:14:28.521309 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:28.521309 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:14:28.522037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:28.522037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:14:28.522037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:14:28.522037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:14:28.522037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:14:28.522037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:14:28.522037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:14:28.522037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:28.522037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:14:28.522037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:28.522037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:14:28.522037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:28.522639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:14:28.522639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:28.522639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:14:28.522639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:28.522639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:14:28.522639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:14:28.522639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:28.522639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:14:28.522639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:28.522639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:14:28.522639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:14:28.522639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:28.522639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:14:28.523201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:28.523201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:14:28.523201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:28.523201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:14:28.523201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:14:28.523201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:14:28.523201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:14:28.523201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:14:28.523201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:14:28.523201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:14:28.523201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:14:28.523201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:14:28.523201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:14:28.523201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:14:28.523201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:14:28.523993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:14:28.523993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:14:28.523993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:14:28.523993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:14:28.523993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:14:28.523993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:28.523993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:14:28.523993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:28.523993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:14:28.523993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:28.523993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:14:28.523993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:28.523993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:14:28.523993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:28.523993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:14:28.523993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:14:28.523993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:28.523993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:14:28.523993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:28.524956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:14:28.524956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:14:28.524956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:28.524956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:14:28.524956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:28.524956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:14:32.930788 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:14:32.931411 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:14:32.931411 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:14:32.932120 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:14:32.969550 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:14:32.969824 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:32.970594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:14:32.970594 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:32.970594 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:14:32.970594 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:14:32.970594 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:32.970594 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:14:32.970594 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:32.970594 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:14:32.971500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:32.971500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:14:32.971500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:14:32.971500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:14:32.971500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:14:32.971500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:14:32.971500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:14:32.971500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:32.971500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:14:32.971500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:32.971500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:14:32.971500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:32.972434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:14:32.972434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:32.972434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:14:32.972434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:32.972434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:14:32.972434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:14:32.972434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:32.972434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:14:32.972434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:32.972434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:14:32.972434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:14:32.972434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:32.972434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:14:32.974050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:32.974050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:14:32.974050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:32.974050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:14:32.974050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:14:32.974050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:14:32.974050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:14:32.974050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:14:32.974050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:14:32.974050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:14:32.974050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:14:32.974050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:14:32.974050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:14:32.974050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:14:32.974050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:14:32.974777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:14:32.974777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:14:32.974777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:14:32.974777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:14:32.974777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:14:32.974777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:32.974777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:14:32.974777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:32.974777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:14:32.974777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:32.974777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:14:32.974777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:32.974777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:14:32.974777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:32.974777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:14:32.974777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:14:32.974777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:32.974777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:14:32.974777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:32.975692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:14:32.975692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:14:32.975692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:32.975692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:14:32.975692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:32.975692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:14:33.441058 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:14:33.442130 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:14:33.442321 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:14:33.443692 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:14:33.444125 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:14:33.444365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:33.445780 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:14:33.445780 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:33.445780 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:14:33.445780 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:14:33.445780 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:33.445780 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:14:33.445780 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:33.445780 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:14:33.447897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:33.447897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:14:33.447897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:14:33.447897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:14:33.447897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:14:33.447897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:14:33.447897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:14:33.447897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:33.447897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:14:33.447897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:33.447897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:14:33.447897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:33.448633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:14:33.448633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:33.448633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:14:33.448633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:33.448633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:14:33.448633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:14:33.448633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:33.448633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:14:33.448633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:33.448633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:14:33.448633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:14:33.448633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:33.448633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:14:33.449252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:33.449252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:14:33.449702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:33.449702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:14:33.449702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:14:33.449702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:14:33.449702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:14:33.449702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:14:33.449702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:14:33.449702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:14:33.449702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:14:33.449702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:14:33.449702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:14:33.449702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:14:33.449702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:14:33.449702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:14:33.449702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:14:33.449702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:14:33.450496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:14:33.450496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:14:33.450496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:33.450496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:14:33.450496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:33.450496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:14:33.450496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:33.450496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:14:33.450496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:33.450496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:14:33.450496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:33.450496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:14:33.450496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:14:33.450496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:33.450496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:14:33.450496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:33.450496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:14:33.450496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:14:33.450496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:33.451084 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:14:33.451084 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:33.451084 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:14:33.451084 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:14:33.451084 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:14:33.451084 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:14:33.451084 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:14:33.451565 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:14:33.451799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:33.452385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:14:33.452385 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:33.452385 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:14:33.452385 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:14:33.452385 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:33.452385 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:14:33.452385 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:33.452385 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:14:33.453252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:33.453252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:14:33.453252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:14:33.453252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:14:33.453252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:33.453252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:14:33.453252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:33.453252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:14:33.453252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:33.453252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:14:33.453252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:33.453252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:14:33.453252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:33.453252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:14:33.453252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:14:33.453778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:33.453778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:14:33.453778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:33.453778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:14:33.453778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:14:33.453778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:33.453778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:14:33.453778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:33.453778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:14:33.454113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:33.454113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:14:33.454113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:14:33.454113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:14:33.454113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:14:33.454113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:14:33.454113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:14:33.454113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:14:33.454113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:14:33.454113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:14:33.454113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:33.454113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:14:33.454490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:33.454490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:14:33.454490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:33.454490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:14:33.454490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:33.454490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:14:33.454490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:33.454490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:14:33.454490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:14:33.454490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:33.454490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:14:33.454490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:33.454490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:14:33.454490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:14:33.454921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:33.454921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:14:33.454921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:33.454921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:14:33.455284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:33.455284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:14:33.455284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:14:33.455284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:14:33.455284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:14:33.455284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:14:33.455284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:14:33.455284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:14:33.455284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:14:33.455284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:14:33.455284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:14:33.455284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:14:33.455284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:14:33.455284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:14:33.455284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:14:33.455284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:14:33.455770 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:14:33.455770 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:14:33.455770 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:14:33.455770 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:14:33.455770 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:14:33.455770 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:33.455770 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:14:33.455770 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:33.455770 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:14:33.455770 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:33.455770 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:14:33.455770 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:33.455770 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:14:33.455770 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:33.455770 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:14:33.455770 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:14:33.455770 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:33.455770 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:14:33.456283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:33.456283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:14:33.456283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:14:33.456283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:33.456283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:14:33.456283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:33.456283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:14:33.510148 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:14:33.510556 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:14:33.510776 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:14:33.510999 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:14:33.511698 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:14:33.512154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:33.513141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:14:33.513141 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:33.513141 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:14:33.513141 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:14:33.513141 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:33.513141 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:14:33.513141 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:33.513141 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:14:33.514509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:33.514509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:14:33.514509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:14:33.514509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:14:33.514509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:33.514509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:14:33.514509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:33.514509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:14:33.514509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:33.514509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:14:33.514509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:33.514509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:14:33.514509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:33.514509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:14:33.514509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:14:33.515207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:33.515207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:14:33.515207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:33.515207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:14:33.515207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:14:33.515207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:33.515207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:14:33.515207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:33.515207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:14:33.516922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:33.516922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:14:33.516922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:14:33.516922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:14:33.516922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:14:33.516922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:14:33.516922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:14:33.516922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:14:33.516922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:14:33.516922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:33.516922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:14:33.517533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:33.517533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:14:33.517533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:33.517533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:14:33.517533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:33.517533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:14:33.517533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:33.517533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:14:33.517533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:14:33.517533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:33.517533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:14:33.517533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:33.517533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:14:33.517533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:14:33.518172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:33.518172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:14:33.518172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:33.518172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:14:33.518172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:33.518172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:14:33.518172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:14:33.518172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:14:33.518172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:14:33.518172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:14:33.518172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:14:33.518172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:14:33.518172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:14:33.518172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:14:33.518833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:14:33.518833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:14:33.518833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:14:33.518833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:14:33.518833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:14:33.518833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:14:33.518833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:14:33.518833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:14:33.518833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:14:33.518833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:14:33.518833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:33.518833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:14:33.518833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:33.518833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:14:33.518833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:33.518833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:14:33.518833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:33.518833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:14:33.518833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:33.519603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:14:33.519603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:14:33.519603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:33.519603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:14:33.519603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:33.519603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:14:33.519603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:14:33.519603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:33.519603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:14:33.519603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:33.519603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:14:33.519603 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:14:33.520148 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:14:33.520148 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:14:33.520148 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:14:33.520593 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:14:33.521024 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:14:33.521254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:33.522105 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:14:33.522105 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:33.522105 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:14:33.522105 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:14:33.522105 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:33.522105 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:14:33.522105 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:33.522105 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:14:33.522964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:33.522964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:14:33.522964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:14:33.522964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:14:33.522964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:14:33.522964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:14:33.522964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:14:33.522964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:33.522964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:14:33.522964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:33.522964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:14:33.522964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:33.524390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:14:33.524390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:33.524390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:14:33.524390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:33.524390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:14:33.524390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:14:33.524390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:33.524390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:14:33.524390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:33.524390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:14:33.524390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:14:33.524390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:33.524390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:14:33.525168 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:33.525168 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:14:33.525168 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:33.525168 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:14:33.525168 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:14:33.525168 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:14:33.525168 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:14:33.525168 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:14:33.525168 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:14:33.525168 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:14:33.525168 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:14:33.525168 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:14:33.525168 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:14:33.525168 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:14:33.525168 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:14:33.525644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:14:33.525644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:14:33.525644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:14:33.525644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:14:33.525644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:14:33.525644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:33.525644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:14:33.525644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:33.525644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:14:33.525644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:33.525644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:14:33.525644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:33.525644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:14:33.525644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:33.525644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:14:33.525644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:14:33.525644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:33.525644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:14:33.525644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:33.526205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:14:33.526205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:14:33.526205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:33.526205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:14:33.526205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:33.526205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:14:37.931442 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:14:37.932533 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:14:37.932533 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:14:37.934190 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:14:37.969712 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:14:37.970017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:37.970766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:14:37.970766 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:37.970766 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:14:37.970766 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:14:37.970766 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:37.970766 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:14:37.970766 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:37.970766 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:14:37.971927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:37.971927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:14:37.971927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:14:37.971927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:14:37.971927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:14:37.971927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:14:37.971927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:14:37.971927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:37.971927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:14:37.971927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:37.971927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:14:37.971927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:37.972664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:14:37.972664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:37.972664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:14:37.972664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:37.972664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:14:37.972664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:14:37.972664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:37.972664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:14:37.972664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:37.972664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:14:37.972664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:14:37.972664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:37.972664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:14:37.973388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:37.973388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:14:37.973388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:37.973388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:14:37.973388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:14:37.973388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:14:37.973388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:14:37.973388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:14:37.973388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:14:37.973388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:14:37.973388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:14:37.973388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:14:37.973388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:14:37.973388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:14:37.973388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:14:37.974310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:14:37.974310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:14:37.974310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:14:37.974310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:14:37.974310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:14:37.974310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:37.974310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:14:37.974310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:37.974310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:14:37.974310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:37.974310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:14:37.974310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:37.974310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:14:37.974310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:37.974310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:14:37.974310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:14:37.974310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:37.974310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:14:37.974310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:37.975368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:14:37.975368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:14:37.975368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:37.975368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:14:37.975368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:37.975368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:14:38.442117 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:14:38.442623 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:14:38.442874 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:14:38.444129 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:14:38.444490 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:14:38.444765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:38.445704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:14:38.445704 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:38.445704 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:14:38.445704 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:14:38.445704 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:38.445704 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:14:38.445704 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:38.445704 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:14:38.447559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:38.447559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:14:38.447559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:14:38.447559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:14:38.447559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:14:38.447559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:14:38.447559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:14:38.447559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:38.447559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:14:38.447559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:38.447559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:14:38.447559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:38.448257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:14:38.448257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:38.448257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:14:38.448257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:38.448257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:14:38.448257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:14:38.448257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:38.448257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:14:38.448257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:38.448257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:14:38.448257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:14:38.448257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:38.448257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:14:38.448834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:38.448834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:14:38.449730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:38.449730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:14:38.449730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:14:38.449730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:14:38.449730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:14:38.449730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:14:38.449730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:14:38.449730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:14:38.449730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:14:38.449730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:14:38.449730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:14:38.449730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:14:38.449730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:14:38.449730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:14:38.449730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:14:38.449730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:14:38.450634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:14:38.450634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:14:38.450634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:38.450634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:14:38.450634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:38.450634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:14:38.450634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:38.450634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:14:38.450634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:38.450634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:14:38.450634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:38.450634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:14:38.450634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:14:38.450634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:38.450634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:14:38.450634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:38.450634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:14:38.450634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:14:38.450634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:38.451513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:14:38.451513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:38.451513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:14:38.451513 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:14:38.451513 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:14:38.451513 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:14:38.451812 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:14:38.452456 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:14:38.452819 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:38.453515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:14:38.453515 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:38.453515 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:14:38.453515 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:14:38.453515 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:38.453515 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:14:38.453515 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:38.453515 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:14:38.454301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:38.454301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:14:38.454301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:14:38.454301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:14:38.454301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:38.454301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:14:38.454301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:38.454301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:14:38.454301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:38.454301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:14:38.454301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:38.454301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:14:38.454301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:38.454301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:14:38.454301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:14:38.454801 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:38.454801 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:14:38.454801 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:38.454801 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:14:38.454801 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:14:38.454801 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:38.454801 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:14:38.454801 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:38.454801 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:14:38.455185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:38.455185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:14:38.455185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:14:38.455185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:14:38.455185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:14:38.455185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:14:38.455185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:14:38.455185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:14:38.455185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:14:38.455185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:14:38.455185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:38.455185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:14:38.455564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:38.455564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:14:38.455564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:38.455564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:14:38.455564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:38.455564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:14:38.455564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:38.455564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:14:38.455564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:14:38.455564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:38.455564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:14:38.455564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:38.455564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:14:38.455564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:14:38.455939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:38.455939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:14:38.455939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:38.455939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:14:38.456258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:38.456258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:14:38.456258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:14:38.456258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:14:38.456258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:14:38.456258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:14:38.456258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:14:38.456258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:14:38.456258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:14:38.456258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:14:38.456258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:14:38.456258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:14:38.456258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:14:38.456258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:14:38.456258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:14:38.456258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:14:38.456694 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:14:38.456694 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:14:38.456694 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:14:38.456694 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:14:38.456694 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:14:38.456694 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:38.456694 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:14:38.456694 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:38.456694 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:14:38.456694 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:38.456694 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:14:38.456694 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:38.456694 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:14:38.456694 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:38.456694 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:14:38.456694 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:14:38.456694 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:38.456694 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:14:38.457245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:38.457245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:14:38.457245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:14:38.457245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:38.457245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:14:38.457245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:38.457245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:14:38.510775 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:14:38.511349 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:14:38.511470 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:14:38.511699 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:14:38.512448 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:14:38.512924 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:38.514164 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:14:38.514164 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:38.514164 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:14:38.514164 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:14:38.514164 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:38.514164 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:14:38.514164 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:38.514164 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:14:38.515305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:38.515305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:14:38.515305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:14:38.515305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:14:38.515305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:38.515305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:14:38.515305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:38.515305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:14:38.515305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:38.515305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:14:38.515305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:38.515305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:14:38.515305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:38.515305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:14:38.515305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:14:38.516185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:38.516185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:14:38.516185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:38.516185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:14:38.516185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:14:38.516185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:38.516185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:14:38.516185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:38.516185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:14:38.516701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:38.516701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:14:38.516701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:14:38.516701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:14:38.516701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:14:38.516701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:14:38.516701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:14:38.516701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:14:38.516701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:14:38.516701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:38.516701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:14:38.517306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:38.517306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:14:38.517306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:38.517306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:14:38.517306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:38.517306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:14:38.517306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:38.517306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:14:38.517306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:14:38.517306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:38.517306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:14:38.517306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:38.517306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:14:38.517306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:14:38.518005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:38.518005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:14:38.518005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:38.518005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:14:38.518005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:38.518005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:14:38.518005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:14:38.518005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:14:38.518005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:14:38.518005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:14:38.518005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:14:38.518005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:14:38.518005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:14:38.518005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:14:38.518948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:14:38.518948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:14:38.518948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:14:38.518948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:14:38.518948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:14:38.518948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:14:38.518948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:14:38.518948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:14:38.518948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:14:38.518948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:14:38.518948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:38.518948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:14:38.518948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:38.518948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:14:38.518948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:38.518948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:14:38.518948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:38.518948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:14:38.518948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:38.519803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:14:38.519803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:14:38.519803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:38.519803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:14:38.519803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:38.519803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:14:38.519803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:14:38.519803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:38.519803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:14:38.519803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:38.519803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:14:38.520423 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:14:38.520423 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:14:38.520677 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:14:38.521086 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:14:38.521844 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:14:38.522410 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:14:38.522614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:38.523498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:14:38.523498 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:38.523498 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:14:38.523498 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:14:38.523498 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:38.523498 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:14:38.523498 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:38.523498 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:14:38.524550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:38.524550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:14:38.524550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:14:38.524550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:14:38.524550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:14:38.524550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:14:38.524550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:14:38.524550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:38.524550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:14:38.524550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:38.524550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:14:38.524550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:38.524839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:14:38.524839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:38.524839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:14:38.524839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:38.524839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:14:38.524839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:14:38.524839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:38.524839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:14:38.524839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:38.524839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:14:38.524839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:14:38.524839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:38.524839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:14:38.525119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:38.525119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:14:38.525671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:38.525671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:14:38.525671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:14:38.525671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:14:38.525671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:14:38.525671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:14:38.525671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:14:38.525671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:14:38.525671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:14:38.525671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:14:38.525671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:14:38.525671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:14:38.525671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:14:38.525671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:14:38.525671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:14:38.525671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:14:38.526009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:14:38.526009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:14:38.526009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:38.526009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:14:38.526009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:38.526009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:14:38.526009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:38.526009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:14:38.526009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:38.526009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:14:38.526009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:38.526009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:14:38.526009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:14:38.526009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:38.526009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:14:38.526009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:38.526009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:14:38.526009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:14:38.526009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:38.526376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:14:38.526376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:38.526376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:14:42.932159 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:14:42.932943 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:14:42.933205 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:14:42.934509 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:14:42.973017 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:14:42.973553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:42.974523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:14:42.974523 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:42.974523 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:14:42.974523 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:14:42.974523 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:42.974523 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:14:42.974523 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:42.974523 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:14:42.975341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:42.975341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:14:42.975341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:14:42.975341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:14:42.975341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:14:42.975341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:14:42.975341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:14:42.975341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:42.975341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:14:42.975341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:42.975341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:14:42.975341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:42.976036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:14:42.976036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:42.976036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:14:42.976036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:42.976036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:14:42.976036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:14:42.976036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:42.976036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:14:42.976036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:42.976036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:14:42.976036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:14:42.976036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:42.976036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:14:42.976801 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:42.976801 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:14:42.976801 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:42.976801 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:14:42.976801 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:14:42.976801 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:14:42.976801 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:14:42.976801 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:14:42.976801 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:14:42.976801 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:14:42.976801 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:14:42.976801 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:14:42.976801 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:14:42.976801 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:14:42.976801 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:14:42.977540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:14:42.977540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:14:42.977540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:14:42.977540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:14:42.977540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:14:42.977540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:42.977540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:14:42.977540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:42.977540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:14:42.977540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:42.977540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:14:42.977540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:42.977540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:14:42.977540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:42.977540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:14:42.977540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:14:42.977540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:42.977540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:14:42.977540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:42.978447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:14:42.978447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:14:42.978447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:42.978447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:14:42.978447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:42.978447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:14:43.442769 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:14:43.443308 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:14:43.443446 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:14:43.444385 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:14:43.444611 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:14:43.444785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:43.445464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:14:43.445464 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:43.445464 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:14:43.445464 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:14:43.445464 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:43.445464 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:14:43.445464 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:43.445464 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:14:43.446608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:43.446608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:14:43.446608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:14:43.446608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:14:43.446608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:14:43.446608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:14:43.446608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:14:43.446608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:43.446608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:14:43.446608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:43.446608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:14:43.446608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:43.447407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:14:43.447407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:43.447407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:14:43.447407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:43.447407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:14:43.447407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:14:43.447407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:43.447407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:14:43.447407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:43.447407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:14:43.447407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:14:43.447407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:43.447407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:14:43.448194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:43.448194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:14:43.448194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:43.448194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:14:43.448194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:14:43.448194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:14:43.448194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:14:43.448194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:14:43.448194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:14:43.448194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:14:43.448194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:14:43.448194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:14:43.448194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:14:43.448194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:14:43.448194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:14:43.449062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:14:43.449062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:14:43.449062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:14:43.449062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:14:43.449062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:14:43.449062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:43.449062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:14:43.449062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:43.449062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:14:43.449062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:43.449062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:14:43.449062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:43.449062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:14:43.449062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:43.449062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:14:43.449062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:14:43.449062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:43.449062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:14:43.449062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:43.450108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:14:43.450108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:14:43.450108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:43.450108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:14:43.450108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:43.450108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:14:43.451421 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:14:43.451846 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:14:43.452099 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:14:43.452309 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:14:43.452728 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:14:43.452946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:43.453471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:14:43.453471 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:43.453471 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:14:43.453471 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:14:43.453471 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:43.453471 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:14:43.453471 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:43.453471 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:14:43.454290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:43.454290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:14:43.454290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:14:43.454290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:14:43.454290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:43.454290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:14:43.454290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:43.454290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:14:43.454290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:43.454290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:14:43.454290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:43.454290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:14:43.454290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:43.454290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:14:43.454290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:14:43.454948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:43.454948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:14:43.454948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:43.454948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:14:43.454948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:14:43.454948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:43.454948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:14:43.454948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:43.454948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:14:43.455218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:43.455218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:14:43.455218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:14:43.455218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:14:43.455218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:14:43.455218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:14:43.455218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:14:43.455218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:14:43.455218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:14:43.455218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:14:43.455218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:43.455218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:14:43.455549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:43.455549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:14:43.455549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:43.455549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:14:43.455549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:43.455549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:14:43.455549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:43.455549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:14:43.455549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:14:43.455549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:43.455549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:14:43.455549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:43.455549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:14:43.455549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:14:43.456064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:43.456064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:14:43.456064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:43.456064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:14:43.456418 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:43.456418 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:14:43.456418 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:14:43.456418 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:14:43.456418 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:14:43.456418 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:14:43.456418 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:14:43.456418 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:14:43.456418 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:14:43.456418 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:14:43.456418 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:14:43.456418 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:14:43.456418 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:14:43.456418 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:14:43.456418 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:14:43.456418 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:14:43.457022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:14:43.457022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:14:43.457022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:14:43.457022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:14:43.457022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:14:43.457022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:43.457022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:14:43.457022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:43.457022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:14:43.457022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:43.457022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:14:43.457022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:43.457022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:14:43.457022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:43.457022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:14:43.457022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:14:43.457022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:43.457022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:14:43.457528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:43.457528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:14:43.457528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:14:43.457528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:43.457528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:14:43.457528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:43.457528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:14:43.511316 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:14:43.511775 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:14:43.511981 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:14:43.512142 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:14:43.512884 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:14:43.513313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:43.514280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:14:43.514280 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:43.514280 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:14:43.514280 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:14:43.514280 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:43.514280 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:14:43.514280 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:43.514280 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:14:43.515677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:43.515677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:14:43.515677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:14:43.515677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:14:43.515677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:43.515677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:14:43.515677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:43.515677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:14:43.515677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:43.515677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:14:43.515677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:43.515677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:14:43.515677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:43.515677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:14:43.515677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:14:43.516563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:43.516563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:14:43.516563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:43.516563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:14:43.516563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:14:43.516563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:43.516563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:14:43.516563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:43.516563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:14:43.517681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:43.517681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:14:43.517681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:14:43.517681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:14:43.517681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:14:43.517681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:14:43.517681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:14:43.517681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:14:43.517681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:14:43.517681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:43.517681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:14:43.518266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:43.518266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:14:43.518266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:43.518266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:14:43.518266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:43.518266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:14:43.518266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:43.518266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:14:43.518266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:14:43.518266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:43.518266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:14:43.518266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:43.518266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:14:43.518266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:14:43.519228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:43.519228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:14:43.519228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:43.519228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:14:43.519228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:43.519228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:14:43.519228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:14:43.519228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:14:43.519228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:14:43.519228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:14:43.519228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:14:43.519228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:14:43.519228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:14:43.519228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:14:43.519972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:14:43.519972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:14:43.519972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:14:43.519972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:14:43.519972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:14:43.519972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:14:43.519972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:14:43.519972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:14:43.519972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:14:43.519972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:14:43.519972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:43.519972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:14:43.519972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:43.519972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:14:43.519972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:43.519972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:14:43.519972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:43.519972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:14:43.519972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:43.520996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:14:43.520996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:14:43.520996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:43.520996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:14:43.520996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:43.520996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:14:43.520996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:14:43.520996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:43.520996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:14:43.520996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:43.520996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:14:43.520996 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:14:43.520996 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:14:43.521384 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:14:43.521384 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:14:43.521558 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:14:43.521927 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:14:43.522163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:43.522903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:14:43.522903 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:43.522903 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:14:43.522903 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:14:43.522903 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:43.522903 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:14:43.522903 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:43.522903 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:14:43.524036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:43.524036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:14:43.524036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:14:43.524036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:14:43.524036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:14:43.524036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:14:43.524036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:14:43.524036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:43.524036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:14:43.524036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:43.524036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:14:43.524036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:43.524523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:14:43.524523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:43.524523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:14:43.524523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:43.524523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:14:43.524523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:14:43.524523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:43.524523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:14:43.524523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:43.524523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:14:43.524523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:14:43.524523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:43.524523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:14:43.524843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:43.524843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:14:43.525200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:43.525200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:14:43.525200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:14:43.525200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:14:43.525200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:14:43.525200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:14:43.525200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:14:43.525200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:14:43.525200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:14:43.525200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:14:43.525200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:14:43.525200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:14:43.525200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:14:43.525200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:14:43.525200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:14:43.525200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:14:43.525602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:14:43.525602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:14:43.525602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:43.525602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:14:43.525602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:43.525602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:14:43.525602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:43.525602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:14:43.525602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:43.525602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:14:43.525602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:43.525602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:14:43.525602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:14:43.525602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:43.525602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:14:43.525602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:43.525602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:14:43.525602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:14:43.525602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:43.526372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:14:43.526372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:43.526372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:14:47.933033 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:14:47.934057 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:14:47.934057 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:14:47.934691 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:14:47.936523 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG dbcounter [-] [106314] Writing DB stats neutron:SELECT=260 {{(pid=106314) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 09 09:14:47.943654 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_for_inconsistencies' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:14:47.977807 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:14:47.977807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:47.978099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:14:47.978099 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:47.978099 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:14:47.978099 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:14:47.978099 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:47.978099 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:14:47.978099 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:47.978099 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:14:47.980998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.044 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:47.980998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:14:47.980998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:14:47.980998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:14:47.980998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:14:47.980998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:14:47.980998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:14:47.980998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:47.980998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:14:47.980998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:47.980998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:14:47.980998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:47.981727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:14:47.981727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:47.981727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:14:47.981727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:47.981727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:14:47.981727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:14:47.981727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:47.981727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:14:47.981727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:47.981727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:14:47.981727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:14:47.981727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:47.981727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:14:47.982406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:47.982406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:14:47.982406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:47.982406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:14:47.982406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:14:47.982406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:14:47.982406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:14:47.982406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:14:47.982406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:14:47.982406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:14:47.982406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:14:47.982406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:14:47.982406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:14:47.982406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:14:47.982406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:14:47.983030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:14:47.983030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:14:47.983030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:14:47.983030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:14:47.983030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:14:47.983030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:47.983030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:14:47.983030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:47.983030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:14:47.983030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:47.983030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:14:47.983030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:47.983030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:14:47.983030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:47.983030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:14:47.983030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:14:47.983030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:47.983030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:14:47.983030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:47.983634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:14:47.983634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:14:47.983634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:47.983634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:14:47.983634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:47.983634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:14:47.983634 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_for_inconsistencies {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:14:47.983634 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Checking Neutron and OVN revision consistency. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:14:47.988137 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Maintenance task: Synchronizing Neutron and OVN databases started {{(pid=106314) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:440}}
May 09 09:14:47.988311 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Maintenance task: Number of inconsistencies found at create/update: networks=4, security_groups=2, routers=1, router_ports=1 {{(pid=106314) _log /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:419}}
May 09 09:14:47.988396 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Maintenance task: Fixing resource f8a38fcb-e8a2-421a-a340-bf7f2f5cc72b (type: networks) at create/update {{(pid=106314) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 09 09:14:48.024361 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-f8a38fcb-e8a2-421a-a340-bf7f2f5cc72b, resource={'id': 'f8a38fcb-e8a2-421a-a340-bf7f2f5cc72b', 'name': 'private', 'tenant_id': 'f4d54dfe001d43fbb344093a1ccabd6a', 'project_id': 'f4d54dfe001d43fbb344093a1ccabd6a', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['96e72a03-97b8-412a-b2b9-1bf048ce1066', 'bbd6957a-953f-4fc1-8761-5d93128564d9'], 'standard_attr_id': 12, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-05-09T09:04:34Z', 'updated_at': '2026-05-09T09:04:37Z', 'revision_number': 3, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 3730}, resource_type=networks, if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:14:48.024876 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-f8a38fcb-e8a2-421a-a340-bf7f2f5cc72b, col_values=(('external_ids', {'neutron:network_name': 'private', 'neutron:mtu': '1372', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:14:48.025456 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-78a32c7c-7637-4b83-b0ab-62aee8bb3038, external_ids_update=None, columns={'tag': 3730}, if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:14:48.025676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:48.026830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:14:48.026830 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:48.026830 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:14:48.026830 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:14:48.026830 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:48.026830 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:14:48.026830 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:48.026830 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:14:48.028290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Maintenance task: Failed to fix resource f8a38fcb-e8a2-421a-a340-bf7f2f5cc72b (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:48.028290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:14:48.028290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 09 09:14:48.028290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 09 09:14:48.028290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 09 09:14:48.028290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 09 09:14:48.028290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2332, in update_network
May 09 09:14:48.028290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:14:48.028290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:48.028290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:14:48.028290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:48.028682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:14:48.028682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:48.028682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:14:48.028682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:48.028682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:14:48.028682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:48.028682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:14:48.028682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:14:48.028682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:48.028682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:14:48.028682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:48.028682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:14:48.028682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:14:48.028682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:48.029060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:14:48.029060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:48.029060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:14:48.029060 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Maintenance task: Fixing resource d18a9fac-2ebf-4d1e-a86c-0f33fba8eeda (type: networks) at create/update {{(pid=106314) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 09 09:14:48.059282 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-d18a9fac-2ebf-4d1e-a86c-0f33fba8eeda, resource={'id': 'd18a9fac-2ebf-4d1e-a86c-0f33fba8eeda', 'name': 'public', 'tenant_id': 'f4d54dfe001d43fbb344093a1ccabd6a', 'project_id': 'f4d54dfe001d43fbb344093a1ccabd6a', 'admin_state_up': True, 'mtu': 1430, 'status': 'ACTIVE', 'subnets': ['088bbd0f-7e81-4a76-8399-24d15c984009', 'bbd6a8cc-80e6-4336-a70b-1c962c561590'], 'standard_attr_id': 18, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': True, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'is_default': True, 'tags': [], 'created_at': '2026-05-09T09:04:40Z', 'updated_at': '2026-05-09T09:04:50Z', 'revision_number': 3, 'provider:network_type': 'flat', 'provider:physical_network': 'public', 'provider:segmentation_id': None}, resource_type=networks, if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:14:48.059723 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-d18a9fac-2ebf-4d1e-a86c-0f33fba8eeda, col_values=(('external_ids', {'neutron:network_name': 'public', 'neutron:mtu': '1430', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'flat', 'neutron:provnet-physical-network': 'public'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false', 'fdb_age_threshold': '300', 'broadcast-arps-to-all-routers': 'true'})), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:14:48.060371 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-1163f525-70cd-4b84-ae9b-7d56378305e9, external_ids_update=None, columns={'tag': []}, if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:14:48.060760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:48.061914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:14:48.061914 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:48.061914 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:14:48.061914 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:14:48.061914 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:48.061914 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:14:48.061914 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:48.061914 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:14:48.063114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Maintenance task: Failed to fix resource d18a9fac-2ebf-4d1e-a86c-0f33fba8eeda (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:48.063114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:14:48.063114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 09 09:14:48.063114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 09 09:14:48.063114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 09 09:14:48.063114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 09 09:14:48.063114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2332, in update_network
May 09 09:14:48.063114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:14:48.063114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:48.063114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:14:48.063114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:48.063802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:14:48.063802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:48.063802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:14:48.063802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:48.063802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:14:48.063802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:48.063802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:14:48.063802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:14:48.063802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:48.063802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:14:48.063802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:48.063802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:14:48.063802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:14:48.063802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:48.064467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:14:48.064467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:48.064467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:14:48.064467 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Maintenance task: Fixing resource 417da935-2882-452f-8ba7-9076d68fcd2b (type: networks) at create/update {{(pid=106314) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 09 09:14:48.097058 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-417da935-2882-452f-8ba7-9076d68fcd2b, resource={'id': '417da935-2882-452f-8ba7-9076d68fcd2b', 'name': 'admin_net', 'tenant_id': '38e536cfc9c94b6cbf529caf55cc7739', 'project_id': '38e536cfc9c94b6cbf529caf55cc7739', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['d3c53d14-f490-4615-a986-3715a315f827'], 'standard_attr_id': 42, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-05-09T09:10:38Z', 'updated_at': '2026-05-09T09:10:39Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 32823}, resource_type=networks, if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:14:48.097615 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-417da935-2882-452f-8ba7-9076d68fcd2b, col_values=(('external_ids', {'neutron:network_name': 'admin_net', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:14:48.098518 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-5738ab2a-b471-447b-9292-7f8c6bc97168, external_ids_update=None, columns={'tag': 32823}, if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:14:48.098812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:48.099735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:14:48.099735 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:48.099735 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:14:48.099735 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:14:48.099735 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:48.099735 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:14:48.099735 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:48.099735 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:14:48.101590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Maintenance task: Failed to fix resource 417da935-2882-452f-8ba7-9076d68fcd2b (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:48.101590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:14:48.101590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 09 09:14:48.101590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 09 09:14:48.101590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 09 09:14:48.101590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 09 09:14:48.101590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2332, in update_network
May 09 09:14:48.101590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:14:48.101590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:48.101590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:14:48.101590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:48.102284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:14:48.102284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:48.102284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:14:48.102284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:48.102284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:14:48.102284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:48.102284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:14:48.102284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:14:48.102284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:48.102284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:14:48.102284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:48.102284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:14:48.102284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:14:48.102284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:48.103020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:14:48.103020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:48.103020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:14:48.103020 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Maintenance task: Fixing resource 433617e8-cdf8-414a-8cc6-63a7b5d0c678 (type: networks) at create/update {{(pid=106314) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 09 09:14:48.133087 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-433617e8-cdf8-414a-8cc6-63a7b5d0c678, resource={'id': '433617e8-cdf8-414a-8cc6-63a7b5d0c678', 'name': 'k8s-clusterapi-cluster-magnum-system-kube-whtb8', 'tenant_id': '3e013d45a44544c3b50eb0e83e4b1efc', 'project_id': '3e013d45a44544c3b50eb0e83e4b1efc', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['9187f065-9264-427a-8df8-672a6981191d'], 'standard_attr_id': 52, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-05-09T09:11:20Z', 'updated_at': '2026-05-09T09:11:20Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 32644}, resource_type=networks, if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:14:48.133615 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-433617e8-cdf8-414a-8cc6-63a7b5d0c678, col_values=(('external_ids', {'neutron:network_name': 'k8s-clusterapi-cluster-magnum-system-kube-whtb8', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:14:48.134460 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-8c3fe586-c182-47ba-9845-d7608113cb71, external_ids_update=None, columns={'tag': 32644}, if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:14:48.134719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:48.135887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:14:48.135887 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:48.135887 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:14:48.135887 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:14:48.135887 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:48.135887 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:14:48.135887 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:48.135887 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:14:48.137434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Maintenance task: Failed to fix resource 433617e8-cdf8-414a-8cc6-63a7b5d0c678 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:48.137434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:14:48.137434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 09 09:14:48.137434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 09 09:14:48.137434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 09 09:14:48.137434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 09 09:14:48.137434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2332, in update_network
May 09 09:14:48.137434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:14:48.137434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:48.137434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:14:48.137434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:48.137911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:14:48.137911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:48.137911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:14:48.137911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:48.137911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:14:48.137911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:48.137911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:14:48.137911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:14:48.137911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:48.137911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:14:48.137911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:48.137911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:14:48.137911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:14:48.137911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:48.138347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:14:48.138347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:48.138347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:14:48.138347 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Maintenance task: Fixing resource 8c957489-7e59-4ce3-80c9-8e97a8c156c6 (type: security_groups) at create/update {{(pid=106314) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 09 09:14:48.174922 np0000173033 neutron-ovn-maintenance-worker[106314]: INFO neutron.db.ovn_revision_numbers_db [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Successfully bumped revision number for resource 8c957489-7e59-4ce3-80c9-8e97a8c156c6 (type: security_groups) to 10
May 09 09:14:48.175162 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Maintenance task: Fixing resource 75c4f405-d30f-4ca8-b272-e95630a848fc (type: security_groups) at create/update {{(pid=106314) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 09 09:14:48.225951 np0000173033 neutron-ovn-maintenance-worker[106314]: INFO neutron.db.ovn_revision_numbers_db [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Successfully bumped revision number for resource 75c4f405-d30f-4ca8-b272-e95630a848fc (type: security_groups) to 15
May 09 09:14:48.225951 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Maintenance task: Fixing resource cab10384-14c5-492c-8bd0-061f5838310c (type: routers) at create/update {{(pid=106314) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 09 09:14:48.337539 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-cab10384-14c5-492c-8bd0-061f5838310c, resource={'id': 'cab10384-14c5-492c-8bd0-061f5838310c', 'name': 'k8s-clusterapi-cluster-magnum-system-kube-whtb8', 'project_id': '3e013d45a44544c3b50eb0e83e4b1efc', 'admin_state_up': True, 'status': 'ACTIVE', 'enable_snat': True, 'external_gateway_info': {'network_id': 'd18a9fac-2ebf-4d1e-a86c-0f33fba8eeda', 'external_fixed_ips': [{'subnet_id': 'bbd6a8cc-80e6-4336-a70b-1c962c561590', 'ip_address': '172.24.5.116'}, {'subnet_id': '088bbd0f-7e81-4a76-8399-24d15c984009', 'ip_address': '2001:db8::1b7'}], 'enable_snat': True}, 'gw_port_id': 'af118f11-dc01-429a-b20a-e97b77b8d5ec', 'description': 'Created by cluster-api-provider-openstack cluster magnum-system-kube-whtb8', 'availability_zones': [], 'distributed': False, 'ha': True, 'ha_vr_id': 0, 'availability_zone_hints': [], 'enable_default_route_ecmp': False, 'enable_default_route_bfd': False, 'external_gateways': [{'network_id': 'd18a9fac-2ebf-4d1e-a86c-0f33fba8eeda', 'external_fixed_ips': [{'ip_address': '172.24.5.116', 'subnet_id': 'bbd6a8cc-80e6-4336-a70b-1c962c561590'}, {'ip_address': '2001:db8::1b7', 'subnet_id': '088bbd0f-7e81-4a76-8399-24d15c984009'}]}], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2026-05-09T09:11:21Z', 'updated_at': '2026-05-09T09:11:24Z', 'revision_number': 3, 'tenant_id': '3e013d45a44544c3b50eb0e83e4b1efc'}, resource_type=routers, if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:14:48.337938 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterCommand(_result=None, name=neutron-cab10384-14c5-492c-8bd0-061f5838310c, columns={'external_ids': {'neutron:router_name': 'k8s-clusterapi-cluster-magnum-system-kube-whtb8', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': ''}, 'enabled': True}, if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:14:48.338085 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): QoSDelCommand(_result=None, switch=neutron-d18a9fac-2ebf-4d1e-a86c-0f33fba8eeda, conditions=[('direction', '=', 'from-lport'), ('priority', '=', 2002), ('match', '=', 'inport == "af118f11-dc01-429a-b20a-e97b77b8d5ec"')], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:14:48.338085 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): QoSDelCommand(_result=None, switch=neutron-d18a9fac-2ebf-4d1e-a86c-0f33fba8eeda, conditions=[('direction', '=', 'to-lport'), ('priority', '=', 2002), ('match', '=', 'outport == "af118f11-dc01-429a-b20a-e97b77b8d5ec"')], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:14:48.338247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:48.338896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:14:48.338896 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:48.338896 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:14:48.338896 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:14:48.338896 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:48.338896 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:14:48.338896 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:48.338896 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:14:48.339309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_client [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Unable to update router cab10384-14c5-492c-8bd0-061f5838310c. Error: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:48.340462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Maintenance task: Failed to fix resource cab10384-14c5-492c-8bd0-061f5838310c (type: routers): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:48.340462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:14:48.340462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 09 09:14:48.340462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 09 09:14:48.340462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 09 09:14:48.340462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 09 09:14:48.340462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1635, in update_router
May 09 09:14:48.340462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with excutils.save_and_reraise_exception():
May 09 09:14:48.340462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 271, in __exit__
May 09 09:14:48.340462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.force_reraise()
May 09 09:14:48.340462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 233, in force_reraise
May 09 09:14:48.341078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise self.value
May 09 09:14:48.341078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1569, in update_router
May 09 09:14:48.341078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:14:48.341078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:48.341078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:14:48.341078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:48.341078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:14:48.341078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:48.341078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:14:48.341078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:48.341078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:14:48.341078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:48.341078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:14:48.341078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:14:48.341791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:48.341791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:14:48.341791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:48.341791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:14:48.341791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:14:48.341791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:48.341791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:14:48.341791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:48.341791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:14:48.341791 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Maintenance task: Fixing resource af118f11-dc01-429a-b20a-e97b77b8d5ec (type: router_ports) at create/update {{(pid=106314) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 09 09:14:48.443602 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:14:48.452012 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:14:48.480109 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=lrp-af118f11-dc01-429a-b20a-e97b77b8d5ec, resource={'id': 'af118f11-dc01-429a-b20a-e97b77b8d5ec', 'name': '', 'network_id': 'd18a9fac-2ebf-4d1e-a86c-0f33fba8eeda', 'project_id': '', 'mac_address': 'fa:16:3e:51:04:94', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'cab10384-14c5-492c-8bd0-061f5838310c', 'device_owner': 'network:router_gateway', 'standard_attr_id': 57, 'fixed_ips': [{'subnet_id': 'bbd6a8cc-80e6-4336-a70b-1c962c561590', 'ip_address': '172.24.5.116'}, {'subnet_id': '088bbd0f-7e81-4a76-8399-24d15c984009', 'ip_address': '2001:db8::1b7'}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'np0000173033', 'binding:vif_type': 'ovs', 'binding:vif_details': {'port_filter': True, 'connectivity': 'l2', 'ovs_create_tap': False, 'bridge_name': 'br-int', 'datapath_type': 'system', 'bound_drivers': {'0': 'ovn'}}, 'port_security_enabled': False, 'tags': [], 'created_at': '2026-05-09T09:11:21Z', 'updated_at': '2026-05-09T09:11:24Z', 'revision_number': 5, 'tenant_id': ''}, resource_type=router_ports, if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:14:48.480457 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterPortCommand(_result=None, name=lrp-af118f11-dc01-429a-b20a-e97b77b8d5ec, columns={'external_ids': {'neutron:revision_number': '5', 'neutron:subnet_ids': 'bbd6a8cc-80e6-4336-a70b-1c962c561590 088bbd0f-7e81-4a76-8399-24d15c984009', 'neutron:network_name': 'neutron-d18a9fac-2ebf-4d1e-a86c-0f33fba8eeda', 'neutron:is_ext_gw': 'True', 'neutron:router_name': 'neutron-cab10384-14c5-492c-8bd0-061f5838310c'}, 'options': {'reside-on-redirect-chassis': 'true', 'gateway_mtu': '1372'}, 'networks': ['172.24.5.116/24', '2001:db8::1b7/64'], 'ipv6_ra_configs': {}}, if_exists=False) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:14:48.481117 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLRouterPortInLSwitchPortCommand(_result=None, lswitch_port=af118f11-dc01-429a-b20a-e97b77b8d5ec, lrouter_port=lrp-af118f11-dc01-429a-b20a-e97b77b8d5ec, is_gw_port=True, if_exists=True, lsp_address=router) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:14:48.482064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:48.483011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:14:48.483011 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:48.483011 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:14:48.483011 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:14:48.483011 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:48.483011 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:14:48.483011 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:48.483011 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:14:48.483911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Maintenance task: Failed to fix resource af118f11-dc01-429a-b20a-e97b77b8d5ec (type: router_ports): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:48.483911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:14:48.483911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 09 09:14:48.483911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 09 09:14:48.483911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 09 09:14:48.483911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 09 09:14:48.483911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1976, in update_router_port
May 09 09:14:48.483911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:14:48.483911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:48.483911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:14:48.483911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:48.484475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:14:48.484475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:48.484475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:14:48.484475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:48.484475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:14:48.484475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:48.484475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:14:48.484475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:14:48.484475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:48.484475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:14:48.484475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:48.484475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:14:48.484475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:14:48.484475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:48.485094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:14:48.485094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:48.485094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:14:48.485094 np0000173033 neutron-ovn-maintenance-worker[106314]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_for_inconsistencies finished in 0.503 seconds
May 09 09:14:48.485094 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:14:48.485094 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:14:48.485094 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:14:48.485500 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:14:48.485500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:48.485812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:14:48.485812 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:48.485812 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:14:48.485812 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:14:48.485812 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:48.485812 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:14:48.485812 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:48.485812 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:14:48.486442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:48.486442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:14:48.486442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:14:48.486442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:14:48.486442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:14:48.486442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:14:48.486442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:14:48.486442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:48.486442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:14:48.486442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:48.486442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:14:48.486442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:48.487141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:14:48.487141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:48.487141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:14:48.487141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:48.487141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:14:48.487141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:14:48.487141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:48.487141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:14:48.487141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:48.487141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:14:48.487141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:14:48.487141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:48.487141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:14:48.487797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:48.487797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:14:48.487797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:48.487797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:14:48.487797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:14:48.487797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:14:48.487797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:14:48.487797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:14:48.487797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:14:48.487797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:14:48.487797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:14:48.487797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:14:48.487797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:14:48.487797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:14:48.487797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:14:48.488561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:14:48.488561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:14:48.488561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:14:48.488561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:14:48.488561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:14:48.488561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:48.488561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:14:48.488561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:48.488561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:14:48.488561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:48.488561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:14:48.488561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:48.488561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:14:48.488561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:48.488561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:14:48.488561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:14:48.488561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:48.488561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:14:48.488561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:48.489639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:14:48.489639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:14:48.489639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:48.489639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:14:48.489639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:48.489639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:14:48.489639 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:14:48.489639 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:14:48.489639 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:14:48.489639 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:14:48.490094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:48.490094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:14:48.490094 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:48.490094 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:14:48.490094 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:14:48.490094 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:48.490094 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:14:48.490094 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:48.490094 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:14:48.490094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:48.490094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:14:48.490094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:14:48.490094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:14:48.490094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:48.490094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:14:48.490094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:48.490094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:14:48.490613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:48.490613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:14:48.490613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:48.490613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:14:48.490613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:48.490613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:14:48.490613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:14:48.490613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:48.490613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:14:48.490613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:48.490613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:14:48.490613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:14:48.490613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:48.490613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:14:48.490613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:48.490613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:14:48.491187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:48.491187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:14:48.491187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:14:48.491187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:14:48.491187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:14:48.491187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:14:48.491187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:14:48.491187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:14:48.491187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:14:48.491187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:14:48.491187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:48.491187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:14:48.491632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:48.491632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:14:48.491632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:48.491632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:14:48.491632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:48.491632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:14:48.491632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:48.491632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:14:48.491632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:14:48.491632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:48.491632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:14:48.491632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:48.491632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:14:48.491632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:14:48.492207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:48.492207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:14:48.492207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:48.492207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:14:48.492207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:48.492207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:14:48.492207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:14:48.492207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:14:48.492207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:14:48.492207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:14:48.492207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:14:48.492207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:14:48.492207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:14:48.492207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:14:48.492842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:14:48.492842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:14:48.492842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:14:48.492842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:14:48.492842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:14:48.492842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:14:48.492842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:14:48.492842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:14:48.492842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:14:48.492842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:14:48.492842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:14:48.492842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:48.492842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:14:48.492842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:48.492842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:14:48.492842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:48.492842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:14:48.492842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:48.492842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:14:48.493642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:48.493642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:14:48.493642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:14:48.493642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:48.493642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:14:48.493642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:48.493642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:14:48.493642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:14:48.493642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:48.493642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:14:48.493642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:48.493642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:14:48.511762 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:14:48.512230 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:14:48.512392 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:14:48.512578 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:14:48.513182 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:14:48.513636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:48.514551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:14:48.514551 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:48.514551 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:14:48.514551 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:14:48.514551 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:48.514551 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:14:48.514551 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:48.514551 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:14:48.515603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:48.515603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:14:48.515603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:14:48.515603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:14:48.515603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:48.515603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:14:48.515603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:48.515603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:14:48.515603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:48.515603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:14:48.515603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:48.515603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:14:48.515603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:48.515603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:14:48.515603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:14:48.516429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:48.516429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:14:48.516429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:48.516429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:14:48.516429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:14:48.516429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:48.516429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:14:48.516429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:48.516429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:14:48.516823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:48.516823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:14:48.516823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:14:48.516823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:14:48.516823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:14:48.516823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:14:48.516823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:14:48.516823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:14:48.516823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:14:48.516823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:48.516823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:14:48.517430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:48.517430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:14:48.517430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:48.517430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:14:48.517430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:48.517430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:14:48.517430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:48.517430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:14:48.517430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:14:48.517430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:48.517430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:14:48.517430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:48.517430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:14:48.517430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:14:48.518033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:48.518033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:14:48.518033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:48.518033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:14:48.518033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:48.518033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:14:48.518033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:14:48.518033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:14:48.518033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:14:48.518033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:14:48.518033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:14:48.518033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:14:48.518033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:14:48.518033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:14:48.518655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:14:48.518655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:14:48.518655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:14:48.518655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:14:48.518655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:14:48.518655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:14:48.518655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:14:48.518655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:14:48.518655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:14:48.518655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:14:48.518655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:48.518655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:14:48.518655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:48.518655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:14:48.518655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:48.518655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:14:48.518655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:48.518655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:14:48.518655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:48.519488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:14:48.519488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:14:48.519488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:48.519488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:14:48.519488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:48.519488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:14:48.519488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:14:48.519488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:48.519488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:14:48.519488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:48.519488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:14:48.520800 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:14:48.521103 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:14:48.521237 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:14:48.521424 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:14:48.521960 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:14:48.522387 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:14:48.522713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:48.523664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:14:48.523664 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:48.523664 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:14:48.523664 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:14:48.523664 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:48.523664 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:14:48.523664 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:48.523664 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:14:48.524523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:48.524523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:14:48.524523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:14:48.524523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:14:48.524523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:14:48.524523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:14:48.524523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:14:48.524523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:48.524523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:14:48.524523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:48.524523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:14:48.524523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:48.524924 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:14:48.524924 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:48.524924 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:14:48.524924 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:48.524924 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:14:48.524924 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:14:48.524924 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:48.524924 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:14:48.524924 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:48.524924 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:14:48.524924 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:14:48.524924 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:48.524924 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:14:48.525359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:48.525359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:14:48.525571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:48.525571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:14:48.525571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:14:48.525571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:14:48.525571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:14:48.525571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:14:48.525571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:14:48.525571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:14:48.525571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:14:48.525571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:14:48.525571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:14:48.525571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:14:48.525571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:14:48.525571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:14:48.525571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:14:48.525571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:14:48.526070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:14:48.526070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:14:48.526070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:48.526070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:14:48.526070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:48.526070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:14:48.526070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:48.526070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:14:48.526070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:48.526070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:14:48.526070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:48.526070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:14:48.526070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:14:48.526070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:48.526070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:14:48.526070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:48.526070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:14:48.526070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:14:48.526070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:48.526638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:14:48.526638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:48.526638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:14:52.933889 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:14:52.934826 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:14:52.934928 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:14:52.936017 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:14:52.969802 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:14:52.970309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:52.971453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:14:52.971453 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:52.971453 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:14:52.971453 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:14:52.971453 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:52.971453 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:14:52.971453 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:52.971453 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:14:52.972587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:52.972587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:14:52.972587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:14:52.972587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:14:52.972587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:14:52.972587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:14:52.972587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:14:52.972587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:52.972587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:14:52.972587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:52.972587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:14:52.972587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:52.973226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:14:52.973226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:52.973226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:14:52.973226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:52.973226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:14:52.973226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:14:52.973226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:52.973226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:14:52.973226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:52.973226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:14:52.973226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:14:52.973226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:52.973226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:14:52.973811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:52.973811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:14:52.973811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:52.973811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:14:52.973811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:14:52.973811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:14:52.973811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:14:52.973811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:14:52.973811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:14:52.973811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:14:52.973811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:14:52.973811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:14:52.973811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:14:52.973811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:14:52.973811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:14:52.974460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:14:52.974460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:14:52.974460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:14:52.974460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:14:52.974460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:14:52.974460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:52.974460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:14:52.974460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:52.974460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:14:52.974460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:52.974460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:14:52.974460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:52.974460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:14:52.974460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:52.974460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:14:52.974460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:14:52.974460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:52.974460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:14:52.974460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:52.975305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:14:52.975305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:14:52.975305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:52.975305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:14:52.975305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:52.975305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:14:53.484773 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:14:53.485275 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:14:53.485400 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:14:53.486292 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:14:53.486510 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:14:53.486712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:53.487470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:14:53.487470 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:53.487470 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:14:53.487470 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:14:53.487470 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:53.487470 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:14:53.487470 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:53.487470 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:14:53.488603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:53.488603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:14:53.488603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:14:53.488603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:14:53.488603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:14:53.488603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:14:53.488603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:14:53.488603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:53.488603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:14:53.488603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:53.488603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:14:53.488603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:53.489141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:14:53.489141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:53.489141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:14:53.489141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:53.489141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:14:53.489141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:14:53.489141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:53.489141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:14:53.489141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:53.489141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:14:53.489141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:14:53.489141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:53.489141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:14:53.489481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:53.489481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:14:53.489481 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:14:53.489973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:53.489973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:14:53.489973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:14:53.489973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:14:53.489973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:14:53.489973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:14:53.489973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:14:53.489973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:14:53.489973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:14:53.489973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:14:53.489973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:14:53.489973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:14:53.489973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:14:53.489973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:14:53.489973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:14:53.489973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:14:53.490430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:14:53.490430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:14:53.490430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:53.490430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:14:53.490430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:53.490430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:14:53.490430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:53.490430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:14:53.490430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:53.490430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:14:53.490430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:53.490430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:14:53.490430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:14:53.490430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:53.490430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:14:53.490430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:53.490430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:14:53.490430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:14:53.490430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:53.490950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:14:53.490950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:53.490950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:14:53.490950 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:14:53.490950 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:14:53.490950 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:14:53.490950 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:14:53.491231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:53.491791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:14:53.491791 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:53.491791 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:14:53.491791 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:14:53.491791 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:53.491791 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:14:53.491791 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:53.491791 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:14:53.492507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:53.492507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:14:53.492507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:14:53.492507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:14:53.492507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:53.492507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:14:53.492507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:53.492507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:14:53.492507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:53.492507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:14:53.492507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:53.492507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:14:53.492507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:53.492507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:14:53.492507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:14:53.493002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:53.493002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:14:53.493002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:53.493002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:14:53.493002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:14:53.493002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:53.493002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:14:53.493002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:53.493002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:14:53.493356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:53.493356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:14:53.493356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:14:53.493356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:14:53.493356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:14:53.493356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:14:53.493356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:14:53.493356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:14:53.493356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:14:53.493356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:14:53.493356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:53.493356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:14:53.493678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:53.493678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:14:53.493678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:53.493678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:14:53.493678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:53.493678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:14:53.493678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:53.493678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:14:53.493678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:14:53.493678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:53.493678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:14:53.493678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:53.493678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:14:53.493678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:14:53.494046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:53.494046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:14:53.494046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:53.494046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:14:53.494341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:53.494341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:14:53.494341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:14:53.494341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:14:53.494341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:14:53.494341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:14:53.494341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:14:53.494341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:14:53.494341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:14:53.494341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:14:53.494341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:14:53.494341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:14:53.494341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:14:53.494341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:14:53.494341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:14:53.494341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:14:53.494815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:14:53.494815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:14:53.494815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:14:53.494815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:14:53.494815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:14:53.494815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:53.494815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:14:53.494815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:53.494815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:14:53.494815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:53.494815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:14:53.494815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:53.494815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:14:53.494815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:53.494815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:14:53.494815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:14:53.494815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:53.494815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:14:53.495297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:53.495297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:14:53.495297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:14:53.495297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:53.495297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:14:53.495297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:53.495297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:14:53.513915 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:14:53.514446 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:14:53.514694 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:14:53.515025 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:14:53.515685 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:14:53.516075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:53.516815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:14:53.516815 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:53.516815 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:14:53.516815 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:14:53.516815 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:53.516815 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:14:53.516815 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:53.516815 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:14:53.517557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:53.517557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:14:53.517557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:14:53.517557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:14:53.517557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:53.517557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:14:53.517557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:53.517557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:14:53.517557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:53.517557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:14:53.517557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:53.517557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:14:53.517557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:53.517557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:14:53.517557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:14:53.518194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:53.518194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:14:53.518194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:53.518194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:14:53.518194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:14:53.518194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:53.518194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:14:53.518194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:53.518194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:14:53.518578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:53.518578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:14:53.518578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:14:53.518578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:14:53.518578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:14:53.518578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:14:53.518578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:14:53.518578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:14:53.518578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:14:53.518578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:53.518578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:14:53.519061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:53.519061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:14:53.519061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:53.519061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:14:53.519061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:53.519061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:14:53.519061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:53.519061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:14:53.519061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:14:53.519061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:53.519061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:14:53.519061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:53.519061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:14:53.519061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:14:53.519645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:53.519645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:14:53.519645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:53.519645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:14:53.519645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:53.519645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:14:53.519645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:14:53.519645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:14:53.519645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:14:53.519645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:14:53.519645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:14:53.519645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:14:53.519645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:14:53.519645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:14:53.520272 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:14:53.520272 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:14:53.520272 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:14:53.520272 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:14:53.520272 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:14:53.520272 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:14:53.520272 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:14:53.520272 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:14:53.520272 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:14:53.520272 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:14:53.520272 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:53.520272 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:14:53.520272 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:53.520272 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:14:53.520272 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:53.520272 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:14:53.520272 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:53.520272 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:14:53.520272 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:53.521570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:14:53.521570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:14:53.521570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:53.521570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:14:53.521570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:53.521570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:14:53.521570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:14:53.521570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:53.521570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:14:53.521570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:53.521570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:14:53.521570 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:14:53.521570 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:14:53.523850 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:14:53.523850 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:14:53.523850 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:14:53.523850 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:14:53.523850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:53.524581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:14:53.524581 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:53.524581 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:14:53.524581 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:14:53.524581 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:53.524581 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:14:53.524581 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:53.524581 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:14:53.525677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:53.525677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:14:53.525677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:14:53.525677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:14:53.525677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:14:53.525677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:14:53.525677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:14:53.525677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:53.525677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:14:53.525677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:53.525677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:14:53.525677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:53.526224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:14:53.526224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:53.526224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:14:53.526224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:53.526224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:14:53.526224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:14:53.526224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:53.526224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:14:53.526224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:53.526224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:14:53.526224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:14:53.526224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:53.526224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:14:53.526599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:53.526599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:14:53.527252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:53.527252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:14:53.527252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:14:53.527252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:14:53.527252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:14:53.527252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:14:53.527252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:14:53.527252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:14:53.527252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:14:53.527252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:14:53.527252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:14:53.527252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:14:53.527252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:14:53.527252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:14:53.527252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:14:53.527252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:14:53.527686 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:14:53.527686 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:14:53.527686 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:53.527686 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:14:53.527686 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:53.527686 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:14:53.527686 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:53.527686 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:14:53.527686 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:53.527686 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:14:53.527686 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:53.527686 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:14:53.527686 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:14:53.527686 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:53.527686 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:14:53.527686 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:53.527686 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:14:53.527686 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:14:53.527686 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:53.528151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:14:53.528151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:53.528151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:14:57.934704 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:14:57.935663 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:14:57.935663 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:14:57.935963 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:14:57.981420 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:14:57.981679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:57.982407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:14:57.982407 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:57.982407 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:14:57.982407 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:14:57.982407 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:57.982407 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:14:57.982407 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:57.982407 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:14:57.983537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.047 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:57.983537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:14:57.983537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:14:57.983537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:14:57.983537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:14:57.983537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:14:57.983537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:14:57.983537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:57.983537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:14:57.983537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:57.983537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:14:57.983537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:57.984260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:14:57.984260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:57.984260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:14:57.984260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:57.984260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:14:57.984260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:14:57.984260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:57.984260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:14:57.984260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:57.984260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:14:57.984260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:14:57.984260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:57.984260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:14:57.985057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:57.985057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:14:57.985057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:57.985057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:14:57.985057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:14:57.985057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:14:57.985057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:14:57.985057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:14:57.985057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:14:57.985057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:14:57.985057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:14:57.985057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:14:57.985057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:14:57.985057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:14:57.985057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:14:57.986078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:14:57.986078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:14:57.986078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:14:57.986078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:14:57.986078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:14:57.986078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:57.986078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:14:57.986078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:57.986078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:14:57.986078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:57.986078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:14:57.986078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:57.986078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:14:57.986078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:57.986078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:14:57.986078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:14:57.986078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:57.986078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:14:57.986078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:57.987187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:14:57.987187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:14:57.987187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:57.987187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:14:57.987187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:57.987187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:14:58.485442 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:14:58.486086 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:14:58.486294 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:14:58.487649 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:14:58.488085 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:14:58.488335 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:58.489433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:14:58.489433 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:58.489433 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:14:58.489433 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:14:58.489433 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:58.489433 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:14:58.489433 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:58.489433 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:14:58.491611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:58.491611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:14:58.491611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:14:58.491611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:14:58.491611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:14:58.491611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:14:58.491611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:14:58.491611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:58.491611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:14:58.491611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:58.491611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:14:58.491611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:58.492461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:14:58.492461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:58.492461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:14:58.492461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:58.492461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:14:58.492461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:14:58.492461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:58.492461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:14:58.492461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:58.492461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:14:58.492461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:14:58.492461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:58.492461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:14:58.493290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:58.493290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:14:58.493883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:58.493883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:14:58.493883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:14:58.493883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:14:58.493883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:14:58.493883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:14:58.493883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:14:58.493883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:14:58.493883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:14:58.493883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:14:58.493883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:14:58.493883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:14:58.493883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:14:58.493883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:14:58.493883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:14:58.493883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:14:58.494869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:14:58.494869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:14:58.494869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:58.494869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:14:58.494869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:58.494869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:14:58.494869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:58.494869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:14:58.494869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:58.494869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:14:58.494869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:58.494869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:14:58.494869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:14:58.494869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:58.494869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:14:58.494869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:58.494869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:14:58.494869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:14:58.494869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:58.496166 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:14:58.496166 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:58.496166 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:14:58.496166 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:14:58.496166 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:14:58.496166 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:14:58.496166 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:14:58.496166 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:14:58.496697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:58.496926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:14:58.496926 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:58.496926 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:14:58.496926 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:14:58.496926 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:58.496926 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:14:58.496926 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:58.496926 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:14:58.497729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:58.497729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:14:58.497729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:14:58.497729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:14:58.497729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:58.497729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:14:58.497729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:58.497729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:14:58.497729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:58.497729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:14:58.497729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:58.497729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:14:58.497729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:58.497729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:14:58.497729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:14:58.498414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:58.498414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:14:58.498414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:58.498414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:14:58.498414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:14:58.498414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:58.498414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:14:58.498414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:58.498414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:14:58.498817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:58.498817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:14:58.498817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:14:58.498817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:14:58.498817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:14:58.498817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:14:58.498817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:14:58.498817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:14:58.498817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:14:58.498817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:14:58.498817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:58.498817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:14:58.499336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:58.499336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:14:58.499336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:58.499336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:14:58.499336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:58.499336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:14:58.499336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:58.499336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:14:58.499336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:14:58.499336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:58.499336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:14:58.499336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:58.499336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:14:58.499336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:14:58.499897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:58.499897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:14:58.499897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:58.499897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:14:58.499897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:58.499897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:14:58.499897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:14:58.499897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:14:58.499897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:14:58.499897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:14:58.499897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:14:58.499897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:14:58.499897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:14:58.499897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:14:58.500479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:14:58.500479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:14:58.500479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:14:58.500479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:14:58.500479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:14:58.500479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:14:58.500479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:14:58.500479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:14:58.500479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:14:58.500479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:14:58.500479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:14:58.500479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:58.500479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:14:58.500479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:58.500479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:14:58.500479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:58.500479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:14:58.500479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:58.500479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:14:58.501493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:58.501493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:14:58.501493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:14:58.501493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:58.501493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:14:58.501493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:58.501493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:14:58.501493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:14:58.501493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:58.501493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:14:58.501493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:58.501493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:14:58.514507 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:14:58.515097 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:14:58.515356 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:14:58.515643 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:14:58.516312 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:14:58.516619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:58.517595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:14:58.517595 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:58.517595 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:14:58.517595 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:14:58.517595 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:58.517595 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:14:58.517595 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:58.517595 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:14:58.519088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:58.519088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:14:58.519088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:14:58.519088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:14:58.519088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:58.519088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:14:58.519088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:58.519088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:14:58.519088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:58.519088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:14:58.519088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:58.519088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:14:58.519088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:58.519088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:14:58.519088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:14:58.519852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:58.519852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:14:58.519852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:58.519852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:14:58.519852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:14:58.519852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:58.519852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:14:58.519852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:58.519852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:14:58.520493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:58.520493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:14:58.520493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:14:58.520493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:14:58.520493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:14:58.520493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:14:58.520493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:14:58.520493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:14:58.520493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:14:58.520493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:58.520493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:14:58.521190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:58.521190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:14:58.521190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:58.521190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:14:58.521190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:58.521190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:14:58.521190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:58.521190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:14:58.521190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:14:58.521190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:58.521190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:14:58.521190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:58.521190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:14:58.521190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:14:58.521862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:58.521862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:14:58.521862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:58.521862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:14:58.522233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:58.522233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:14:58.522233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:14:58.522233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:14:58.522233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:14:58.522233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:14:58.522233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:14:58.522233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:14:58.522233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:14:58.522233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:14:58.522233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:14:58.522233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:14:58.522233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:14:58.522233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:14:58.522233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:14:58.522233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:14:58.522233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:14:58.522765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:14:58.522765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:14:58.522765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:14:58.522765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:58.522765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:14:58.522765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:58.522765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:14:58.522765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:58.522765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:14:58.522765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:58.522765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:14:58.522765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:58.522765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:14:58.522765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:14:58.522765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:58.522765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:14:58.522765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:58.522765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:14:58.522765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:14:58.523352 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:58.523352 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:14:58.523352 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:58.523352 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:14:58.523352 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:14:58.523352 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:14:58.523352 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:14:58.523352 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:14:58.523726 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:14:58.523936 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:14:58.524150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:58.524802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:14:58.524802 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:58.524802 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:14:58.524802 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:14:58.524802 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:58.524802 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:14:58.524802 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:58.524802 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:14:58.526130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:58.526130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:14:58.526130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:14:58.526130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:14:58.526130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:14:58.526130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:14:58.526130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:14:58.526130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:58.526130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:14:58.526130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:58.526130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:14:58.526130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:58.526565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:14:58.526565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:58.526565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:14:58.526565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:58.526565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:14:58.526565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:14:58.526565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:58.526565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:14:58.526565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:58.526565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:14:58.526565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:14:58.526565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:58.526565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:14:58.526941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:58.526941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:14:58.527092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:58.527092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:14:58.527092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:14:58.527092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:14:58.527092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:14:58.527092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:14:58.527092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:14:58.527092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:14:58.527092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:14:58.527092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:14:58.527092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:14:58.527092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:14:58.527092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:14:58.527092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:14:58.527092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:14:58.527092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:14:58.527532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:14:58.527532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:14:58.527532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:58.527532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:14:58.527532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:14:58.527532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:14:58.527532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:14:58.527532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:14:58.527532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:14:58.527532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:14:58.527532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:14:58.527532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:14:58.527532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:14:58.527532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:14:58.527532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:14:58.527532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:14:58.527532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:14:58.527532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:14:58.527532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:14:58.528080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:14:58.528080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:14:58.528080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:15:02.935503 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:15:02.936697 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:15:02.936697 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:15:02.937804 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:15:02.983191 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:15:02.983482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:02.984301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:15:02.984301 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:02.984301 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:15:02.984301 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:15:02.984301 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:02.984301 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:15:02.984301 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:02.984301 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:15:02.985445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.048 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:02.985445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:15:02.985445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:15:02.985445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:15:02.985445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:15:02.985445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:15:02.985445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:15:02.985445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:02.985445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:15:02.985445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:02.985445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:15:02.985445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:02.986214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:15:02.986214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:02.986214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:15:02.986214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:02.986214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:15:02.986214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:15:02.986214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:02.986214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:15:02.986214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:02.986214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:15:02.986214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:15:02.986214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:02.986214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:15:02.987055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:02.987055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:15:02.987055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:02.987055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:15:02.987055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:15:02.987055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:15:02.987055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:15:02.987055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:15:02.987055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:15:02.987055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:15:02.987055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:15:02.987055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:15:02.987055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:15:02.987055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:15:02.987055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:15:02.987624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:15:02.987624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:15:02.987624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:15:02.987624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:15:02.987624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:15:02.987624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:02.987624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:15:02.987624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:02.987624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:15:02.987624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:02.987624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:15:02.987624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:02.987624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:15:02.987624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:02.987624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:15:02.987624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:15:02.987624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:02.987624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:15:02.987624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:02.988290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:15:02.988290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:15:02.988290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:02.988290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:15:02.988290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:02.988290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:15:03.486284 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:15:03.487190 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:15:03.487465 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:15:03.488650 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:15:03.489237 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:15:03.489485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:03.490337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:15:03.490337 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:03.490337 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:15:03.490337 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:15:03.490337 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:03.490337 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:15:03.490337 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:03.490337 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:15:03.492919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:03.492919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:15:03.492919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:15:03.492919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:15:03.492919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:15:03.492919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:15:03.492919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:15:03.492919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:03.492919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:15:03.492919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:03.492919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:15:03.492919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:03.493371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:15:03.493371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:03.493371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:15:03.493371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:03.493371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:15:03.493371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:15:03.493371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:03.493371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:15:03.493371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:03.493371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:15:03.493371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:15:03.493371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:03.493371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:15:03.493931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:03.493931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:15:03.495009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:03.495009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:15:03.495009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:15:03.495009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:15:03.495009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:15:03.495009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:15:03.495009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:15:03.495009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:15:03.495009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:15:03.495009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:15:03.495009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:15:03.495009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:15:03.495009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:15:03.495009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:15:03.495009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:15:03.495009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:15:03.495546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:15:03.495546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:15:03.495546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:03.495546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:15:03.495546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:03.495546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:15:03.495546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:03.495546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:15:03.495546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:03.495546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:15:03.495546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:03.495546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:15:03.495546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:15:03.495546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:03.495546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:15:03.495546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:03.495546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:15:03.495546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:15:03.495546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:03.496232 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:15:03.496232 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:03.496232 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:15:03.496232 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:15:03.496232 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:15:03.496232 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:15:03.496455 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:15:03.496979 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:15:03.497337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:03.498158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:15:03.498158 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:03.498158 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:15:03.498158 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:15:03.498158 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:03.498158 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:15:03.498158 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:03.498158 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:15:03.499185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:03.499185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:15:03.499185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:15:03.499185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:15:03.499185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:03.499185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:15:03.499185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:03.499185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:15:03.499185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:03.499185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:15:03.499185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:03.499185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:15:03.499185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:03.499185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:15:03.499185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:15:03.499728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:03.499728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:15:03.499728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:03.499728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:15:03.499728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:15:03.499728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:03.499728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:15:03.499728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:03.499728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:15:03.500079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:03.500079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:15:03.500079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:15:03.500079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:15:03.500079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:15:03.500079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:15:03.500079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:15:03.500079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:15:03.500079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:15:03.500079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:15:03.500079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:03.500079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:15:03.500508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:03.500508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:15:03.500508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:03.500508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:15:03.500508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:03.500508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:15:03.500508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:03.500508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:15:03.500508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:15:03.500508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:03.500508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:15:03.500508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:03.500508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:15:03.500508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:15:03.500990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:03.500990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:15:03.500990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:03.500990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:15:03.500990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:03.500990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:15:03.500990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:15:03.500990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:15:03.500990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:15:03.500990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:15:03.500990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:15:03.500990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:15:03.500990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:15:03.500990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:15:03.501462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:15:03.501462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:15:03.501462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:15:03.501462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:15:03.501462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:15:03.501462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:15:03.501462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:15:03.501462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:15:03.501462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:15:03.501462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:15:03.501462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:15:03.501462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:03.501462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:15:03.501462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:03.501462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:15:03.501462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:03.501462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:15:03.501462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:03.501462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:15:03.502124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:03.502124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:15:03.502124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:15:03.502124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:03.502124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:15:03.502124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:03.502124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:15:03.502124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:15:03.502124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:03.502124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:15:03.502124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:03.502124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:15:03.515165 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:15:03.515943 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:15:03.516217 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:15:03.516440 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:15:03.519008 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:15:03.519008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:03.519008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:15:03.519008 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:03.519008 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:15:03.519008 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:15:03.519008 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:03.519008 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:15:03.519008 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:03.519008 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:15:03.519951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:03.519951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:15:03.519951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:15:03.519951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:15:03.519951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:03.519951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:15:03.519951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:03.519951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:15:03.519951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:03.519951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:15:03.519951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:03.519951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:15:03.519951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:03.519951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:15:03.519951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:15:03.520908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:03.520908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:15:03.520908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:03.520908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:15:03.520908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:15:03.520908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:03.520908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:15:03.520908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:03.520908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:15:03.521671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:03.521671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:15:03.521671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:15:03.521671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:15:03.521671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:15:03.521671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:15:03.521671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:15:03.521671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:15:03.521671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:15:03.521671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:03.521671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:15:03.522215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:03.522215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:15:03.522215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:03.522215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:15:03.522215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:03.522215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:15:03.522215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:03.522215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:15:03.522215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:15:03.522215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:03.522215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:15:03.522215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:03.522215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:15:03.522215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:15:03.522775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:03.522775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:15:03.522775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:03.522775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:15:03.523017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:03.523017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:15:03.523017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:15:03.523017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:15:03.523017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:15:03.523017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:15:03.523017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:15:03.523017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:15:03.523017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:15:03.523017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:15:03.523017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:15:03.523017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:15:03.523017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:15:03.523017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:15:03.523017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:15:03.523017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:15:03.523017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:15:03.523772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:15:03.523772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:15:03.523772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:15:03.523772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:03.523772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:15:03.523772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:03.523772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:15:03.523772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:03.523772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:15:03.523772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:03.523772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:15:03.523772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:03.523772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:15:03.523772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:15:03.523772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:03.523772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:15:03.523772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:03.523772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:15:03.523772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:15:03.524420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:03.524420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:15:03.524420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:03.524420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:15:03.524420 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:15:03.524420 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:15:03.524420 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:15:03.524420 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:15:03.524711 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:15:03.524711 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:15:03.529857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:03.529857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:15:03.529857 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:03.529857 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:15:03.529857 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:15:03.529857 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:03.529857 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:15:03.529857 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:03.529857 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:15:03.529857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:03.529857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:15:03.529857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:15:03.529857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:15:03.529857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:15:03.529857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:15:03.530463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:15:03.530463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:03.530463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:15:03.530463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:03.530463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:15:03.530463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:03.530463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:15:03.530463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:03.530463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:15:03.530463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:03.530463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:15:03.530463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:15:03.530463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:03.530463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:15:03.530920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:03.530920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:15:03.530920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:15:03.530920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:03.530920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:15:03.530920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:03.530920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:15:03.530920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:03.530920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:15:03.530920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:15:03.530920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:15:03.530920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:15:03.530920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:15:03.531270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:15:03.531270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:15:03.531270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:15:03.531270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:15:03.531270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:15:03.531270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:15:03.531270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:15:03.531270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:15:03.531270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:15:03.531270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:15:03.531270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:15:03.531270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:15:03.531270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:03.531270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:15:03.531270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:03.531270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:15:03.531270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:03.531270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:15:03.531270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:03.531270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:15:03.533018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:03.533018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:15:03.533018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:15:03.533018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:03.533018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:15:03.533018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:03.533018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:15:03.533018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:15:03.533018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:03.533018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:15:03.533018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:03.533018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:15:07.936445 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:15:07.937567 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:15:07.937567 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:15:07.938328 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:15:07.973149 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:15:07.973431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:07.974496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:15:07.974496 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:07.974496 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:15:07.974496 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:15:07.974496 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:07.974496 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:15:07.974496 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:07.974496 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:15:07.975551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:07.975551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:15:07.975551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:15:07.975551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:15:07.975551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:15:07.975551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:15:07.975551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:15:07.975551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:07.975551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:15:07.975551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:07.975551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:15:07.975551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:07.976794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:15:07.976794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:07.976794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:15:07.976794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:07.976794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:15:07.976794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:15:07.976794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:07.976794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:15:07.976794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:07.976794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:15:07.976794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:15:07.976794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:07.976794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:15:07.977328 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:07.977328 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:15:07.977328 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:07.977328 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:15:07.977328 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:15:07.977328 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:15:07.977328 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:15:07.977328 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:15:07.977328 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:15:07.977328 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:15:07.977328 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:15:07.977328 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:15:07.977328 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:15:07.977328 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:15:07.977328 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:15:07.977966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:15:07.977966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:15:07.977966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:15:07.977966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:15:07.977966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:15:07.977966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:07.977966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:15:07.977966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:07.977966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:15:07.977966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:07.977966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:15:07.977966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:07.977966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:15:07.977966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:07.977966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:15:07.977966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:15:07.977966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:07.977966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:15:07.977966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:07.978721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:15:07.978721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:15:07.978721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:07.978721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:15:07.978721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:07.978721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:15:08.487218 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:15:08.487905 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:15:08.488111 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:15:08.489421 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:15:08.489763 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:15:08.490047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:08.491067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:15:08.491067 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:08.491067 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:15:08.491067 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:15:08.491067 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:08.491067 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:15:08.491067 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:08.491067 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:15:08.492764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:08.492764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:15:08.492764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:15:08.492764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:15:08.492764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:15:08.492764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:15:08.492764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:15:08.492764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:08.492764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:15:08.492764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:08.492764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:15:08.492764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:08.493996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:15:08.493996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:08.493996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:15:08.493996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:08.493996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:15:08.493996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:15:08.493996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:08.493996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:15:08.493996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:08.493996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:15:08.493996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:15:08.493996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:08.493996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:15:08.494554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:08.494554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:15:08.494554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:08.494554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:15:08.494554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:15:08.494554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:15:08.494554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:15:08.494554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:15:08.494554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:15:08.494554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:15:08.494554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:15:08.494554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:15:08.494554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:15:08.494554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:15:08.494554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:15:08.495274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:15:08.495274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:15:08.495274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:15:08.495274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:15:08.495274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:15:08.495274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:08.495274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:15:08.495274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:08.495274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:15:08.495274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:08.495274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:15:08.495274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:08.495274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:15:08.495274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:08.495274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:15:08.495274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:15:08.495274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:08.495274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:15:08.495274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:08.496151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:15:08.496151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:15:08.496151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:08.496151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:15:08.496151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:08.496151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:15:08.496151 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:15:08.496524 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:15:08.496524 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:15:08.496524 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:15:08.496993 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:15:08.497216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:08.497825 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:15:08.497825 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:08.497825 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:15:08.497825 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:15:08.497825 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:08.497825 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:15:08.497825 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:08.497825 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:15:08.498556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:08.498556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:15:08.498556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:15:08.498556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:15:08.498556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:08.498556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:15:08.498556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:08.498556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:15:08.498556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:08.498556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:15:08.498556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:08.498556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:15:08.498556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:08.498556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:15:08.498556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:15:08.499367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:08.499367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:15:08.499367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:08.499367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:15:08.499367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:15:08.499367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:08.499367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:15:08.499367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:08.499367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:15:08.499367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:08.499367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:15:08.499367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:15:08.499367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:15:08.499367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:15:08.499994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:15:08.499994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:15:08.499994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:15:08.499994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:15:08.499994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:15:08.499994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:08.499994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:15:08.499994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:08.499994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:15:08.499994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:08.499994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:15:08.499994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:08.499994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:15:08.500732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:08.500732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:15:08.500732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:15:08.500732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:08.500732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:15:08.500732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:08.500732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:15:08.500732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:15:08.500732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:08.500732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:15:08.500732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:08.500732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:15:08.501223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:08.501223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:15:08.501223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:15:08.501223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:15:08.501223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:15:08.501223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:15:08.501223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:15:08.501223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:15:08.501223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:15:08.501223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:15:08.501223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:15:08.501223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:15:08.501223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:15:08.501223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:15:08.501223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:15:08.501223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:15:08.501681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:15:08.501681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:15:08.501681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:15:08.501681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:15:08.501681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:15:08.501681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:08.501681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:15:08.501681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:08.501681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:15:08.501681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:08.501681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:15:08.501681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:08.501681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:15:08.501681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:08.501681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:15:08.501681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:15:08.501681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:08.501681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:15:08.502241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:08.502241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:15:08.502241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:15:08.502241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:08.502241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:15:08.502241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:08.502241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:15:08.515825 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:15:08.516279 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:15:08.516468 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:15:08.516764 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:15:08.517469 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:15:08.517907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:08.518735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:15:08.518735 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:08.518735 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:15:08.518735 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:15:08.518735 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:08.518735 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:15:08.518735 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:08.518735 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:15:08.520113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:08.520113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:15:08.520113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:15:08.520113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:15:08.520113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:08.520113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:15:08.520113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:08.520113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:15:08.520113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:08.520113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:15:08.520113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:08.520113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:15:08.520113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:08.520113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:15:08.520113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:15:08.521018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:08.521018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:15:08.521018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:08.521018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:15:08.521018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:15:08.521018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:08.521018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:15:08.521018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:08.521018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:15:08.521843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:08.521843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:15:08.521843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:15:08.521843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:15:08.521843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:15:08.521843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:15:08.521843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:15:08.521843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:15:08.521843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:15:08.521843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:08.521843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:15:08.522496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:08.522496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:15:08.522496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:08.522496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:15:08.522496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:08.522496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:15:08.522496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:08.522496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:15:08.522496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:15:08.522496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:08.522496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:15:08.522496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:08.522496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:15:08.522496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:15:08.523109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:08.523109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:15:08.523109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:08.523109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:15:08.524079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:08.524079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:15:08.524079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:15:08.524079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:15:08.524079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:15:08.524079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:15:08.524079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:15:08.524079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:15:08.524079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:15:08.524079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:15:08.524079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:15:08.524079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:15:08.524079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:15:08.524079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:15:08.524079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:15:08.524079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:15:08.524079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:15:08.524957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:15:08.524957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:15:08.524957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:15:08.524957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:08.524957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:15:08.524957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:08.524957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:15:08.524957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:08.524957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:15:08.524957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:08.524957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:15:08.524957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:08.524957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:15:08.524957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:15:08.524957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:08.524957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:15:08.524957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:08.524957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:15:08.524957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:15:08.525719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:08.525719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:15:08.525719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:08.525719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:15:08.525719 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:15:08.525719 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:15:08.525719 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:15:08.525719 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:15:08.525998 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:15:08.526316 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:15:08.526631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:08.527452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:15:08.527452 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:08.527452 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:15:08.527452 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:15:08.527452 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:08.527452 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:15:08.527452 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:08.527452 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:15:08.528313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:08.528313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:15:08.528313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:15:08.528313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:15:08.528313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:15:08.528313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:15:08.528313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:15:08.528313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:08.528313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:15:08.528313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:08.528313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:15:08.528313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:08.528785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:15:08.528785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:08.528785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:15:08.528785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:08.528785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:15:08.528785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:15:08.528785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:08.528785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:15:08.528785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:08.528785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:15:08.528785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:15:08.528785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:08.528785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:15:08.529188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:08.529188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:15:08.529343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:08.529343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:15:08.529343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:15:08.529343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:15:08.529343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:15:08.529343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:15:08.529343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:15:08.529343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:15:08.529343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:15:08.529343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:15:08.529343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:15:08.529343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:15:08.529343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:15:08.529343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:15:08.529343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:15:08.529343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:15:08.529867 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:15:08.529867 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:15:08.529867 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:08.529867 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:15:08.529867 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:08.529867 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:15:08.529867 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:08.529867 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:15:08.529867 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:08.529867 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:15:08.529867 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:08.529867 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:15:08.529867 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:15:08.529867 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:08.529867 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:15:08.529867 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:08.529867 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:15:08.529867 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:15:08.529867 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:08.530406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:15:08.530406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:08.530406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:15:12.937199 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:15:12.937993 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:15:12.937993 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:15:12.939019 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:15:12.983391 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:15:12.983906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:12.984911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:15:12.984911 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:12.984911 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:15:12.984911 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:15:12.984911 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:12.984911 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:15:12.984911 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:12.984911 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:15:12.985850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.047 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:12.985850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:15:12.985850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:15:12.985850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:15:12.985850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:15:12.985850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:15:12.985850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:15:12.985850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:12.985850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:15:12.985850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:12.985850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:15:12.985850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:12.986192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:15:12.986192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:12.986192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:15:12.986192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:12.986192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:15:12.986192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:15:12.986192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:12.986192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:15:12.986192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:12.986192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:15:12.986192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:15:12.986192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:12.986192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:15:12.986579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:12.986579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:15:12.986765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:12.986765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:15:12.986765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:15:12.986765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:15:12.986765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:15:12.986765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:15:12.986765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:15:12.986765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:15:12.986765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:15:12.986765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:15:12.986765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:15:12.986765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:15:12.986765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:15:12.986765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:15:12.986765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:15:12.986765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:15:12.987205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:15:12.987205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:15:12.987205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:12.987205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:15:12.987205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:12.987205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:15:12.987205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:12.987205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:15:12.987205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:12.987205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:15:12.987205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:12.987205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:15:12.987205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:15:12.987205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:12.987205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:15:12.987205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:12.987205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:15:12.987205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:15:12.987205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:12.987677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:15:12.987677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:12.987677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:15:13.488129 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:15:13.488628 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:15:13.488873 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:15:13.490222 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:15:13.490545 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:15:13.490813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:13.491576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:15:13.491576 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:13.491576 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:15:13.491576 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:15:13.491576 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:13.491576 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:15:13.491576 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:13.491576 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:15:13.492662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:13.492662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:15:13.492662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:15:13.492662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:15:13.492662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:15:13.492662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:15:13.492662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:15:13.492662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:13.492662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:15:13.492662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:13.492662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:15:13.492662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:13.493259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:15:13.493259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:13.493259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:15:13.493259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:13.493259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:15:13.493259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:15:13.493259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:13.493259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:15:13.493259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:13.493259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:15:13.493259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:15:13.493259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:13.493259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:15:13.493887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:13.493887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:15:13.494184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:13.494184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:15:13.494184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:15:13.494184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:15:13.494184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:15:13.494184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:15:13.494184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:15:13.494184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:15:13.494184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:15:13.494184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:15:13.494184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:15:13.494184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:15:13.494184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:15:13.494184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:15:13.494184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:15:13.494184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:15:13.495048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:15:13.495048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:15:13.495048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:13.495048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:15:13.495048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:13.495048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:15:13.495048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:13.495048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:15:13.495048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:13.495048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:15:13.495048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:13.495048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:15:13.495048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:15:13.495048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:13.495048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:15:13.495048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:13.495048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:15:13.495048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:15:13.495048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:13.495663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:15:13.495663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:13.495663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:15:13.496341 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:15:13.496655 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:15:13.496849 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:15:13.497030 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:15:13.497381 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:15:13.497568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:13.498068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:15:13.498068 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:13.498068 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:15:13.498068 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:15:13.498068 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:13.498068 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:15:13.498068 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:13.498068 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:15:13.498965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:13.498965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:15:13.498965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:15:13.498965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:15:13.498965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:13.498965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:15:13.498965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:13.498965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:15:13.498965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:13.498965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:15:13.498965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:13.498965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:15:13.498965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:13.498965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:15:13.498965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:15:13.499627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:13.499627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:15:13.499627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:13.499627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:15:13.499627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:15:13.499627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:13.499627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:15:13.499627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:13.499627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:15:13.500151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:13.500151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:15:13.500151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:15:13.500151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:15:13.500151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:15:13.500151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:15:13.500151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:15:13.500151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:15:13.500151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:15:13.500151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:15:13.500151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:13.500151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:15:13.500661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:13.500661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:15:13.500661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:13.500661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:15:13.500661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:13.500661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:15:13.500661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:13.500661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:15:13.500661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:15:13.500661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:13.500661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:15:13.500661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:13.500661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:15:13.500661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:15:13.501254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:13.501254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:15:13.501254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:13.501254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:15:13.501602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:13.501602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:15:13.501602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:15:13.501602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:15:13.501602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:15:13.501602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:15:13.501602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:15:13.501602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:15:13.501602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:15:13.501602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:15:13.501602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:15:13.501602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:15:13.501602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:15:13.501602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:15:13.501602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:15:13.501602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:15:13.502104 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:15:13.502104 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:15:13.502104 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:15:13.502104 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:15:13.502104 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:15:13.502104 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:13.502104 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:15:13.502104 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:13.502104 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:15:13.502104 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:13.502104 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:15:13.502104 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:13.502104 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:15:13.502104 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:13.502104 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:15:13.502104 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:15:13.502104 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:13.502104 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:15:13.502619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:13.502619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:15:13.502619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:15:13.502619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:13.502619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:15:13.502619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:13.502619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:15:13.517650 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:15:13.518155 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:15:13.518447 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:15:13.518733 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:15:13.519295 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:15:13.519783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:13.520597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:15:13.520597 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:13.520597 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:15:13.520597 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:15:13.520597 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:13.520597 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:15:13.520597 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:13.520597 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:15:13.522026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:13.522026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:15:13.522026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:15:13.522026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:15:13.522026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:13.522026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:15:13.522026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:13.522026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:15:13.522026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:13.522026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:15:13.522026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:13.522026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:15:13.522026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:13.522026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:15:13.522026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:15:13.522867 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:13.522867 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:15:13.522867 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:13.522867 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:15:13.522867 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:15:13.522867 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:13.522867 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:15:13.522867 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:13.522867 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:15:13.524252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:13.524252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:15:13.524252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:15:13.524252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:15:13.524252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:15:13.524252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:15:13.524252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:15:13.524252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:15:13.524252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:15:13.524252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:13.524252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:15:13.524815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:13.524815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:15:13.524815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:13.524815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:15:13.524815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:13.524815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:15:13.524815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:13.524815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:15:13.524815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:15:13.524815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:13.524815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:15:13.524815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:13.524815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:15:13.524815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:15:13.525480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:13.525480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:15:13.525480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:13.525480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:15:13.526678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:13.526678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:15:13.526678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:15:13.526678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:15:13.526678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:15:13.526678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:15:13.526678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:15:13.526678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:15:13.526678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:15:13.526678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:15:13.526678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:15:13.526678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:15:13.526678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:15:13.526678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:15:13.526678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:15:13.526678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:15:13.526678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:15:13.527472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:15:13.527472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:15:13.527472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:15:13.527472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:13.527472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:15:13.527472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:13.527472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:15:13.527472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:13.527472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:15:13.527472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:13.527472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:15:13.527472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:13.527472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:15:13.527472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:15:13.527472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:13.527472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:15:13.527472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:13.527472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:15:13.527472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:15:13.528378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:13.528378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:15:13.528378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:13.528378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:15:13.528378 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:15:13.528378 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:15:13.528378 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:15:13.528378 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:15:13.529120 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:15:13.529120 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:15:13.529120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:13.529120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:15:13.529120 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:13.529120 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:15:13.529120 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:15:13.529120 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:13.529120 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:15:13.529120 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:13.529120 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:15:13.530021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:13.530021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:15:13.530021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:15:13.530021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:15:13.530021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:15:13.530021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:15:13.530021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:15:13.530021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:13.530021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:15:13.530021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:13.530021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:15:13.530021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:13.530434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:15:13.530434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:13.530434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:15:13.530434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:13.530434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:15:13.530434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:15:13.530434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:13.530434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:15:13.530434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:13.530434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:15:13.530434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:15:13.530434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:13.530434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:15:13.530995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:13.530995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:15:13.532195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:13.532195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:15:13.532195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:15:13.532195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:15:13.532195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:15:13.532195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:15:13.532195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:15:13.532195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:15:13.532195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:15:13.532195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:15:13.532195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:15:13.532195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:15:13.532195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:15:13.532195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:15:13.532195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:15:13.532195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:15:13.532861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:15:13.532861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:15:13.532861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:13.532861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:15:13.532861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:13.532861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:15:13.532861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:13.532861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:15:13.532861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:13.532861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:15:13.532861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:13.532861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:15:13.532861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:15:13.532861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:13.532861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:15:13.532861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:13.532861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:15:13.532861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:15:13.532861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:13.533441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:15:13.533441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:13.533441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:15:17.937951 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:15:17.939667 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:15:17.939916 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:15:17.941394 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:15:17.978168 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:15:17.978595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:17.979985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:15:17.979985 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:17.979985 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:15:17.979985 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:15:17.979985 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:17.979985 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:15:17.979985 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:17.979985 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:15:17.981231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:17.981231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:15:17.981231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:15:17.981231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:15:17.981231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:15:17.981231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:15:17.981231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:15:17.981231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:17.981231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:15:17.981231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:17.981231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:15:17.981231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:17.982010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:15:17.982010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:17.982010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:15:17.982010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:17.982010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:15:17.982010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:15:17.982010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:17.982010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:15:17.982010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:17.982010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:15:17.982010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:15:17.982010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:17.982010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:15:17.983163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:17.983163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:15:17.983163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:17.983163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:15:17.983163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:15:17.983163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:15:17.983163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:15:17.983163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:15:17.983163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:15:17.983163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:15:17.983163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:15:17.983163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:15:17.983163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:15:17.983163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:15:17.983163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:15:17.983884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:15:17.983884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:15:17.983884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:15:17.983884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:15:17.983884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:15:17.983884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:17.983884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:15:17.983884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:17.983884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:15:17.983884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:17.983884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:15:17.983884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:17.983884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:15:17.983884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:17.983884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:15:17.983884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:15:17.983884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:17.983884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:15:17.983884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:17.984704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:15:17.984704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:15:17.984704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:17.984704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:15:17.984704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:17.984704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:15:18.491101 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:15:18.491101 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:15:18.491101 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:15:18.491101 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:15:18.491101 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:15:18.491101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:18.492474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:15:18.492474 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:18.492474 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:15:18.492474 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:15:18.492474 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:18.492474 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:15:18.492474 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:18.492474 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:15:18.492474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:18.492474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:15:18.492474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:15:18.492474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:15:18.492474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:15:18.492474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:15:18.492474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:15:18.494286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:18.494286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:15:18.494286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:18.494286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:15:18.494286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:18.494286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:15:18.494286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:18.494286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:15:18.494286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:18.494286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:15:18.494286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:15:18.494286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:18.494286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:15:18.494286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:18.495506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:15:18.495506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:15:18.495506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:18.495506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:15:18.495506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:18.495506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:15:18.495506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:18.495506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:15:18.495506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:15:18.495506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:15:18.495506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:15:18.495506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:15:18.495506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:15:18.495506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:15:18.497039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:15:18.497039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:15:18.497039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:15:18.497039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:15:18.497039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:15:18.497039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:15:18.497039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:15:18.497039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:15:18.497039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:15:18.497039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:15:18.497039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:18.497039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:15:18.497039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:18.497039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:15:18.497039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:18.497039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:15:18.497039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:18.497039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:15:18.497039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:18.497039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:15:18.498093 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:15:18.498093 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:18.498093 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:15:18.498093 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:18.498093 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:15:18.498093 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:15:18.498093 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:18.498093 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:15:18.498093 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:18.498093 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:15:18.498093 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:15:18.498093 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:15:18.498708 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:15:18.498708 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:15:18.498708 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:15:18.498708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:18.498708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:15:18.498708 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:18.498708 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:15:18.498708 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:15:18.498708 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:18.498708 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:15:18.498708 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:18.498708 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:15:18.500929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:18.500929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:15:18.500929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:15:18.500929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:15:18.500929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:18.500929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:15:18.500929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:18.500929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:15:18.500929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:18.500929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:15:18.500929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:18.500929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:15:18.500929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:18.500929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:15:18.500929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:15:18.501710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:18.501710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:15:18.501710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:18.501710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:15:18.501710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:15:18.501710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:18.501710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:15:18.501710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:18.501710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:15:18.501710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:18.501710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:15:18.501710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:15:18.501710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:15:18.501710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:15:18.502342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:15:18.502342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:15:18.502342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:15:18.502342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:15:18.502342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:15:18.502342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:18.502342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:15:18.502342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:18.502342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:15:18.502342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:18.502342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:15:18.502342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:18.502342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:15:18.502970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:18.502970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:15:18.502970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:15:18.502970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:18.502970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:15:18.502970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:18.502970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:15:18.502970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:15:18.502970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:18.502970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:15:18.502970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:18.502970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:15:18.503197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:18.503197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:15:18.503197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:15:18.503197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:15:18.503197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:15:18.503197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:15:18.503197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:15:18.503197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:15:18.503197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:15:18.503197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:15:18.503197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:15:18.503197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:15:18.503197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:15:18.503197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:15:18.503197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:15:18.503197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:15:18.503572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:15:18.503572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:15:18.503572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:15:18.503572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:15:18.503572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:15:18.503572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:18.503572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:15:18.503572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:18.503572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:15:18.503572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:18.503572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:15:18.503572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:18.503572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:15:18.503572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:18.503572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:15:18.503572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:15:18.503572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:18.503572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:15:18.504173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:18.504173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:15:18.504173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:15:18.504173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:18.504173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:15:18.504173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:18.504173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:15:18.518228 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:15:18.518720 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:15:18.518720 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:15:18.518720 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:15:18.519196 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:15:18.519414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:18.519972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:15:18.519972 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:18.519972 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:15:18.519972 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:15:18.519972 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:18.519972 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:15:18.519972 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:18.519972 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:15:18.520789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:18.520789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:15:18.520789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:15:18.520789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:15:18.520789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:18.520789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:15:18.520789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:18.520789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:15:18.520789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:18.520789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:15:18.520789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:18.520789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:15:18.520789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:18.520789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:15:18.520789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:15:18.521614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:18.521614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:15:18.521614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:18.521614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:15:18.521614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:15:18.521614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:18.521614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:15:18.521614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:18.521614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:15:18.521614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:18.521614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:15:18.521614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:15:18.521614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:15:18.521614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:15:18.522293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:15:18.522293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:15:18.522293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:15:18.522293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:15:18.522293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:18.522293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:15:18.522293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:18.522293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:15:18.522293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:18.522293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:15:18.522293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:18.522293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:15:18.522293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:18.522907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:15:18.522907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:15:18.522907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:18.522907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:15:18.522907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:18.522907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:15:18.522907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:15:18.522907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:18.522907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:15:18.522907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:18.522907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:15:18.522907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:18.523684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:15:18.523684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:15:18.523684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:15:18.523684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:15:18.523684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:15:18.523684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:15:18.523684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:15:18.523684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:15:18.523684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:15:18.523684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:15:18.523684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:15:18.523684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:15:18.523684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:15:18.523684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:15:18.523684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:15:18.523684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:15:18.523684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:15:18.523684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:15:18.523684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:15:18.526404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:18.526404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:15:18.526404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:18.526404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:15:18.526404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:18.526404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:15:18.526404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:18.526404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:15:18.526404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:18.526404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:15:18.526404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:15:18.526404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:18.526404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:15:18.526404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:18.526404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:15:18.526404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:15:18.526404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:18.526404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:15:18.526404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:18.526404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:15:18.527939 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:15:18.527939 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:15:18.527939 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:15:18.528165 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:15:18.528235 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:15:18.528482 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:15:18.528627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:18.529077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:15:18.529077 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:18.529077 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:15:18.529077 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:15:18.529077 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:18.529077 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:15:18.529077 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:18.529077 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:15:18.529887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:18.529887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:15:18.529887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:15:18.529887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:15:18.529887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:15:18.529887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:15:18.529887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:15:18.529887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:18.529887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:15:18.529887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:18.529887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:15:18.529887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:18.530511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:15:18.530511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:18.530511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:15:18.530511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:18.530511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:15:18.530511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:15:18.530511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:18.530511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:15:18.530511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:18.530511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:15:18.530511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:15:18.530511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:18.530511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:15:18.531111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:18.531111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:15:18.531111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:18.531111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:15:18.531111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:15:18.531111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:15:18.531111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:15:18.531111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:15:18.531111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:15:18.531111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:15:18.531111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:15:18.531111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:15:18.531111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:15:18.531111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:15:18.531111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:15:18.531835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:15:18.531835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:15:18.531835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:15:18.531835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:15:18.531835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:15:18.531835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:18.531835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:15:18.531835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:18.531835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:15:18.531835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:18.531835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:15:18.531835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:18.531835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:15:18.531835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:18.531835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:15:18.531835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:15:18.531835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:18.531835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:15:18.531835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:18.532496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:15:18.532496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:15:18.532496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:18.532496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:15:18.532496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:18.532496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:15:22.939812 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:15:22.940684 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:15:22.940684 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:15:22.941993 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:15:22.977700 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:15:22.978046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:22.978612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:15:22.978612 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:22.978612 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:15:22.978612 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:15:22.978612 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:22.978612 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:15:22.978612 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:22.978612 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:15:22.979343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:22.979343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:15:22.979343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:15:22.979343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:15:22.979343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:15:22.979343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:15:22.979343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:15:22.979343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:22.979343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:15:22.979343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:22.979343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:15:22.979343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:22.979886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:15:22.979886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:22.979886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:15:22.979886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:22.979886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:15:22.979886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:15:22.979886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:22.979886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:15:22.979886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:22.979886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:15:22.979886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:15:22.979886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:22.979886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:15:22.980447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:22.980447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:15:22.980447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:22.980447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:15:22.980447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:15:22.980447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:15:22.980447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:15:22.980447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:15:22.980447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:15:22.980447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:15:22.980447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:15:22.980447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:15:22.980447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:15:22.980447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:15:22.980447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:15:22.981080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:15:22.981080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:15:22.981080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:15:22.981080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:15:22.981080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:15:22.981080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:22.981080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:15:22.981080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:22.981080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:15:22.981080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:22.981080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:15:22.981080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:22.981080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:15:22.981080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:22.981080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:15:22.981080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:15:22.981080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:22.981080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:15:22.981080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:22.981900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:15:22.981900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:15:22.981900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:22.981900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:15:22.981900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:22.981900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:15:23.489690 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:15:23.490149 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:15:23.490345 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:15:23.490943 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:15:23.491136 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:15:23.491310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:23.491928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:15:23.491928 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:23.491928 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:15:23.491928 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:15:23.491928 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:23.491928 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:15:23.491928 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:23.491928 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:15:23.492850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:23.492850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:15:23.492850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:15:23.492850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:15:23.492850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:15:23.492850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:15:23.492850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:15:23.492850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:23.492850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:15:23.492850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:23.492850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:15:23.492850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:23.493872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:15:23.493872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:23.493872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:15:23.493872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:23.493872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:15:23.493872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:15:23.493872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:23.493872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:15:23.493872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:23.493872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:15:23.493872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:15:23.493872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:23.493872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:15:23.495094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:23.495094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:15:23.495094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:23.495094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:15:23.495094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:15:23.495094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:15:23.495094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:15:23.495094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:15:23.495094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:15:23.495094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:15:23.495094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:15:23.495094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:15:23.495094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:15:23.495094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:15:23.495094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:15:23.495516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:15:23.495516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:15:23.495516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:15:23.495516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:15:23.495516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:15:23.495516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:23.495516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:15:23.495516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:23.495516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:15:23.495516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:23.495516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:15:23.495516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:23.495516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:15:23.495516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:23.495516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:15:23.495516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:15:23.495516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:23.495516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:15:23.495516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:23.496087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:15:23.496087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:15:23.496087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:23.496087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:15:23.496087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:23.496087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:15:23.497343 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:15:23.497869 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:15:23.497991 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:15:23.498136 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:15:23.498780 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:15:23.499182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:23.500121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:15:23.500121 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:23.500121 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:15:23.500121 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:15:23.500121 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:23.500121 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:15:23.500121 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:23.500121 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:15:23.501140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:23.501140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:15:23.501140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:15:23.501140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:15:23.501140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:23.501140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:15:23.501140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:23.501140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:15:23.501140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:23.501140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:15:23.501140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:23.501140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:15:23.501140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:23.501140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:15:23.501140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:15:23.502054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:23.502054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:15:23.502054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:23.502054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:15:23.502054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:15:23.502054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:23.502054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:15:23.502054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:23.502054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:15:23.502054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:23.502054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:15:23.502054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:15:23.502054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:15:23.502054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:15:23.502782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:15:23.502782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:15:23.502782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:15:23.502782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:15:23.502782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:15:23.502782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:23.502782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:15:23.502782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:23.502782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:15:23.502782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:23.502782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:15:23.502782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:23.502782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:15:23.503623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:23.503623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:15:23.503623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:15:23.503623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:23.503623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:15:23.503623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:23.503623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:15:23.503623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:15:23.503623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:23.503623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:15:23.503623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:23.503623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:15:23.504237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:23.504237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:15:23.504237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:15:23.504237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:15:23.504237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:15:23.504237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:15:23.504237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:15:23.504237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:15:23.504237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:15:23.504237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:15:23.504237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:15:23.504237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:15:23.504237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:15:23.504237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:15:23.504237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:15:23.504237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:15:23.505193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:15:23.505193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:15:23.505193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:15:23.505193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:15:23.505193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:15:23.505193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:23.505193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:15:23.505193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:23.505193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:15:23.505193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:23.505193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:15:23.505193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:23.505193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:15:23.505193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:23.505193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:15:23.505193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:15:23.505193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:23.505193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:15:23.506069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:23.506069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:15:23.506069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:15:23.506069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:23.506069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:15:23.506069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:23.506069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:15:23.518612 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:15:23.519178 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:15:23.519339 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:15:23.519509 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:15:23.520510 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:15:23.521091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:23.522032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:15:23.522032 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:23.522032 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:15:23.522032 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:15:23.522032 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:23.522032 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:15:23.522032 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:23.522032 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:15:23.531385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:23.531385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:15:23.531385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:15:23.531385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:15:23.531385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:23.531385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:15:23.531385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:23.531385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:15:23.531385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:23.531385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:15:23.531385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:23.531385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:15:23.531385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:23.531385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:15:23.531385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:15:23.531943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:23.531943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:15:23.531943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:23.531943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:15:23.531943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:15:23.531943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:23.531943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:15:23.531943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:23.531943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:15:23.531943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:23.531943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:15:23.531943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:15:23.531943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:15:23.531943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:15:23.532435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:15:23.532435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:15:23.532435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:15:23.532435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:15:23.532435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:23.532435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:15:23.532435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:23.532435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:15:23.532435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:23.532435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:15:23.532435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:23.532435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:15:23.532435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:23.532908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:15:23.532908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:15:23.532908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:23.532908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:15:23.532908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:23.532908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:15:23.532908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:15:23.532908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:23.532908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:15:23.532908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:23.532908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:15:23.532908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:23.533270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:15:23.533270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:15:23.533270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:15:23.533270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:15:23.533270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:15:23.533270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:15:23.533270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:15:23.533270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:15:23.533270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:15:23.533270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:15:23.533270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:15:23.533270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:15:23.533270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:15:23.533270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:15:23.533270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:15:23.533270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:15:23.533270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:15:23.533270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:15:23.533270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:15:23.533943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:23.533943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:15:23.533943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:23.533943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:15:23.533943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:23.533943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:15:23.533943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:23.533943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:15:23.533943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:23.533943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:15:23.533943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:15:23.533943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:23.533943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:15:23.533943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:23.533943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:15:23.533943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:15:23.533943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:23.533943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:15:23.533943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:23.533943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:15:23.534505 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:15:23.534505 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:15:23.534505 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:15:23.534505 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:15:23.534505 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:15:23.534505 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:15:23.534842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:23.534842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:15:23.534842 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:23.534842 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:15:23.534842 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:15:23.534842 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:23.534842 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:15:23.534842 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:23.534842 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:15:23.534842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:23.534842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:15:23.534842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:15:23.534842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:15:23.534842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:15:23.534842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:15:23.535274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:15:23.535274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:23.535274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:15:23.535274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:23.535274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:15:23.535274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:23.535274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:15:23.535274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:23.535274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:15:23.535274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:23.535274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:15:23.535274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:15:23.535274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:23.535274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:15:23.535673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:23.535673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:15:23.535673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:15:23.535673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:23.535673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:15:23.535673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:23.535673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:15:23.535673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:23.535673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:15:23.535673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:15:23.535673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:15:23.535673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:15:23.535673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:15:23.536110 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:15:23.536110 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:15:23.536110 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:15:23.536110 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:15:23.536110 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:15:23.536110 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:15:23.536110 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:15:23.536110 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:15:23.536110 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:15:23.536110 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:15:23.536110 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:15:23.536110 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:15:23.536110 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:23.536110 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:15:23.536110 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:23.536110 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:15:23.536110 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:23.536110 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:15:23.536110 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:23.536110 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:15:23.536657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:23.536657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:15:23.536657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:15:23.536657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:23.536657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:15:23.536657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:23.536657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:15:23.536657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:15:23.536657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:23.536657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:15:23.536657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:23.536657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:15:27.940495 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:15:27.941546 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:15:27.941546 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:15:27.942450 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:15:27.984904 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:15:27.984904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:27.985702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:15:27.985702 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:27.985702 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:15:27.985702 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:15:27.985702 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:27.985702 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:15:27.985702 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:27.985702 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:15:27.986882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.045 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:27.986882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:15:27.986882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:15:27.986882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:15:27.986882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:15:27.986882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:15:27.986882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:15:27.986882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:27.986882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:15:27.986882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:27.986882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:15:27.986882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:27.987614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:15:27.987614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:27.987614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:15:27.987614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:27.987614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:15:27.987614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:15:27.987614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:27.987614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:15:27.987614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:27.987614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:15:27.987614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:15:27.987614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:27.987614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:15:27.988319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:27.988319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:15:27.988319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:27.988319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:15:27.988319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:15:27.988319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:15:27.988319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:15:27.988319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:15:27.988319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:15:27.988319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:15:27.988319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:15:27.988319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:15:27.988319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:15:27.988319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:15:27.988319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:15:27.989213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:15:27.989213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:15:27.989213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:15:27.989213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:15:27.989213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:15:27.989213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:27.989213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:15:27.989213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:27.989213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:15:27.989213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:27.989213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:15:27.989213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:27.989213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:15:27.989213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:27.989213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:15:27.989213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:15:27.989213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:27.989213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:15:27.989213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:27.990183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:15:27.990183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:15:27.990183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:27.990183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:15:27.990183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:27.990183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:15:28.490413 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:15:28.490903 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:15:28.491093 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:15:28.492295 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:15:28.492693 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:15:28.493016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:28.493870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:15:28.493870 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:28.493870 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:15:28.493870 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:15:28.493870 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:28.493870 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:15:28.493870 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:28.493870 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:15:28.494807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:28.494807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:15:28.494807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:15:28.494807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:15:28.494807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:15:28.494807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:15:28.494807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:15:28.494807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:28.494807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:15:28.494807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:28.494807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:15:28.494807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:28.495410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:15:28.495410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:28.495410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:15:28.495410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:28.495410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:15:28.495410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:15:28.495410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:28.495410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:15:28.495410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:28.495410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:15:28.495410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:15:28.495410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:28.495410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:15:28.495948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:28.495948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:15:28.495948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:28.495948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:15:28.495948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:15:28.495948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:15:28.495948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:15:28.495948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:15:28.495948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:15:28.495948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:15:28.495948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:15:28.495948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:15:28.495948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:15:28.495948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:15:28.495948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:15:28.496586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:15:28.496586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:15:28.496586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:15:28.496586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:15:28.496586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:15:28.496586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:28.496586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:15:28.496586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:28.496586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:15:28.496586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:28.496586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:15:28.496586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:28.496586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:15:28.496586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:28.496586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:15:28.496586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:15:28.496586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:28.496586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:15:28.496586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:28.497399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:15:28.497399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:15:28.497399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:28.497399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:15:28.497399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:28.497399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:15:28.497972 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:15:28.498254 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:15:28.498355 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:15:28.498493 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:15:28.498948 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:15:28.499303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:28.499950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:15:28.499950 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:28.499950 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:15:28.499950 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:15:28.499950 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:28.499950 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:15:28.499950 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:28.499950 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:15:28.500666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:28.500666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:15:28.500666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:15:28.500666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:15:28.500666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:28.500666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:15:28.500666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:28.500666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:15:28.500666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:28.500666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:15:28.500666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:28.500666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:15:28.500666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:28.500666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:15:28.500666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:15:28.501382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:28.501382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:15:28.501382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:28.501382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:15:28.501382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:15:28.501382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:28.501382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:15:28.501382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:28.501382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:15:28.501809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:28.501809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:15:28.501809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:15:28.501809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:15:28.501809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:15:28.501809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:15:28.501809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:15:28.501809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:15:28.501809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:15:28.501809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:15:28.501809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:28.501809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:15:28.502344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:28.502344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:15:28.502344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:28.502344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:15:28.502344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:28.502344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:15:28.502344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:28.502344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:15:28.502344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:15:28.502344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:28.502344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:15:28.502344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:28.502344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:15:28.502344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:15:28.502951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:28.502951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:15:28.502951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:28.502951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:15:28.502951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:28.502951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:15:28.502951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:15:28.502951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:15:28.502951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:15:28.502951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:15:28.502951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:15:28.502951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:15:28.502951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:15:28.502951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:15:28.503624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:15:28.503624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:15:28.503624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:15:28.503624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:15:28.503624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:15:28.503624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:15:28.503624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:15:28.503624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:15:28.503624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:15:28.503624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:15:28.503624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:15:28.503624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:28.503624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:15:28.503624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:28.503624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:15:28.503624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:28.503624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:15:28.503624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:28.503624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:15:28.504394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:28.504394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:15:28.504394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:15:28.504394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:28.504394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:15:28.504394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:28.504394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:15:28.504394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:15:28.504394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:28.504394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:15:28.504394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:28.504394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:15:28.519174 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:15:28.519634 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:15:28.519727 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:15:28.519901 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:15:28.520575 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:15:28.521042 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:28.521793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:15:28.521793 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:28.521793 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:15:28.521793 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:15:28.521793 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:28.521793 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:15:28.521793 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:28.521793 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:15:28.522523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:28.522523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:15:28.522523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:15:28.522523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:15:28.522523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:28.522523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:15:28.522523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:28.522523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:15:28.522523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:28.522523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:15:28.522523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:28.522523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:15:28.522523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:28.522523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:15:28.522523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:15:28.522963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:28.522963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:15:28.522963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:28.522963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:15:28.522963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:15:28.522963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:28.522963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:15:28.522963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:28.522963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:15:28.523468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:28.523468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:15:28.523468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:15:28.523468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:15:28.523468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:15:28.523468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:15:28.523468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:15:28.523468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:15:28.523468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:15:28.523468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:28.523468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:15:28.523814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:28.523814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:15:28.523814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:28.523814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:15:28.523814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:28.523814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:15:28.523814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:28.523814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:15:28.523814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:15:28.523814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:28.523814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:15:28.523814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:28.523814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:15:28.523814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:15:28.524177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:28.524177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:15:28.524177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:28.524177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:15:28.524560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:28.524560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:15:28.524560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:15:28.524560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:15:28.524560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:15:28.524560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:15:28.524560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:15:28.524560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:15:28.524560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:15:28.524560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:15:28.524560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:15:28.524560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:15:28.524560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:15:28.524560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:15:28.524560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:15:28.524560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:15:28.524560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:15:28.525140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:15:28.525140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:15:28.525140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:15:28.525140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:28.525140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:15:28.525140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:28.525140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:15:28.525140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:28.525140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:15:28.525140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:28.525140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:15:28.525140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:28.525140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:15:28.525140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:15:28.525140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:28.525140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:15:28.525140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:28.525140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:15:28.525140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:15:28.525581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:28.525581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:15:28.525581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:28.525581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:15:28.528160 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:15:28.528340 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:15:28.528453 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:15:28.528595 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:15:28.528875 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:15:28.529109 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:15:28.529299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:28.529766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:15:28.529766 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:28.529766 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:15:28.529766 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:15:28.529766 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:28.529766 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:15:28.529766 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:28.529766 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:15:28.530578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:28.530578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:15:28.530578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:15:28.530578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:15:28.530578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:15:28.530578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:15:28.530578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:15:28.530578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:28.530578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:15:28.530578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:28.530578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:15:28.530578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:28.531066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:15:28.531066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:28.531066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:15:28.531066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:28.531066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:15:28.531066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:15:28.531066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:28.531066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:15:28.531066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:28.531066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:15:28.531066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:15:28.531066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:28.531066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:15:28.531536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:28.531536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:15:28.531614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:28.531614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:15:28.531614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:15:28.531614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:15:28.531614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:15:28.531614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:15:28.531614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:15:28.531614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:15:28.531614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:15:28.531614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:15:28.531614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:15:28.531614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:15:28.531614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:15:28.531614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:15:28.531614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:15:28.531614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:15:28.532189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:15:28.532189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:15:28.532189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:28.532189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:15:28.532189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:28.532189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:15:28.532189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:28.532189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:15:28.532189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:28.532189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:15:28.532189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:28.532189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:15:28.532189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:15:28.532189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:28.532189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:15:28.532189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:28.532189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:15:28.532189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:15:28.532189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:28.532854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:15:28.532854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:28.532854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:15:32.941208 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:15:32.942425 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:15:32.942676 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:15:32.944148 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:15:32.984142 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:15:32.984413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:32.985454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:15:32.985454 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:32.985454 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:15:32.985454 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:15:32.985454 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:32.985454 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:15:32.985454 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:32.985454 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:15:32.986710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.043 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:32.986710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:15:32.986710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:15:32.986710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:15:32.986710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:15:32.986710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:15:32.986710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:15:32.986710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:32.986710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:15:32.986710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:32.986710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:15:32.986710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:32.987461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:15:32.987461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:32.987461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:15:32.987461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:32.987461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:15:32.987461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:15:32.987461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:32.987461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:15:32.987461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:32.987461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:15:32.987461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:15:32.987461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:32.987461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:15:32.988155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:32.988155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:15:32.988155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:32.988155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:15:32.988155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:15:32.988155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:15:32.988155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:15:32.988155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:15:32.988155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:15:32.988155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:15:32.988155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:15:32.988155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:15:32.988155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:15:32.988155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:15:32.988155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:15:32.988963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:15:32.988963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:15:32.988963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:15:32.988963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:15:32.988963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:15:32.988963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:32.988963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:15:32.988963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:32.988963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:15:32.988963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:32.988963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:15:32.988963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:32.988963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:15:32.988963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:32.988963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:15:32.988963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:15:32.988963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:32.988963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:15:32.988963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:32.990255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:15:32.990255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:15:32.990255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:32.990255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:15:32.990255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:32.990255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:15:33.491160 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:15:33.491860 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:15:33.492140 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:15:33.493263 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:15:33.493675 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:15:33.494072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:33.495021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:15:33.495021 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:33.495021 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:15:33.495021 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:15:33.495021 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:33.495021 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:15:33.495021 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:33.495021 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:15:33.496506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:33.496506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:15:33.496506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:15:33.496506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:15:33.496506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:15:33.496506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:15:33.496506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:15:33.496506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:33.496506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:15:33.496506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:33.496506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:15:33.496506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:33.496890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:15:33.496890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:33.496890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:15:33.496890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:33.496890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:15:33.496890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:15:33.496890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:33.496890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:15:33.496890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:33.496890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:15:33.496890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:15:33.496890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:33.496890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:15:33.497269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:33.497269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:15:33.498203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:33.498203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:15:33.498203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:15:33.498203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:15:33.498203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:15:33.498203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:15:33.498203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:15:33.498203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:15:33.498203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:15:33.498203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:15:33.498203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:15:33.498203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:15:33.498203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:15:33.498203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:15:33.498203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:15:33.498203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:15:33.498653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:15:33.498653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:15:33.498653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:33.498653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:15:33.498653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:33.498653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:15:33.498653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:33.498653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:15:33.498653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:33.498653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:15:33.498653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:33.498653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:15:33.498653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:15:33.498653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:33.498653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:15:33.498653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:33.498653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:15:33.498653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:15:33.498653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:33.499163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:15:33.499163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:33.499163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:15:33.499163 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:15:33.499163 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:15:33.499163 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:15:33.499358 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:15:33.499760 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:15:33.500000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:33.500645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:15:33.500645 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:33.500645 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:15:33.500645 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:15:33.500645 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:33.500645 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:15:33.500645 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:33.500645 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:15:33.501985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:33.501985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:15:33.501985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:15:33.501985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:15:33.501985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:33.501985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:15:33.501985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:33.501985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:15:33.501985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:33.501985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:15:33.501985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:33.501985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:15:33.501985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:33.501985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:15:33.501985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:15:33.502415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:33.502415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:15:33.502415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:33.502415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:15:33.502415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:15:33.502415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:33.502415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:15:33.502415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:33.502415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:15:33.503674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:33.503674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:15:33.503674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:15:33.503674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:15:33.503674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:15:33.503674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:15:33.503674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:15:33.503674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:15:33.503674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:15:33.503674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:15:33.503674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:33.503674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:15:33.504064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:33.504064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:15:33.504064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:33.504064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:15:33.504064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:33.504064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:15:33.504064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:33.504064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:15:33.504064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:15:33.504064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:33.504064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:15:33.504064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:33.504064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:15:33.504064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:15:33.504461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:33.504461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:15:33.504461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:33.504461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:15:33.505588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:33.505588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:15:33.505588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:15:33.505588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:15:33.505588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:15:33.505588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:15:33.505588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:15:33.505588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:15:33.505588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:15:33.505588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:15:33.505588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:15:33.505588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:15:33.505588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:15:33.505588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:15:33.505588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:15:33.505588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:15:33.506048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:15:33.506048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:15:33.506048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:15:33.506048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:15:33.506048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:15:33.506048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:33.506048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:15:33.506048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:33.506048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:15:33.506048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:33.506048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:15:33.506048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:33.506048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:15:33.506048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:33.506048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:15:33.506048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:15:33.506048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:33.506048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:15:33.506540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:33.506540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:15:33.506540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:15:33.506540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:33.506540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:15:33.506540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:33.506540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:15:33.519624 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:15:33.520083 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:15:33.520293 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:15:33.520503 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:15:33.521146 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:15:33.521578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:33.522591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:15:33.522591 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:33.522591 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:15:33.522591 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:15:33.522591 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:33.522591 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:15:33.522591 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:33.522591 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:15:33.549430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:33.549430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:15:33.549430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:15:33.549430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:15:33.549430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:33.549430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:15:33.549430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:33.549430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:15:33.549430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:33.549430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:15:33.549430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:33.549430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:15:33.549430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:33.549430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:15:33.549430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:15:33.550452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:33.550452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:15:33.550452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:33.550452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:15:33.550452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:15:33.550452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:33.550452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:15:33.550452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:33.550452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:15:33.550452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:33.550452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:15:33.550452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:15:33.550452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:15:33.550452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:15:33.551002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:15:33.551002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:15:33.551002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:15:33.551002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:15:33.551002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:33.551002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:15:33.551002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:33.551002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:15:33.551002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:33.551002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:15:33.551002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:33.551002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:15:33.551002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:33.551475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:15:33.551475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:15:33.551475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:33.551475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:15:33.551475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:33.551475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:15:33.551475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:15:33.551475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:33.551475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:15:33.551475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:33.551475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:15:33.551475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:33.551966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:15:33.551966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:15:33.551966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:15:33.551966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:15:33.551966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:15:33.551966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:15:33.551966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:15:33.551966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:15:33.551966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:15:33.551966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:15:33.551966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:15:33.551966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:15:33.551966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:15:33.551966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:15:33.551966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:15:33.551966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:15:33.551966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:15:33.551966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:15:33.551966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:15:33.552560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:33.552560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:15:33.552560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:33.552560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:15:33.552560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:33.552560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:15:33.552560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:33.552560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:15:33.552560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:33.552560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:15:33.552560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:15:33.552560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:33.552560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:15:33.552560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:33.552560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:15:33.552560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:15:33.552560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:33.552560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:15:33.552560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:33.552560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:15:33.553235 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:15:33.553235 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:15:33.553235 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:15:33.553235 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:15:33.553235 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:15:33.553235 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:15:33.553450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:33.553450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:15:33.553450 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:33.553450 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:15:33.553450 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:15:33.553450 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:33.553450 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:15:33.553450 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:33.553450 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:15:33.553450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:33.553450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:15:33.553450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:15:33.553450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:15:33.553450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:15:33.553450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:15:33.553897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:15:33.553897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:33.553897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:15:33.553897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:33.553897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:15:33.553897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:33.553897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:15:33.553897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:33.553897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:15:33.553897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:33.553897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:15:33.553897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:15:33.553897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:33.553897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:15:33.554284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:33.554284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:15:33.554284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:15:33.554284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:33.554284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:15:33.554284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:33.554284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:15:33.554284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:33.554284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:15:33.554284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:15:33.554284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:15:33.554284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:15:33.554284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:15:33.554706 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:15:33.554706 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:15:33.554706 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:15:33.554706 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:15:33.554706 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:15:33.554706 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:15:33.554706 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:15:33.554706 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:15:33.554706 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:15:33.554706 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:15:33.554706 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:15:33.554706 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:15:33.554706 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:33.554706 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:15:33.554706 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:33.554706 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:15:33.554706 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:33.554706 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:15:33.554706 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:33.554706 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:15:33.555319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:33.555319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:15:33.555319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:15:33.555319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:33.555319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:15:33.555319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:33.555319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:15:33.555319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:15:33.555319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:33.555319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:15:33.555319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:33.555319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:15:37.942395 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:15:37.943149 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:15:37.943391 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:15:37.946770 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:15:37.986883 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:15:37.987086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:37.987806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:15:37.987806 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:37.987806 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:15:37.987806 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:15:37.987806 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:37.987806 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:15:37.987806 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:37.987806 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:15:37.988838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.044 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:37.988838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:15:37.988838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:15:37.988838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:15:37.988838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:15:37.988838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:15:37.988838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:15:37.988838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:37.988838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:15:37.988838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:37.988838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:15:37.988838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:37.989439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:15:37.989439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:37.989439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:15:37.989439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:37.989439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:15:37.989439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:15:37.989439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:37.989439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:15:37.989439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:37.989439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:15:37.989439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:15:37.989439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:37.989439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:15:37.989885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:37.989885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:15:37.990211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:37.990211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:15:37.990211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:15:37.990211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:15:37.990211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:15:37.990211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:15:37.990211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:15:37.990211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:15:37.990211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:15:37.990211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:15:37.990211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:15:37.990211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:15:37.990211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:15:37.990211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:15:37.990211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:15:37.990211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:15:37.990898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:15:37.990898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:15:37.990898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:37.990898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:15:37.990898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:37.990898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:15:37.990898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:37.990898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:15:37.990898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:37.990898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:15:37.990898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:37.990898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:15:37.990898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:15:37.990898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:37.990898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:15:37.990898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:37.990898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:15:37.990898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:15:37.990898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:37.991406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:15:37.991406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:37.991406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:15:38.492292 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:15:38.493159 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:15:38.495122 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:15:38.495487 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:15:38.495719 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:15:38.495931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:38.498930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:15:38.498930 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:38.498930 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:15:38.498930 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:15:38.498930 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:38.498930 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:15:38.498930 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:38.498930 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:15:38.498930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:38.498930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:15:38.498930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:15:38.498930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:15:38.498930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:15:38.498930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:15:38.498930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:15:38.499860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:38.499860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:15:38.499860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:38.499860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:15:38.499860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:38.499860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:15:38.499860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:38.499860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:15:38.499860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:38.499860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:15:38.499860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:15:38.499860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:38.499860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:15:38.499860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:38.500523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:15:38.500523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:15:38.500523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:38.500523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:15:38.500523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:38.500523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:15:38.502321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:38.502321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:15:38.502321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:15:38.502321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:15:38.502321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:15:38.502321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:15:38.502321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:15:38.502321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:15:38.502321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:15:38.502321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:15:38.502321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:15:38.502321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:15:38.502321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:15:38.502321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:15:38.502321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:15:38.502321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:15:38.503140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:15:38.503140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:15:38.503140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:38.503140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:15:38.503140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:38.503140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:15:38.503140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:38.503140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:15:38.503140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:38.503140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:15:38.503140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:38.503140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:15:38.503140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:15:38.503140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:38.503140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:15:38.503140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:38.503140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:15:38.503140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:15:38.503140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:38.503936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:15:38.503936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:38.503936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:15:38.504283 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:15:38.504922 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:15:38.504922 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:15:38.504922 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:15:38.505198 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:15:38.505396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:38.507892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:15:38.507892 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:38.507892 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:15:38.507892 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:15:38.507892 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:38.507892 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:15:38.507892 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:38.507892 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:15:38.507892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:38.507892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:15:38.507892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:15:38.507892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:15:38.507892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:38.507892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:15:38.507892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:38.507892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:15:38.507892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:38.507892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:15:38.508295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:38.508295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:15:38.508295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:38.508295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:15:38.508295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:15:38.508295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:38.508295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:15:38.508295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:38.508295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:15:38.508295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:15:38.508295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:38.508295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:15:38.508295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:38.508295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:15:38.508295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:38.508597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:15:38.508597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:15:38.508597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:15:38.508597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:15:38.508597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:15:38.508597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:15:38.508597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:15:38.508597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:15:38.508597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:15:38.508597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:38.508597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:15:38.508597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:38.508597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:15:38.508857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:38.508857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:15:38.508857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:38.508857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:15:38.508857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:38.508857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:15:38.508857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:15:38.508857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:38.508857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:15:38.508857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:38.508857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:15:38.508857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:15:38.508857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:38.508857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:15:38.509163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:38.509163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:15:38.509163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:38.509163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:15:38.509163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:15:38.509163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:15:38.509163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:15:38.509163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:15:38.509163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:15:38.509163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:15:38.509163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:15:38.509163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:15:38.509163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:15:38.509163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:15:38.509163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:15:38.509453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:15:38.509453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:15:38.509453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:15:38.509453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:15:38.509453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:15:38.509453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:15:38.509453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:15:38.509453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:15:38.509453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:38.509453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:15:38.509453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:38.509453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:15:38.509453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:38.509453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:15:38.509453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:38.509453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:15:38.509453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:38.509453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:15:38.509453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:15:38.509807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:38.509807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:15:38.509807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:38.509807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:15:38.509807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:15:38.509807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:38.509807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:15:38.509807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:38.509807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:15:38.520048 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:15:38.520811 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:15:38.520811 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:15:38.520811 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:15:38.521138 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:15:38.521447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:38.524789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:15:38.524789 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:38.524789 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:15:38.524789 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:15:38.524789 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:38.524789 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:15:38.524789 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:38.524789 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:15:38.524789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:38.524789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:15:38.524789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:15:38.524789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:15:38.524789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:38.524789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:15:38.524789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:38.524789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:15:38.524789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:38.524789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:15:38.525669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:38.525669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:15:38.525669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:38.525669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:15:38.525669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:15:38.525669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:38.525669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:15:38.525669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:38.525669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:15:38.525669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:15:38.525669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:38.525669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:15:38.525669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:38.525669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:15:38.527163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:38.527163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:15:38.527163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:15:38.527163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:15:38.527163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:15:38.527163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:15:38.527163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:15:38.527163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:15:38.527163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:15:38.527163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:38.527163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:15:38.527793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:38.527793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:15:38.527793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:38.527793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:15:38.527793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:38.527793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:15:38.527793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:38.527793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:15:38.527793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:15:38.527793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:38.527793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:15:38.527793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:38.527793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:15:38.527793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:15:38.528477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:38.528477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:15:38.528477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:38.528477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:15:38.529244 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:15:38.531018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:38.531018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:15:38.531018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:15:38.531018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:15:38.531018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:15:38.531018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:15:38.531018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:15:38.531018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:15:38.531018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:15:38.531018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:15:38.531018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:15:38.531018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:15:38.531018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:15:38.531018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:15:38.531018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:15:38.531018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:15:38.531018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:15:38.531390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:15:38.531390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:15:38.531390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:15:38.531390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:38.531390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:15:38.531390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:38.531390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:15:38.531390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:38.531390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:15:38.531390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:38.531390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:15:38.531390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:38.531390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:15:38.531390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:15:38.531390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:38.531390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:15:38.531390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:38.531390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:15:38.531390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:15:38.532123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:38.532123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:15:38.532123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:38.532123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:15:38.532123 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:15:38.532123 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:15:38.532123 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:15:38.532123 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:15:38.532413 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:15:38.532413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:38.534507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:15:38.534507 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:38.534507 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:15:38.534507 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:15:38.534507 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:38.534507 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:15:38.534507 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:38.534507 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:15:38.534507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:38.534507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:15:38.534507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:15:38.534507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:15:38.534507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:15:38.534507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:15:38.534507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:15:38.535029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:38.535029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:15:38.535029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:38.535029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:15:38.535029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:38.535029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:15:38.535029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:38.535029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:15:38.535029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:38.535029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:15:38.535029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:15:38.535029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:38.535029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:15:38.535029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:38.535281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:15:38.535281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:15:38.535281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:38.535281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:15:38.535281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:38.535281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:15:38.535281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:38.535281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:15:38.535281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:15:38.535281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:15:38.535281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:15:38.535281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:15:38.535281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:15:38.535281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:15:38.535619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:15:38.535619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:15:38.535619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:15:38.535619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:15:38.535619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:15:38.535619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:15:38.535619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:15:38.535619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:15:38.535619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:15:38.535619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:15:38.535619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:38.535619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:15:38.535619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:38.535619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:15:38.535619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:38.535619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:15:38.535619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:38.535619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:15:38.535619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:38.535619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:15:38.535995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:15:38.535995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:38.535995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:15:38.535995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:38.535995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:15:38.535995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:15:38.535995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:38.535995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:15:38.535995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:38.535995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:15:42.943178 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:15:42.944112 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:15:42.944112 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:15:42.944855 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:15:42.979085 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:15:42.979429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:42.980230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:15:42.980230 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:42.980230 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:15:42.980230 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:15:42.980230 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:42.980230 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:15:42.980230 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:42.980230 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:15:42.981272 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:42.981272 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:15:42.981272 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:15:42.981272 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:15:42.981272 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:15:42.981272 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:15:42.981272 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:15:42.981272 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:42.981272 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:15:42.981272 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:42.981272 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:15:42.981272 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:42.982189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:15:42.982189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:42.982189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:15:42.982189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:42.982189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:15:42.982189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:15:42.982189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:42.982189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:15:42.982189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:42.982189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:15:42.982189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:15:42.982189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:42.982189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:15:42.982930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:42.982930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:15:42.982930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:42.982930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:15:42.982930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:15:42.982930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:15:42.982930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:15:42.982930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:15:42.982930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:15:42.982930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:15:42.982930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:15:42.982930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:15:42.982930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:15:42.982930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:15:42.982930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:15:42.983669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:15:42.983669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:15:42.983669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:15:42.983669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:15:42.983669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:15:42.983669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:42.983669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:15:42.983669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:42.983669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:15:42.983669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:42.983669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:15:42.983669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:42.983669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:15:42.983669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:42.983669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:15:42.983669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:15:42.983669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:42.983669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:15:42.983669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:42.984669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:15:42.984669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:15:42.984669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:42.984669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:15:42.984669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:42.984669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:15:43.494144 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:15:43.494838 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:15:43.494971 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:15:43.497483 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:15:43.497885 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:15:43.498341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:43.499469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:15:43.499469 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:43.499469 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:15:43.499469 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:15:43.499469 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:43.499469 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:15:43.499469 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:43.499469 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:15:43.500906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:43.500906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:15:43.500906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:15:43.500906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:15:43.500906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:15:43.500906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:15:43.500906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:15:43.500906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:43.500906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:15:43.500906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:43.500906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:15:43.500906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:43.501517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:15:43.501517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:43.501517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:15:43.501517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:43.501517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:15:43.501517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:15:43.501517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:43.501517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:15:43.501517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:43.501517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:15:43.501517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:15:43.501517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:43.501517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:15:43.502235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:43.502235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:15:43.502404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:43.502404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:15:43.502404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:15:43.502404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:15:43.502404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:15:43.502404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:15:43.502404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:15:43.502404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:15:43.502404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:15:43.502404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:15:43.502404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:15:43.502404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:15:43.502404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:15:43.502404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:15:43.502404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:15:43.502404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:15:43.503167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:15:43.503167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:15:43.503167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:43.503167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:15:43.503167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:43.503167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:15:43.503167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:43.503167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:15:43.503167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:43.503167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:15:43.503167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:43.503167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:15:43.503167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:15:43.503167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:43.503167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:15:43.503167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:43.503167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:15:43.503167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:15:43.503167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:43.504457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:15:43.504457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:43.504457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:15:43.504769 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:15:43.506116 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:15:43.506197 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:15:43.506296 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:15:43.508637 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:15:43.508736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:43.509718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:15:43.509718 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:43.509718 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:15:43.509718 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:15:43.509718 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:43.509718 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:15:43.509718 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:43.509718 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:15:43.510840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:43.510840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:15:43.510840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:15:43.510840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:15:43.510840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:43.510840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:15:43.510840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:43.510840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:15:43.510840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:43.510840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:15:43.510840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:43.510840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:15:43.510840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:43.510840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:15:43.510840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:15:43.511297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:43.511297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:15:43.511297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:43.511297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:15:43.511297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:15:43.511297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:43.511297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:15:43.511297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:43.511297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:15:43.511850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:43.511850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:15:43.511850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:15:43.511850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:15:43.511850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:15:43.511850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:15:43.511850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:15:43.511850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:15:43.511850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:15:43.511850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:15:43.511850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:43.511850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:15:43.512258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:43.512258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:15:43.512258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:43.512258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:15:43.512258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:43.512258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:15:43.512258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:43.512258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:15:43.512258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:15:43.512258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:43.512258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:15:43.512258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:43.512258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:15:43.512258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:15:43.512628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:43.512628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:15:43.512628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:43.512628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:15:43.513287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:43.513287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:15:43.513287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:15:43.513287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:15:43.513287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:15:43.513287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:15:43.513287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:15:43.513287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:15:43.513287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:15:43.513287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:15:43.513287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:15:43.513287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:15:43.513287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:15:43.513287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:15:43.513287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:15:43.513287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:15:43.514603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:15:43.514603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:15:43.514603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:15:43.514603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:15:43.514603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:15:43.514603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:43.514603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:15:43.514603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:43.514603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:15:43.514603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:43.514603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:15:43.514603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:43.514603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:15:43.514603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:43.514603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:15:43.514603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:15:43.514603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:43.514603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:15:43.515121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:43.515121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:15:43.515121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:15:43.515121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:43.515121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:15:43.515121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:43.515121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:15:43.520491 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:15:43.520701 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:15:43.520846 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:15:43.521105 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:15:43.521630 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:15:43.521941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:43.522531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:15:43.522531 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:43.522531 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:15:43.522531 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:15:43.522531 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:43.522531 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:15:43.522531 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:43.522531 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:15:43.523330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:43.523330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:15:43.523330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:15:43.523330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:15:43.523330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:43.523330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:15:43.523330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:43.523330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:15:43.523330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:43.523330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:15:43.523330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:43.523330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:15:43.523330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:43.523330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:15:43.523330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:15:43.523765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:43.523765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:15:43.523765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:43.523765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:15:43.523765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:15:43.523765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:43.523765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:15:43.523765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:43.523765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:15:43.524247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:43.524247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:15:43.524247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:15:43.524247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:15:43.524247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:15:43.524247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:15:43.524247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:15:43.524247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:15:43.524247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:15:43.524247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:43.524247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:15:43.524584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:43.524584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:15:43.524584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:43.524584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:15:43.524584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:43.524584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:15:43.524584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:43.524584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:15:43.524584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:15:43.524584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:43.524584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:15:43.524584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:43.524584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:15:43.524584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:15:43.524990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:43.524990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:15:43.524990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:43.524990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:15:43.525365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:43.525365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:15:43.525365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:15:43.525365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:15:43.525365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:15:43.525365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:15:43.525365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:15:43.525365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:15:43.525365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:15:43.525365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:15:43.525365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:15:43.525365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:15:43.525365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:15:43.525365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:15:43.525365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:15:43.525365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:15:43.525365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:15:43.525939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:15:43.525939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:15:43.525939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:15:43.525939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:43.525939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:15:43.525939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:43.525939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:15:43.525939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:43.525939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:15:43.525939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:43.525939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:15:43.525939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:43.525939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:15:43.525939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:15:43.525939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:43.525939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:15:43.525939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:43.525939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:15:43.525939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:15:43.526453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:43.526453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:15:43.526453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:43.526453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:15:43.530868 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:15:43.531060 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:15:43.531170 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:15:43.531319 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:15:43.531941 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:15:43.532231 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:15:43.532410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:43.533072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:15:43.533072 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:43.533072 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:15:43.533072 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:15:43.533072 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:43.533072 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:15:43.533072 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:43.533072 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:15:43.533905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:43.533905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:15:43.533905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:15:43.533905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:15:43.533905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:15:43.533905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:15:43.533905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:15:43.533905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:43.533905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:15:43.533905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:43.533905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:15:43.533905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:43.534266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:15:43.534266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:43.534266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:15:43.534266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:43.534266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:15:43.534266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:15:43.534266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:43.534266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:15:43.534266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:43.534266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:15:43.534266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:15:43.534266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:43.534266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:15:43.534668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:43.534668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:15:43.534886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:43.534886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:15:43.534886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:15:43.534886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:15:43.534886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:15:43.534886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:15:43.534886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:15:43.534886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:15:43.534886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:15:43.534886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:15:43.534886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:15:43.534886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:15:43.534886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:15:43.534886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:15:43.534886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:15:43.534886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:15:43.535404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:15:43.535404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:15:43.535404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:43.535404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:15:43.535404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:43.535404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:15:43.535404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:43.535404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:15:43.535404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:43.535404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:15:43.535404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:43.535404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:15:43.535404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:15:43.535404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:43.535404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:15:43.535404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:43.535404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:15:43.535404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:15:43.535404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:43.536024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:15:43.536024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:43.536024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:15:47.944063 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:15:47.944807 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:15:47.944807 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:15:47.945973 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:15:47.948370 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG dbcounter [-] [106314] Writing DB stats neutron:SELECT=471,neutron:UPDATE=2 {{(pid=106314) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 09 09:15:48.216230 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:15:48.216522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:48.227401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:15:48.227401 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:48.227401 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:15:48.227401 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:15:48.227401 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:48.227401 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:15:48.227401 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:48.227401 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:15:48.227401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.273 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:48.227401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:15:48.227401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:15:48.227401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:15:48.227401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:15:48.227401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:15:48.227401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:15:48.227964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:48.227964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:15:48.227964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:48.227964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:15:48.227964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:48.227964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:15:48.227964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:48.227964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:15:48.227964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:48.227964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:15:48.227964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:15:48.227964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:48.227964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:15:48.227964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:48.228350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:15:48.228350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:15:48.228350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:48.228350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:15:48.228350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:48.228350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:15:48.228350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:48.228350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:15:48.228350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:15:48.228350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:15:48.228350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:15:48.228350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:15:48.228350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:15:48.228350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:15:48.228874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:15:48.228874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:15:48.228874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:15:48.228874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:15:48.228874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:15:48.228874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:15:48.228874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:15:48.228874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:15:48.228874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:15:48.228874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:15:48.228874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:48.228874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:15:48.228874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:48.228874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:15:48.228874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:48.228874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:15:48.228874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:48.228874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:15:48.228874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:48.228874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:15:48.228874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:15:48.229509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:48.229509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:15:48.229509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:48.229509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:15:48.229509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:15:48.229509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:48.229509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:15:48.229509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:48.229509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:15:48.494819 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:15:48.495310 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:15:48.495507 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:15:48.496703 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:15:48.497115 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:15:48.497452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:48.498544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:15:48.498544 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:48.498544 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:15:48.498544 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:15:48.498544 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:48.498544 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:15:48.498544 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:48.498544 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:15:48.500229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:48.500229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:15:48.500229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:15:48.500229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:15:48.500229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:15:48.500229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:15:48.500229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:15:48.500229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:48.500229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:15:48.500229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:48.500229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:15:48.500229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:48.500736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:15:48.500736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:48.500736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:15:48.500736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:48.500736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:15:48.500736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:15:48.500736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:48.500736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:15:48.500736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:48.500736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:15:48.500736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:15:48.500736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:48.500736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:15:48.501224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:48.501224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:15:48.502163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:48.502163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:15:48.502163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:15:48.502163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:15:48.502163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:15:48.502163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:15:48.502163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:15:48.502163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:15:48.502163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:15:48.502163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:15:48.502163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:15:48.502163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:15:48.502163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:15:48.502163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:15:48.502163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:15:48.502163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:15:48.502816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:15:48.502816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:15:48.502816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:48.502816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:15:48.502816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:48.502816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:15:48.502816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:48.502816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:15:48.502816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:48.502816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:15:48.502816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:48.502816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:15:48.502816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:15:48.502816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:48.502816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:15:48.502816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:48.502816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:15:48.502816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:15:48.502816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:48.503614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:15:48.503614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:48.503614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:15:48.506846 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:15:48.506846 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:15:48.506846 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:15:48.506846 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:15:48.507614 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:15:48.507931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:48.510372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:15:48.510372 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:48.510372 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:15:48.510372 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:15:48.510372 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:48.510372 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:15:48.510372 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:48.510372 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:15:48.511216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:48.511216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:15:48.511216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:15:48.511216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:15:48.511216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:48.511216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:15:48.511216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:48.511216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:15:48.511216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:48.511216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:15:48.511216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:48.511216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:15:48.511216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:48.511216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:15:48.511216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:15:48.511777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:48.511777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:15:48.511777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:48.511777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:15:48.511777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:15:48.511777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:48.511777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:15:48.511777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:48.511777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:15:48.512120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:48.512120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:15:48.512120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:15:48.512120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:15:48.512120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:15:48.512120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:15:48.512120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:15:48.512120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:15:48.512120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:15:48.512120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:15:48.512120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:48.512120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:15:48.512611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:48.512611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:15:48.512611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:48.512611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:15:48.512611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:48.512611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:15:48.512611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:48.512611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:15:48.512611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:15:48.512611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:48.512611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:15:48.512611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:48.512611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:15:48.512611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:15:48.513127 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:48.513127 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:15:48.513127 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:48.513127 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:15:48.515730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:48.515730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:15:48.515730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:15:48.515730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:15:48.515730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:15:48.515730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:15:48.515730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:15:48.515730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:15:48.515730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:15:48.515730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:15:48.515730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:15:48.515730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:15:48.515730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:15:48.515730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:15:48.515730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:15:48.515730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:15:48.516360 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:15:48.516360 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:15:48.516360 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:15:48.516360 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:15:48.516360 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:15:48.516360 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:48.516360 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:15:48.516360 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:48.516360 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:15:48.516360 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:48.516360 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:15:48.516360 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:48.516360 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:15:48.516360 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:48.516360 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:15:48.516360 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:15:48.516360 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:48.516360 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:15:48.517052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:48.517052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:15:48.517052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:15:48.517052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:48.517052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:15:48.517052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:48.517052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:15:48.520910 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:15:48.521222 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:15:48.521433 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:15:48.521535 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:15:48.524280 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:15:48.524683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:48.526090 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:15:48.526090 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:48.526090 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:15:48.526090 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:15:48.526090 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:48.526090 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:15:48.526090 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:48.526090 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:15:48.526979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:48.526979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:15:48.526979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:15:48.526979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:15:48.526979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:48.526979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:15:48.526979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:48.526979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:15:48.526979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:48.526979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:15:48.526979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:48.526979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:15:48.526979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:48.526979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:15:48.526979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:15:48.527474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:48.527474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:15:48.527474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:48.527474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:15:48.527474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:15:48.527474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:48.527474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:15:48.527474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:48.527474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:15:48.528517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.006 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:48.528517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:15:48.528517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:15:48.528517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:15:48.528517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:15:48.528517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:15:48.528517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:15:48.528517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:15:48.528517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:15:48.528517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:48.528517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:15:48.529048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:48.529048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:15:48.529048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:48.529048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:15:48.529048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:48.529048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:15:48.529048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:48.529048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:15:48.529048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:15:48.529048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:48.529048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:15:48.529048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:48.529048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:15:48.529048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:15:48.529401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:48.529401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:15:48.529401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:48.529401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:15:48.530085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:48.530085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:15:48.530085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:15:48.530085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:15:48.530085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:15:48.530085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:15:48.530085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:15:48.530085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:15:48.530085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:15:48.530085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:15:48.530085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:15:48.530085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:15:48.530085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:15:48.530085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:15:48.530085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:15:48.530085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:15:48.530085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:15:48.531035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:15:48.531035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:15:48.531035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:15:48.531035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:48.531035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:15:48.531035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:48.531035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:15:48.531035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:48.531035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:15:48.531035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:48.531035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:15:48.531035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:48.531035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:15:48.531035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:15:48.531035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:48.531035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:15:48.531035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:48.531035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:15:48.531035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:15:48.532008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:48.532008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:15:48.532008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:48.532008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:15:48.532008 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:15:48.532008 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:15:48.532008 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:15:48.532008 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:15:48.532620 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:15:48.532941 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:15:48.533790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:48.534341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:15:48.534341 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:48.534341 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:15:48.534341 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:15:48.534341 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:48.534341 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:15:48.534341 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:48.534341 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:15:48.535339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:48.535339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:15:48.535339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:15:48.535339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:15:48.535339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:15:48.535339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:15:48.535339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:15:48.535339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:48.535339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:15:48.535339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:48.535339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:15:48.535339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:48.536110 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:15:48.536110 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:48.536110 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:15:48.536110 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:48.536110 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:15:48.536110 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:15:48.536110 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:48.536110 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:15:48.536110 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:48.536110 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:15:48.536110 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:15:48.536110 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:48.536110 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:15:48.536685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:48.536685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:15:48.536685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:48.536685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:15:48.536685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:15:48.536685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:15:48.536685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:15:48.536685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:15:48.536685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:15:48.536685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:15:48.536685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:15:48.536685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:15:48.536685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:15:48.536685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:15:48.536685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:15:48.537413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:15:48.537413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:15:48.537413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:15:48.537413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:15:48.537413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:15:48.537413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:48.537413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:15:48.537413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:48.537413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:15:48.537413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:48.537413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:15:48.537413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:48.537413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:15:48.537413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:48.537413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:15:48.537413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:15:48.537413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:48.537413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:15:48.537413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:48.538304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:15:48.538304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:15:48.538304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:48.538304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:15:48.538304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:48.538304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:15:52.944662 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:15:52.945305 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:15:52.945305 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:15:52.946021 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:15:52.981796 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:15:52.981796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:52.982305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:15:52.982305 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:52.982305 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:15:52.982305 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:15:52.982305 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:52.982305 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:15:52.982305 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:52.982305 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:15:52.983121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:52.983121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:15:52.983121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:15:52.983121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:15:52.983121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:15:52.983121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:15:52.983121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:15:52.983121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:52.983121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:15:52.983121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:52.983121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:15:52.983121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:52.984080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:15:52.984080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:52.984080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:15:52.984080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:52.984080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:15:52.984080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:15:52.984080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:52.984080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:15:52.984080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:52.984080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:15:52.984080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:15:52.984080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:52.984080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:15:52.984857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:52.984857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:15:52.984857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:52.984857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:15:52.984857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:15:52.984857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:15:52.984857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:15:52.984857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:15:52.984857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:15:52.984857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:15:52.984857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:15:52.984857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:15:52.984857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:15:52.984857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:15:52.984857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:15:52.985737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:15:52.985737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:15:52.985737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:15:52.985737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:15:52.985737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:15:52.985737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:52.985737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:15:52.985737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:52.985737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:15:52.985737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:52.985737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:15:52.985737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:52.985737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:15:52.985737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:52.985737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:15:52.985737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:15:52.985737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:52.985737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:15:52.985737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:52.986997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:15:52.986997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:15:52.986997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:52.986997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:15:52.986997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:52.986997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:15:53.503124 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:15:53.503124 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:15:53.503124 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:15:53.503124 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:15:53.503124 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:15:53.503124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:53.505187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:15:53.505187 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:53.505187 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:15:53.505187 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:15:53.505187 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:53.505187 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:15:53.505187 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:53.505187 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:15:53.505187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:53.505187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:15:53.505187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:15:53.505187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:15:53.505187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:15:53.505187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:15:53.505187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:15:53.506429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:53.506429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:15:53.506429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:53.506429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:15:53.506429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:53.506429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:15:53.506429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:53.506429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:15:53.506429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:53.506429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:15:53.506429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:15:53.506429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:53.506429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:15:53.506429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:53.507268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:15:53.507268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:15:53.507268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:53.507268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:15:53.507268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:53.507268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:15:53.507268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:53.507268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:15:53.507268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:15:53.507268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:15:53.507268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:15:53.507268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:15:53.507268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:15:53.507268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:15:53.508157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:15:53.508157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:15:53.508157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:15:53.508157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:15:53.508157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:15:53.508157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:15:53.508157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:15:53.508157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:15:53.508157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:15:53.508157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:15:53.508157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:53.508157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:15:53.508157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:53.508157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:15:53.508157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:53.508157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:15:53.508157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:53.508157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:15:53.508157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:53.508157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:15:53.508861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:15:53.508861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:53.508861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:15:53.508861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:53.508861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:15:53.508861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:15:53.508861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:53.508861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:15:53.508861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:53.508861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:15:53.508861 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:15:53.508861 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:15:53.509282 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:15:53.509282 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:15:53.509282 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:15:53.509282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:53.509282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:15:53.509282 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:53.509282 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:15:53.509282 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:15:53.509282 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:53.509282 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:15:53.509282 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:53.509282 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:15:53.509698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:53.509698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:15:53.509698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:15:53.509698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:15:53.509698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:53.509698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:15:53.509698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:53.509698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:15:53.509698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:53.509698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:15:53.509698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:53.509698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:15:53.509698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:53.509698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:15:53.509698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:15:53.510308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:53.510308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:15:53.510308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:53.510308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:15:53.510308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:15:53.510308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:53.510308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:15:53.510308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:53.510308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:15:53.510308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:53.510308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:15:53.510308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:15:53.510308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:15:53.510308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:15:53.510805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:15:53.510805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:15:53.510805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:15:53.510805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:15:53.510805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:15:53.510805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:53.510805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:15:53.510805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:53.510805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:15:53.510805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:53.510805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:15:53.510805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:53.510805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:15:53.511294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:53.511294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:15:53.511294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:15:53.511294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:53.511294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:15:53.511294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:53.511294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:15:53.511294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:15:53.511294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:53.511294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:15:53.511294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:53.511294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:15:53.511819 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:53.511819 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:15:53.511819 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:15:53.511819 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:15:53.511819 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:15:53.511819 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:15:53.511819 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:15:53.511819 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:15:53.511819 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:15:53.511819 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:15:53.511819 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:15:53.511819 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:15:53.511819 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:15:53.511819 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:15:53.511819 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:15:53.511819 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:15:53.512467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:15:53.512467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:15:53.512467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:15:53.512467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:15:53.512467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:15:53.512467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:53.512467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:15:53.512467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:53.512467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:15:53.512467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:53.512467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:15:53.512467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:53.512467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:15:53.512467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:53.512467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:15:53.512467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:15:53.512467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:53.512467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:15:53.513149 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:53.513149 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:15:53.513149 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:15:53.513149 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:53.513149 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:15:53.513149 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:53.513149 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:15:53.521898 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:15:53.522328 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:15:53.522470 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:15:53.522625 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:15:53.526117 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:15:53.526286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:53.526843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:15:53.526843 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:53.526843 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:15:53.526843 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:15:53.526843 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:53.526843 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:15:53.526843 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:53.526843 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:15:53.528344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:53.528344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:15:53.528344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:15:53.528344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:15:53.528344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:53.528344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:15:53.528344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:53.528344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:15:53.528344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:53.528344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:15:53.528344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:53.528344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:15:53.528344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:53.528344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:15:53.528344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:15:53.529132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:53.529132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:15:53.529132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:53.529132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:15:53.529132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:15:53.529132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:53.529132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:15:53.529132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:53.529132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:15:53.530002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.006 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:53.530002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:15:53.530002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:15:53.530002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:15:53.530002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:15:53.530002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:15:53.530002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:15:53.530002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:15:53.530002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:15:53.530002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:53.530002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:15:53.530926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:53.530926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:15:53.530926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:53.530926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:15:53.530926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:53.530926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:15:53.530926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:53.530926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:15:53.530926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:15:53.530926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:53.530926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:15:53.530926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:53.530926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:15:53.530926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:15:53.532782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:53.532782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:15:53.532782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:53.532782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:15:53.534403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:53.534403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:15:53.534403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:15:53.534403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:15:53.534403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:15:53.534403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:15:53.534403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:15:53.534403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:15:53.534403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:15:53.534403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:15:53.534403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:15:53.534403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:15:53.534403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:15:53.534403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:15:53.534403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:15:53.534403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:15:53.534403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:15:53.535315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:15:53.535315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:15:53.535315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:15:53.535315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:53.535315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:15:53.535315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:53.535315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:15:53.535315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:53.535315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:15:53.535315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:53.535315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:15:53.535315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:53.535315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:15:53.535315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:15:53.535315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:53.535315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:15:53.535315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:53.535315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:15:53.535315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:15:53.536160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:53.536160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:15:53.536160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:53.536160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:15:53.536160 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:15:53.537653 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:15:53.537772 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:15:53.537966 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:15:53.540963 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:15:53.541118 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:15:53.541227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:53.541710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:15:53.541710 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:53.541710 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:15:53.541710 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:15:53.541710 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:53.541710 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:15:53.541710 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:53.541710 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:15:53.542642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:53.542642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:15:53.542642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:15:53.542642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:15:53.542642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:15:53.542642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:15:53.542642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:15:53.542642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:53.542642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:15:53.542642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:53.542642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:15:53.542642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:53.543056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:15:53.543056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:53.543056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:15:53.543056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:53.543056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:15:53.543056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:15:53.543056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:53.543056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:15:53.543056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:53.543056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:15:53.543056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:15:53.543056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:53.543056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:15:53.543393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:53.543393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:15:53.543662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:53.543662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:15:53.543662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:15:53.543662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:15:53.543662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:15:53.543662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:15:53.543662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:15:53.543662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:15:53.543662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:15:53.543662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:15:53.543662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:15:53.543662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:15:53.543662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:15:53.543662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:15:53.543662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:15:53.543662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:15:53.544301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:15:53.544301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:15:53.544301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:53.544301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:15:53.544301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:53.544301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:15:53.544301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:53.544301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:15:53.544301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:53.544301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:15:53.544301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:53.544301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:15:53.544301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:15:53.544301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:53.544301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:15:53.544301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:53.544301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:15:53.544301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:15:53.544301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:53.544954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:15:53.544954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:53.544954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:15:57.945336 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:15:57.946007 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:15:57.946183 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:15:57.946859 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:15:57.996695 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:15:57.996937 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:57.997575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:15:57.997575 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:57.997575 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:15:57.997575 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:15:57.997575 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:57.997575 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:15:57.997575 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:57.997575 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:15:57.998474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.051 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:57.998474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:15:57.998474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:15:57.998474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:15:57.998474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:15:57.998474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:15:57.998474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:15:57.998474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:57.998474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:15:57.998474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:57.998474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:15:57.998474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:57.998826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:15:57.998826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:57.998826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:15:57.998826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:57.998826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:15:57.998826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:15:57.998826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:57.998826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:15:57.998826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:57.998826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:15:57.998826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:15:57.998826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:57.998826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:15:57.999135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:57.999135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:15:57.999365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:57.999365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:15:57.999365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:15:57.999365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:15:57.999365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:15:57.999365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:15:57.999365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:15:57.999365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:15:57.999365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:15:57.999365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:15:57.999365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:15:57.999365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:15:57.999365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:15:57.999365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:15:57.999365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:15:57.999365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:15:57.999814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:15:57.999814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:15:57.999814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:57.999814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:15:57.999814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:57.999814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:15:57.999814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:57.999814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:15:57.999814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:57.999814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:15:57.999814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:57.999814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:15:57.999814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:15:57.999814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:57.999814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:15:57.999814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:57.999814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:15:57.999814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:15:57.999814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:58.000343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:15:58.000343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:58.000343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:15:58.496070 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:15:58.496504 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:15:58.496685 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:15:58.500008 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:15:58.500137 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:15:58.500269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:58.501025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:15:58.501025 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:58.501025 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:15:58.501025 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:15:58.501025 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:58.501025 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:15:58.501025 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:58.501025 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:15:58.502390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:58.502390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:15:58.502390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:15:58.502390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:15:58.502390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:15:58.502390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:15:58.502390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:15:58.502390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:58.502390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:15:58.502390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:58.502390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:15:58.502390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:58.503002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:15:58.503002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:58.503002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:15:58.503002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:58.503002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:15:58.503002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:15:58.503002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:58.503002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:15:58.503002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:58.503002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:15:58.503002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:15:58.503002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:58.503002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:15:58.503484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:58.503484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:15:58.504134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:58.504134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:15:58.504134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:15:58.504134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:15:58.504134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:15:58.504134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:15:58.504134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:15:58.504134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:15:58.504134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:15:58.504134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:15:58.504134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:15:58.504134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:15:58.504134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:15:58.504134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:15:58.504134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:15:58.504134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:15:58.504861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:15:58.504861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:15:58.504861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:58.504861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:15:58.504861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:58.504861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:15:58.504861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:58.504861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:15:58.504861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:58.504861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:15:58.504861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:58.504861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:15:58.504861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:15:58.504861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:58.504861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:15:58.504861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:58.504861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:15:58.504861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:15:58.504861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:58.505648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:15:58.505648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:58.505648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:15:58.506935 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:15:58.507141 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:15:58.507256 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:15:58.507387 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:15:58.507668 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:15:58.507812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:58.508186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:15:58.508186 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:58.508186 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:15:58.508186 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:15:58.508186 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:58.508186 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:15:58.508186 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:58.508186 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:15:58.508970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:58.508970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:15:58.508970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:15:58.508970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:15:58.508970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:58.508970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:15:58.508970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:58.508970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:15:58.508970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:58.508970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:15:58.508970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:58.508970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:15:58.508970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:58.508970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:15:58.508970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:15:58.509666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:58.509666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:15:58.509666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:58.509666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:15:58.509666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:15:58.509666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:58.509666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:15:58.509666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:58.509666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:15:58.509988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:58.509988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:15:58.509988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:15:58.509988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:15:58.509988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:15:58.509988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:15:58.509988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:15:58.509988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:15:58.509988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:15:58.509988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:15:58.509988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:58.509988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:15:58.510372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:58.510372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:15:58.510372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:58.510372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:15:58.510372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:58.510372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:15:58.510372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:58.510372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:15:58.510372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:15:58.510372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:58.510372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:15:58.510372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:58.510372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:15:58.510372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:15:58.510836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:58.510836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:15:58.510836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:58.510836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:15:58.511090 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:58.511090 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:15:58.511090 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:15:58.511090 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:15:58.511090 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:15:58.511090 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:15:58.511090 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:15:58.511090 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:15:58.511090 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:15:58.511090 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:15:58.511090 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:15:58.511090 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:15:58.511090 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:15:58.511090 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:15:58.511090 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:15:58.511090 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:15:58.511591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:15:58.511591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:15:58.511591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:15:58.511591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:15:58.511591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:15:58.511591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:58.511591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:15:58.511591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:58.511591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:15:58.511591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:58.511591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:15:58.511591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:58.511591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:15:58.511591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:58.511591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:15:58.511591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:15:58.511591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:58.511591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:15:58.512186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:58.512186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:15:58.512186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:15:58.512186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:58.512186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:15:58.512186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:58.512186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:15:58.522462 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:15:58.522840 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:15:58.522966 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:15:58.523106 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:15:58.523431 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:15:58.523630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:58.524158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:15:58.524158 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:58.524158 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:15:58.524158 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:15:58.524158 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:58.524158 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:15:58.524158 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:58.524158 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:15:58.525849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:58.525849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:15:58.525849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:15:58.525849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:15:58.525849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:58.525849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:15:58.525849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:58.525849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:15:58.525849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:58.525849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:15:58.525849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:58.525849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:15:58.525849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:58.525849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:15:58.525849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:15:58.526204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:58.526204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:15:58.526204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:58.526204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:15:58.526204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:15:58.526204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:58.526204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:15:58.526204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:58.526204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:15:58.527856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:58.527856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:15:58.527856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:15:58.527856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:15:58.527856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:15:58.527856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:15:58.527856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:15:58.527856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:15:58.527856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:15:58.527856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:58.527856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:15:58.528384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:58.528384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:15:58.528384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:58.528384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:15:58.528384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:58.528384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:15:58.528384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:58.528384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:15:58.528384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:15:58.528384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:58.528384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:15:58.528384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:58.528384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:15:58.528384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:15:58.529037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:58.529037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:15:58.529037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:58.529037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:15:58.530317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:58.530317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:15:58.530317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:15:58.530317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:15:58.530317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:15:58.530317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:15:58.530317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:15:58.530317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:15:58.530317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:15:58.530317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:15:58.530317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:15:58.530317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:15:58.530317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:15:58.530317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:15:58.530317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:15:58.530317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:15:58.530317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:15:58.531130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:15:58.531130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:15:58.531130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:15:58.531130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:58.531130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:15:58.531130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:58.531130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:15:58.531130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:58.531130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:15:58.531130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:58.531130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:15:58.531130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:58.531130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:15:58.531130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:15:58.531130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:58.531130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:15:58.531130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:58.531130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:15:58.531130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:15:58.531897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:58.531897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:15:58.531897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:58.531897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:15:58.538000 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:15:58.538000 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:15:58.538120 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:15:58.538280 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:15:58.538622 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:15:58.538906 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:15:58.539068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:58.539650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:15:58.539650 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:58.539650 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:15:58.539650 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:15:58.539650 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:58.539650 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:15:58.539650 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:58.539650 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:15:58.540503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:58.540503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:15:58.540503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:15:58.540503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:15:58.540503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:15:58.540503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:15:58.540503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:15:58.540503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:58.540503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:15:58.540503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:58.540503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:15:58.540503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:58.540792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:15:58.540792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:58.540792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:15:58.540792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:58.540792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:15:58.540792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:15:58.540792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:58.540792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:15:58.540792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:58.540792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:15:58.540792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:15:58.540792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:58.540792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:15:58.541056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:58.541056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:15:58.541589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:58.541589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:15:58.541589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:15:58.541589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:15:58.541589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:15:58.541589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:15:58.541589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:15:58.541589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:15:58.541589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:15:58.541589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:15:58.541589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:15:58.541589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:15:58.541589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:15:58.541589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:15:58.541589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:15:58.541589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:15:58.542115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:15:58.542115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:15:58.542115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:58.542115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:15:58.542115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:15:58.542115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:15:58.542115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:15:58.542115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:15:58.542115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:15:58.542115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:15:58.542115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:15:58.542115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:15:58.542115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:15:58.542115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:15:58.542115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:15:58.542115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:15:58.542115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:15:58.542115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:15:58.542115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:15:58.542563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:15:58.542563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:15:58.542563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:16:02.946410 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:16:02.947158 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:16:02.947158 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:16:02.948360 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:16:02.991657 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:16:02.991999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:02.992903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:16:02.992903 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:02.992903 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:16:02.992903 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:16:02.992903 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:02.992903 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:16:02.992903 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:02.992903 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:16:02.994034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.046 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:02.994034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:16:02.994034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:16:02.994034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:16:02.994034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:16:02.994034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:16:02.994034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:16:02.994034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:02.994034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:16:02.994034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:02.994034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:16:02.994034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:02.994397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:16:02.994397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:02.994397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:16:02.994397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:02.994397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:16:02.994397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:16:02.994397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:02.994397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:16:02.994397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:02.994397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:16:02.994397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:16:02.994397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:02.994397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:16:02.994833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:02.994833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:16:02.995128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:02.995128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:16:02.995128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:16:02.995128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:16:02.995128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:16:02.995128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:16:02.995128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:16:02.995128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:16:02.995128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:16:02.995128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:16:02.995128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:16:02.995128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:16:02.995128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:16:02.995128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:16:02.995128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:16:02.995128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:16:02.995563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:16:02.995563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:16:02.995563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:02.995563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:16:02.995563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:02.995563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:16:02.995563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:02.995563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:16:02.995563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:02.995563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:16:02.995563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:02.995563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:16:02.995563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:16:02.995563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:02.995563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:16:02.995563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:02.995563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:16:02.995563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:16:02.995563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:02.996263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:16:02.996263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:02.996263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:16:03.497072 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:16:03.501271 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:16:03.501348 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:16:03.502274 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:16:03.502683 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:16:03.503061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:03.503857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:16:03.503857 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:03.503857 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:16:03.503857 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:16:03.503857 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:03.503857 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:16:03.503857 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:03.503857 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:16:03.504621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:03.504621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:16:03.504621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:16:03.504621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:16:03.504621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:16:03.504621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:16:03.504621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:16:03.504621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:03.504621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:16:03.504621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:03.504621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:16:03.504621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:03.504913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:16:03.504913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:03.504913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:16:03.504913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:03.504913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:16:03.504913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:16:03.504913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:03.504913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:16:03.504913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:03.504913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:16:03.504913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:16:03.504913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:03.504913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:16:03.505156 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:03.505156 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:16:03.505400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:03.505400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:16:03.505400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:16:03.505400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:16:03.505400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:16:03.505400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:16:03.505400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:16:03.505400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:16:03.505400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:16:03.505400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:16:03.505400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:16:03.505400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:16:03.505400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:16:03.505400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:16:03.505400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:16:03.505400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:16:03.505765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:16:03.505765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:16:03.505765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:03.505765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:16:03.505765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:03.505765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:16:03.505765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:03.505765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:16:03.505765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:03.505765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:16:03.505765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:03.505765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:16:03.505765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:16:03.505765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:03.505765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:16:03.505765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:03.505765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:16:03.505765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:16:03.505765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:03.506113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:16:03.506113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:03.506113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:16:03.507370 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:16:03.508073 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:16:03.508306 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:16:03.508360 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:16:03.509180 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:16:03.509538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:03.510403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:16:03.510403 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:03.510403 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:16:03.510403 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:16:03.510403 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:03.510403 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:16:03.510403 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:03.510403 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:16:03.511381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:03.511381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:16:03.511381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:16:03.511381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:16:03.511381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:03.511381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:16:03.511381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:03.511381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:16:03.511381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:03.511381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:16:03.511381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:03.511381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:16:03.511381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:03.511381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:16:03.511381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:16:03.511678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:03.511678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:16:03.511678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:03.511678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:16:03.511678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:16:03.511678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:03.511678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:16:03.511678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:03.511678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:16:03.512350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:03.512350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:16:03.512350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:16:03.512350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:16:03.512350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:16:03.512350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:16:03.512350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:16:03.512350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:16:03.512350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:16:03.512350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:16:03.512350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:03.512350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:16:03.512620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:03.512620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:16:03.512620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:03.512620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:16:03.512620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:03.512620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:16:03.512620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:03.512620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:16:03.512620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:16:03.512620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:03.512620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:16:03.512620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:03.512620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:16:03.512620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:16:03.512911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:03.512911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:16:03.512911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:03.512911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:16:03.513705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:03.513705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:16:03.513705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:16:03.513705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:16:03.513705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:16:03.513705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:16:03.513705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:16:03.513705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:16:03.513705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:16:03.513705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:16:03.513705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:16:03.513705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:16:03.513705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:16:03.513705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:16:03.513705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:16:03.513705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:16:03.514091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:16:03.514091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:16:03.514091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:16:03.514091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:16:03.514091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:16:03.514091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:03.514091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:16:03.514091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:03.514091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:16:03.514091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:03.514091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:16:03.514091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:03.514091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:16:03.514091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:03.514091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:16:03.514091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:16:03.514091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:03.514091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:16:03.514399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:03.514399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:16:03.514399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:16:03.514399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:03.514399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:16:03.514399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:03.514399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:16:03.522972 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:16:03.523256 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:16:03.523402 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:16:03.523590 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:16:03.524320 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:16:03.524826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:03.526251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:16:03.526251 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:03.526251 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:16:03.526251 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:16:03.526251 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:03.526251 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:16:03.526251 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:03.526251 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:16:03.527636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:03.527636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:16:03.527636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:16:03.527636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:16:03.527636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:03.527636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:16:03.527636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:03.527636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:16:03.527636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:03.527636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:16:03.527636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:03.527636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:16:03.527636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:03.527636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:16:03.527636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:16:03.528232 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:03.528232 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:16:03.528232 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:03.528232 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:16:03.528232 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:16:03.528232 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:03.528232 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:16:03.528232 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:03.528232 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:16:03.533002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:03.533002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:16:03.533002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:16:03.533002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:16:03.533002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:16:03.533002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:16:03.533002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:16:03.533002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:16:03.533002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:16:03.533002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:03.533002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:16:03.533333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:03.533333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:16:03.533333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:03.533333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:16:03.533333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:03.533333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:16:03.533333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:03.533333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:16:03.533333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:16:03.533333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:03.533333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:16:03.533333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:03.533333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:16:03.533333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:16:03.533701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:03.533701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:16:03.533701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:03.533701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:16:03.535896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:03.535896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:16:03.535896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:16:03.535896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:16:03.535896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:16:03.535896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:16:03.535896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:16:03.535896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:16:03.535896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:16:03.535896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:16:03.535896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:16:03.535896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:16:03.535896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:16:03.535896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:16:03.535896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:16:03.535896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:16:03.535896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:16:03.536760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:16:03.536760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:16:03.536760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:16:03.536760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:03.536760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:16:03.536760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:03.536760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:16:03.536760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:03.536760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:16:03.536760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:03.536760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:16:03.536760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:03.536760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:16:03.536760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:16:03.536760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:03.536760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:16:03.536760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:03.536760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:16:03.536760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:16:03.541202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:03.541202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:16:03.541202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:03.541202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:16:03.541202 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:16:03.541202 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:16:03.541202 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:16:03.541202 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:16:03.541404 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:16:03.541404 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:16:03.541404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:03.541404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:16:03.541404 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:03.541404 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:16:03.541404 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:16:03.541404 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:03.541404 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:16:03.541404 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:03.541404 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:16:03.541404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:03.541664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:16:03.541664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:16:03.541664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:16:03.541664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:16:03.541664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:16:03.541664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:16:03.541664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:03.541664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:16:03.541664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:03.541664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:16:03.541664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:03.541664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:16:03.541664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:03.541664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:16:03.542228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:03.542228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:16:03.542228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:16:03.542228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:03.542228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:16:03.542228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:03.542228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:16:03.542228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:16:03.542228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:03.542228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:16:03.542228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:03.542228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:16:03.542481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:03.542481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:16:03.542481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:16:03.542481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:16:03.542481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:16:03.542481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:16:03.542481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:16:03.542481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:16:03.542481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:16:03.542481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:16:03.542481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:16:03.542481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:16:03.542481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:16:03.542481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:16:03.542481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:16:03.542481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:16:03.543104 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:16:03.543104 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:16:03.543104 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:03.543104 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:16:03.543104 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:03.543104 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:16:03.543104 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:03.543104 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:16:03.543104 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:03.543104 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:16:03.543104 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:03.543104 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:16:03.543104 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:16:03.543104 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:03.543104 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:16:03.543104 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:03.543104 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:16:03.543104 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:16:03.543104 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:03.543459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:16:03.543459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:03.543459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:16:07.947069 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:16:07.947680 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:16:07.947965 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:16:07.949193 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:16:07.982227 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:16:07.982633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:07.983489 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:16:07.983489 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:07.983489 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:16:07.983489 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:16:07.983489 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:07.983489 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:16:07.983489 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:07.983489 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:16:07.994564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:07.994564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:16:07.994564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:16:07.994564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:16:07.994564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:16:07.994564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:16:07.994564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:16:07.994564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:07.994564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:16:07.994564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:07.994564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:16:07.994564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:07.994996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:16:07.994996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:07.994996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:16:07.994996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:07.994996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:16:07.994996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:16:07.994996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:07.994996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:16:07.994996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:07.994996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:16:07.994996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:16:07.994996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:07.994996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:16:07.995343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:07.995343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:16:07.995343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:07.995343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:16:07.995343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:16:07.995343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:16:07.995343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:16:07.995343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:16:07.995343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:16:07.995343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:16:07.995343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:16:07.995343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:16:07.995343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:16:07.995343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:16:07.995343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:16:07.995797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:16:07.995797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:16:07.995797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:16:07.995797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:16:07.995797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:16:07.995797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:07.995797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:16:07.995797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:07.995797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:16:07.995797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:07.995797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:16:07.995797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:07.995797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:16:07.995797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:07.995797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:16:07.995797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:16:07.995797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:07.995797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:16:07.995797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:07.996289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:16:07.996289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:16:07.996289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:07.996289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:16:07.996289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:07.996289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:16:08.503384 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:16:08.505841 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:16:08.505991 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:16:08.507022 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:16:08.507366 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:16:08.507583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:08.508406 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:16:08.508823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:16:08.508823 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:08.508823 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:16:08.508823 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:16:08.508823 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:08.508823 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:16:08.508823 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:08.508823 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:16:08.509891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:08.509891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:16:08.509891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:16:08.509891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:16:08.509891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:16:08.509891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:16:08.509891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:16:08.509891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:08.509891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:16:08.509891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:08.509891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:16:08.509891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:08.510178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:16:08.510178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:08.510178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:16:08.510178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:08.510178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:16:08.510178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:16:08.510178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:08.510178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:16:08.510178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:08.510178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:16:08.510178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:16:08.510178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:08.510178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:16:08.510416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:08.510416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:16:08.511050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:08.511050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:16:08.511050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:16:08.511050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:16:08.511050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:16:08.511050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:16:08.511050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:16:08.511050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:16:08.511050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:16:08.511050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:16:08.511050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:16:08.511050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:16:08.511050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:16:08.511050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:16:08.511050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:16:08.511050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:16:08.511430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:16:08.511430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:16:08.511430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:08.511430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:16:08.511430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:08.511430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:16:08.511430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:08.511430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:16:08.511430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:08.511430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:16:08.511430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:08.511430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:16:08.511430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:16:08.511430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:08.511430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:16:08.511430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:08.511430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:16:08.511430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:16:08.511430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:08.511894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:16:08.511894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:08.511894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:16:08.511894 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:16:08.511894 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:16:08.511894 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:16:08.512461 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:16:08.512730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:08.513282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:16:08.513282 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:08.513282 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:16:08.513282 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:16:08.513282 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:08.513282 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:16:08.513282 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:08.513282 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:16:08.514072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:08.514072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:16:08.514072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:16:08.514072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:16:08.514072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:08.514072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:16:08.514072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:08.514072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:16:08.514072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:08.514072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:16:08.514072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:08.514072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:16:08.514072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:08.514072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:16:08.514072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:16:08.514986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:08.514986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:16:08.514986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:08.514986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:16:08.514986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:16:08.514986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:08.514986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:16:08.514986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:08.514986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:16:08.514986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:08.514986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:16:08.514986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:16:08.514986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:16:08.514986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:16:08.515527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:16:08.515527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:16:08.515527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:16:08.515527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:16:08.515527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:16:08.515527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:08.515527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:16:08.515527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:08.515527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:16:08.515527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:08.515527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:16:08.515527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:08.515527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:16:08.516077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:08.516077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:16:08.516077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:16:08.516077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:08.516077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:16:08.516077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:08.516077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:16:08.516077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:16:08.516077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:08.516077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:16:08.516077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:08.516077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:16:08.516617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:08.516617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:16:08.516617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:16:08.516617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:16:08.516617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:16:08.516617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:16:08.516617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:16:08.516617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:16:08.516617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:16:08.516617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:16:08.516617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:16:08.516617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:16:08.516617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:16:08.516617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:16:08.516617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:16:08.516617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:16:08.518902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:16:08.518902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:16:08.518902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:16:08.518902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:16:08.518902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:16:08.518902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:08.518902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:16:08.518902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:08.518902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:16:08.518902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:08.518902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:16:08.518902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:08.518902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:16:08.518902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:08.518902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:16:08.518902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:16:08.518902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:08.518902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:16:08.520923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:08.520923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:16:08.520923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:16:08.520923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:08.520923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:16:08.520923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:08.520923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:16:08.523505 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:16:08.524251 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:16:08.524612 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:16:08.525014 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:16:08.525504 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:16:08.525798 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:08.526518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:16:08.526518 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:08.526518 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:16:08.526518 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:16:08.526518 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:08.526518 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:16:08.526518 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:08.526518 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:16:08.528571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:08.528571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:16:08.528571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:16:08.528571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:16:08.528571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:08.528571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:16:08.528571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:08.528571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:16:08.528571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:08.528571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:16:08.528571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:08.528571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:16:08.528571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:08.528571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:16:08.528571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:16:08.529167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:08.529167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:16:08.529167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:08.529167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:16:08.529167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:16:08.529167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:08.529167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:16:08.529167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:08.529167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:16:08.530766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:08.530766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:16:08.530766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:16:08.530766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:16:08.530766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:16:08.530766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:16:08.530766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:16:08.530766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:16:08.530766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:16:08.530766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:08.530766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:16:08.531061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:08.531061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:16:08.531061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:08.531061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:16:08.531061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:08.531061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:16:08.531061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:08.531061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:16:08.531061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:16:08.531061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:08.531061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:16:08.531061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:08.531061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:16:08.531061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:16:08.531399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:08.531399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:16:08.531399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:08.531399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:16:08.532637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:08.532637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:16:08.532637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:16:08.532637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:16:08.532637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:16:08.532637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:16:08.532637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:16:08.532637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:16:08.532637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:16:08.532637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:16:08.532637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:16:08.532637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:16:08.532637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:16:08.532637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:16:08.532637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:16:08.532637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:16:08.532637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:16:08.533090 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:16:08.533090 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:16:08.533090 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:16:08.533090 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:08.533090 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:16:08.533090 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:08.533090 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:16:08.533090 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:08.533090 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:16:08.533090 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:08.533090 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:16:08.533090 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:08.533090 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:16:08.533090 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:16:08.533090 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:08.533090 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:16:08.533090 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:08.533090 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:16:08.533090 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:16:08.533555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:08.533555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:16:08.533555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:08.533555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:16:08.538851 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:16:08.539337 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:16:08.539544 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:16:08.539814 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:16:08.540573 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:16:08.541015 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:16:08.541250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:08.542091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:16:08.542091 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:08.542091 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:16:08.542091 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:16:08.542091 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:08.542091 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:16:08.542091 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:08.542091 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:16:08.542758 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:08.542758 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:16:08.542758 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:16:08.542758 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:16:08.542758 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:16:08.542758 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:16:08.542758 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:16:08.542758 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:08.542758 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:16:08.542758 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:08.542758 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:16:08.542758 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:08.543116 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:16:08.543116 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:08.543116 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:16:08.543116 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:08.543116 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:16:08.543116 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:16:08.543116 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:08.543116 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:16:08.543116 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:08.543116 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:16:08.543116 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:16:08.543116 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:08.543116 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:16:08.543446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:08.543446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:16:08.543506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:08.543506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:16:08.543506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:16:08.543506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:16:08.543506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:16:08.543506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:16:08.543506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:16:08.543506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:16:08.543506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:16:08.543506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:16:08.543506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:16:08.543506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:16:08.543506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:16:08.543506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:16:08.543506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:16:08.543506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:16:08.543946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:16:08.543946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:16:08.543946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:08.543946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:16:08.543946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:08.543946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:16:08.543946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:08.543946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:16:08.543946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:08.543946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:16:08.543946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:08.543946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:16:08.543946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:16:08.543946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:08.543946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:16:08.543946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:08.543946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:16:08.543946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:16:08.543946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:08.544449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:16:08.544449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:08.544449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:16:12.948784 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:16:12.948784 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:16:12.948784 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:16:12.949534 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:16:12.991492 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:16:12.991765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:12.992293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:16:12.992293 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:12.992293 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:16:12.992293 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:16:12.992293 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:12.992293 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:16:12.992293 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:12.992293 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:16:12.993202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.044 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:12.993202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:16:12.993202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:16:12.993202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:16:12.993202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:16:12.993202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:16:12.993202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:16:12.993202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:12.993202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:16:12.993202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:12.993202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:16:12.993202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:12.993560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:16:12.993560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:12.993560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:16:12.993560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:12.993560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:16:12.993560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:16:12.993560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:12.993560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:16:12.993560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:12.993560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:16:12.993560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:16:12.993560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:12.993560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:16:12.993940 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:12.993940 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:16:12.994912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:12.994912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:16:12.994912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:16:12.994912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:16:12.994912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:16:12.994912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:16:12.994912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:16:12.994912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:16:12.994912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:16:12.994912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:16:12.994912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:16:12.994912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:16:12.994912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:16:12.994912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:16:12.994912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:16:12.994912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:16:12.995722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:16:12.995722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:16:12.995722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:12.995722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:16:12.995722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:12.995722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:16:12.995722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:12.995722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:16:12.995722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:12.995722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:16:12.995722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:12.995722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:16:12.995722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:16:12.995722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:12.995722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:16:12.995722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:12.995722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:16:12.995722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:16:12.995722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:12.996111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:16:12.996111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:12.996111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:16:13.506091 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:16:13.506612 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:16:13.506977 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:16:13.507776 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:16:13.508044 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:16:13.509830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:13.509830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:16:13.509830 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:13.509830 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:16:13.509830 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:16:13.509830 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:13.509830 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:16:13.509830 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:13.509830 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:16:13.510167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:13.510167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:16:13.510167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:16:13.510167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:16:13.510167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:16:13.510167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:16:13.510167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:16:13.510167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:13.510167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:16:13.510167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:13.510167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:16:13.510167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:13.510609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:16:13.510609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:13.510609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:16:13.510609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:13.510609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:16:13.510609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:16:13.510609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:13.510609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:16:13.510609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:13.510609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:16:13.510609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:16:13.510609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:13.510609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:16:13.511113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:13.511113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:16:13.512943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:13.512943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:16:13.512943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:16:13.512943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:16:13.512943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:16:13.512943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:16:13.512943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:16:13.512943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:16:13.512943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:16:13.512943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:16:13.512943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:16:13.512943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:16:13.512943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:16:13.512943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:16:13.512943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:16:13.512943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:16:13.514939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:16:13.514939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:16:13.514939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:13.514939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:16:13.514939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:13.514939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:16:13.514939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:13.514939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:16:13.514939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:13.514939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:16:13.514939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:13.514939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:16:13.514939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:16:13.514939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:13.514939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:16:13.514939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:13.514939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:16:13.514939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:16:13.514939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:13.515854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:16:13.515854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:13.515854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:16:13.515854 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:16:13.515854 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:16:13.515854 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:16:13.515854 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:16:13.515854 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:16:13.516130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:13.516130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:16:13.516130 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:13.516130 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:16:13.516130 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:16:13.516130 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:13.516130 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:16:13.516130 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:13.516130 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:16:13.516130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:13.516130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:16:13.516130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:16:13.516130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:16:13.516130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:13.516130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:16:13.516130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:13.516130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:16:13.516560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:13.516560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:16:13.516560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:13.516560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:16:13.516560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:13.516560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:16:13.516560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:16:13.516560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:13.516560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:16:13.516560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:13.516560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:16:13.516560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:16:13.516560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:13.516560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:16:13.516560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:13.516560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:16:13.518647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:13.518647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:16:13.518647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:16:13.518647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:16:13.518647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:16:13.518647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:16:13.518647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:16:13.518647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:16:13.518647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:16:13.518647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:16:13.518647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:13.518647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:16:13.519322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:13.519322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:16:13.519322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:13.519322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:16:13.519322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:13.519322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:16:13.519322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:13.519322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:16:13.519322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:16:13.519322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:13.519322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:16:13.519322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:13.519322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:16:13.519322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:16:13.519631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:13.519631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:16:13.519631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:13.519631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:16:13.519631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:13.519631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:16:13.519631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:16:13.519631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:16:13.519631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:16:13.519631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:16:13.519631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:16:13.519631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:16:13.519631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:16:13.519631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:16:13.519912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:16:13.519912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:16:13.519912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:16:13.519912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:16:13.519912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:16:13.519912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:16:13.519912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:16:13.519912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:16:13.519912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:16:13.519912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:16:13.519912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:16:13.519912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:13.519912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:16:13.519912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:13.519912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:16:13.519912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:13.519912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:16:13.519912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:13.519912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:16:13.520237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:13.520237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:16:13.520237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:16:13.520237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:13.520237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:16:13.520237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:13.520237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:16:13.520237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:16:13.520237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:13.520237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:16:13.520237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:13.520237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:16:13.524187 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:16:13.524241 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:16:13.524362 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:16:13.524464 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:16:13.524918 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:16:13.525267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:13.527524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:16:13.527524 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:13.527524 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:16:13.527524 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:16:13.527524 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:13.527524 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:16:13.527524 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:13.527524 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:16:13.528443 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:13.528443 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:16:13.528443 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:16:13.528443 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:16:13.528443 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:13.528443 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:16:13.528443 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:13.528443 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:16:13.528443 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:13.528443 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:16:13.528443 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:13.528443 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:16:13.528443 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:13.528443 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:16:13.528443 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:16:13.528893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:13.528893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:16:13.528893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:13.528893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:16:13.528893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:16:13.528893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:13.528893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:16:13.528893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:13.528893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:16:13.529649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:13.529649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:16:13.529649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:16:13.529649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:16:13.529649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:16:13.529649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:16:13.529649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:16:13.529649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:16:13.529649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:16:13.529649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:13.529649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:16:13.530570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:13.530570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:16:13.530570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:13.530570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:16:13.530570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:13.530570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:16:13.530570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:13.530570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:16:13.530570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:16:13.530570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:13.530570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:16:13.530570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:13.530570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:16:13.530570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:16:13.531253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:13.531253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:16:13.531253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:13.531253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:16:13.531253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:13.531253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:16:13.531253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:16:13.531253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:16:13.531253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:16:13.531253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:16:13.531253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:16:13.531253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:16:13.531253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:16:13.531253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:16:13.531636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:16:13.531636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:16:13.531636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:16:13.531636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:16:13.531636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:16:13.531636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:16:13.531636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:16:13.531636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:16:13.531636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:16:13.531636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:16:13.531636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:13.531636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:16:13.531636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:13.531636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:16:13.531636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:13.531636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:16:13.531636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:13.531636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:16:13.531636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:13.532180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:16:13.532180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:16:13.532180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:13.532180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:16:13.532180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:13.532180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:16:13.532180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:16:13.532180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:13.532180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:16:13.532180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:13.532180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:16:13.539358 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:16:13.539636 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:16:13.539777 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:16:13.539966 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:16:13.540369 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:16:13.540679 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:16:13.540922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:13.541557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:16:13.541557 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:13.541557 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:16:13.541557 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:16:13.541557 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:13.541557 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:16:13.541557 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:13.541557 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:16:13.542467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:13.542467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:16:13.542467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:16:13.542467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:16:13.542467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:16:13.542467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:16:13.542467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:16:13.542467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:13.542467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:16:13.542467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:13.542467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:16:13.542467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:13.542842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:16:13.542842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:13.542842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:16:13.542842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:13.542842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:16:13.542842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:16:13.542842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:13.542842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:16:13.542842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:13.542842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:16:13.542842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:16:13.542842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:13.542842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:16:13.543190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:13.543190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:16:13.543579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:13.543579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:16:13.543579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:16:13.543579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:16:13.543579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:16:13.543579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:16:13.543579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:16:13.543579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:16:13.543579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:16:13.543579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:16:13.543579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:16:13.543579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:16:13.543579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:16:13.543579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:16:13.543579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:16:13.543579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:16:13.544055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:16:13.544055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:16:13.544055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:13.544055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:16:13.544055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:13.544055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:16:13.544055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:13.544055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:16:13.544055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:13.544055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:16:13.544055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:13.544055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:16:13.544055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:16:13.544055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:13.544055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:16:13.544055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:13.544055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:16:13.544055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:16:13.544055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:13.544533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:16:13.544533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:13.544533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:16:17.948570 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:16:17.949182 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:16:17.949182 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:16:17.950166 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:16:17.993798 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:16:17.994298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:17.995196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:16:17.995196 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:17.995196 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:16:17.995196 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:16:17.995196 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:17.995196 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:16:17.995196 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:17.995196 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:16:17.996223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.046 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:17.996223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:16:17.996223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:16:17.996223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:16:17.996223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:16:17.996223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:16:17.996223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:16:17.996223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:17.996223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:16:17.996223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:17.996223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:16:17.996223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:17.996676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:16:17.996676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:17.996676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:16:17.996676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:17.996676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:16:17.996676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:16:17.996676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:17.996676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:16:17.996676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:17.996676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:16:17.996676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:16:17.996676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:17.996676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:16:17.997114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:17.997114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:16:17.997194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:17.997194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:16:17.997194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:16:17.997194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:16:17.997194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:16:17.997194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:16:17.997194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:16:17.997194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:16:17.997194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:16:17.997194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:16:17.997194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:16:17.997194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:16:17.997194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:16:17.997194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:16:17.997194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:16:17.997194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:16:17.997726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:16:17.997726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:16:17.997726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:17.997726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:16:17.997726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:17.997726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:16:17.997726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:17.997726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:16:17.997726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:17.997726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:16:17.997726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:17.997726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:16:17.997726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:16:17.997726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:17.997726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:16:17.997726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:17.997726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:16:17.997726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:16:17.997726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:17.998273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:16:17.998273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:17.998273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:16:18.506637 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:16:18.507158 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:16:18.507423 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:16:18.508965 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:16:18.508965 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:16:18.508965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:18.508965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:16:18.508965 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:18.508965 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:16:18.508965 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:16:18.508965 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:18.508965 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:16:18.508965 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:18.508965 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:16:18.510150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:18.510150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:16:18.510150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:16:18.510150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:16:18.510150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:16:18.510150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:16:18.510150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:16:18.510150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:18.510150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:16:18.510150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:18.510150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:16:18.510150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:18.510548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:16:18.510548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:18.510548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:16:18.510548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:18.510548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:16:18.510548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:16:18.510548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:18.510548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:16:18.510548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:18.510548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:16:18.510548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:16:18.510548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:18.510548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:16:18.511035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:18.511035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:16:18.511285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:18.511285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:16:18.511285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:16:18.511285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:16:18.511285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:16:18.511285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:16:18.511285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:16:18.511285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:16:18.511285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:16:18.511285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:16:18.511285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:16:18.511285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:16:18.511285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:16:18.511285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:16:18.511285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:16:18.511285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:16:18.511731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:16:18.511731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:16:18.511731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:18.511731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:16:18.511731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:18.511731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:16:18.511731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:18.511731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:16:18.511731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:18.511731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:16:18.511731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:18.511731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:16:18.511731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:16:18.511731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:18.511731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:16:18.511731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:18.511731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:16:18.511731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:16:18.511731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:18.512294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:16:18.512294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:18.512294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:16:18.512997 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:16:18.513286 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:16:18.513406 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:16:18.513532 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:16:18.514042 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:16:18.514188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:18.514722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:16:18.514722 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:18.514722 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:16:18.514722 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:16:18.514722 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:18.514722 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:16:18.514722 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:18.514722 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:16:18.515440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:18.515440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:16:18.515440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:16:18.515440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:16:18.515440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:18.515440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:16:18.515440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:18.515440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:16:18.515440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:18.515440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:16:18.515440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:18.515440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:16:18.515440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:18.515440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:16:18.515440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:16:18.516438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:18.516438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:16:18.516438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:18.516438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:16:18.516438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:16:18.516438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:18.516438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:16:18.516438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:18.516438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:16:18.516438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:18.516438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:16:18.516438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:16:18.516438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:16:18.516438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:16:18.516859 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:16:18.516859 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:16:18.516859 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:16:18.516859 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:16:18.516859 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:16:18.516859 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:18.516859 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:16:18.516859 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:18.516859 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:16:18.516859 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:18.516859 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:16:18.516859 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:18.516859 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:16:18.517250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:18.517250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:16:18.517250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:16:18.517250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:18.517250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:16:18.517250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:18.517250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:16:18.517250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:16:18.517250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:18.517250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:16:18.517250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:18.517250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:16:18.517724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:18.517724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:16:18.517724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:16:18.517724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:16:18.517724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:16:18.517724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:16:18.517724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:16:18.517724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:16:18.517724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:16:18.517724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:16:18.517724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:16:18.517724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:16:18.517724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:16:18.517724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:16:18.517724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:16:18.517724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:16:18.518427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:16:18.518427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:16:18.518427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:16:18.518427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:16:18.518427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:16:18.518427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:18.518427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:16:18.518427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:18.518427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:16:18.518427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:18.518427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:16:18.518427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:18.518427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:16:18.518427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:18.518427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:16:18.518427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:16:18.518427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:18.518427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:16:18.519082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:18.519082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:16:18.519082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:16:18.519082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:18.519082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:16:18.519082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:18.519082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:16:18.527882 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:16:18.527882 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:16:18.527882 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:16:18.527882 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:16:18.527882 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:16:18.527882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:18.528313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:16:18.528313 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:18.528313 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:16:18.528313 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:16:18.528313 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:18.528313 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:16:18.528313 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:18.528313 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:16:18.528313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:18.528313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:16:18.528313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:16:18.528313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:16:18.528313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:18.528313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:16:18.528313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:18.528313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:16:18.528313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:18.528313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:16:18.528811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:18.528811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:16:18.528811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:18.528811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:16:18.528811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:16:18.528811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:18.528811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:16:18.528811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:18.528811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:16:18.528811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:16:18.528811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:18.528811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:16:18.528811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:18.528811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:16:18.529198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:18.529198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:16:18.529198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:16:18.529198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:16:18.529198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:16:18.529198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:16:18.529198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:16:18.529198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:16:18.529198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:16:18.529198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:18.529198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:16:18.529729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:18.529729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:16:18.529729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:18.529729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:16:18.529729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:18.529729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:16:18.529729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:18.529729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:16:18.529729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:16:18.529729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:18.529729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:16:18.529729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:18.529729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:16:18.529729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:16:18.530134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:18.530134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:16:18.530134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:18.530134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:16:18.530722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:18.530722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:16:18.530722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:16:18.530722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:16:18.530722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:16:18.530722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:16:18.530722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:16:18.530722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:16:18.530722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:16:18.530722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:16:18.530722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:16:18.530722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:16:18.530722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:16:18.530722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:16:18.530722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:16:18.530722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:16:18.530722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:16:18.531270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:16:18.531270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:16:18.531270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:16:18.531270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:18.531270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:16:18.531270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:18.531270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:16:18.531270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:18.531270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:16:18.531270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:18.531270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:16:18.531270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:18.531270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:16:18.531270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:16:18.531270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:18.531270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:16:18.531270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:18.531270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:16:18.531270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:16:18.531830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:18.531830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:16:18.531830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:18.531830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:16:18.539887 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:16:18.540218 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:16:18.540288 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:16:18.540475 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:16:18.540918 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:16:18.541268 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:16:18.541442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:18.542008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:16:18.542008 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:18.542008 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:16:18.542008 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:16:18.542008 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:18.542008 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:16:18.542008 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:18.542008 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:16:18.542685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:18.542685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:16:18.542685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:16:18.542685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:16:18.542685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:16:18.542685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:16:18.542685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:16:18.542685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:18.542685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:16:18.542685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:18.542685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:16:18.542685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:18.542981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:16:18.542981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:18.542981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:16:18.542981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:18.542981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:16:18.542981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:16:18.542981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:18.542981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:16:18.542981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:18.542981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:16:18.542981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:16:18.542981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:18.542981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:16:18.543269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:18.543269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:16:18.543415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:18.543415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:16:18.543415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:16:18.543415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:16:18.543415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:16:18.543415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:16:18.543415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:16:18.543415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:16:18.543415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:16:18.543415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:16:18.543415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:16:18.543415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:16:18.543415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:16:18.543415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:16:18.543415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:16:18.543415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:16:18.543732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:16:18.543732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:16:18.543732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:18.543732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:16:18.543732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:18.543732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:16:18.543732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:18.543732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:16:18.543732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:18.543732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:16:18.543732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:18.543732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:16:18.543732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:16:18.543732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:18.543732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:16:18.543732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:18.543732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:16:18.543732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:16:18.543732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:18.544151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:16:18.544151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:18.544151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:16:22.949300 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:16:22.950229 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:16:22.950229 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:16:22.951572 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:16:22.994122 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:16:22.994579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:22.995530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:16:22.995530 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:22.995530 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:16:22.995530 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:16:22.995530 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:22.995530 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:16:22.995530 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:22.995530 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:16:22.996428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.045 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:22.996428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:16:22.996428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:16:22.996428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:16:22.996428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:16:22.996428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:16:22.996428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:16:22.996428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:22.996428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:16:22.996428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:22.996428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:16:22.996428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:22.996799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:16:22.996799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:22.996799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:16:22.996799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:22.996799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:16:22.996799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:16:22.996799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:22.996799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:16:22.996799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:22.996799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:16:22.996799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:16:22.996799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:22.996799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:16:22.997642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:22.997642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:16:22.997642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:22.997642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:16:22.997642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:16:22.997642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:16:22.997642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:16:22.997642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:16:22.997642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:16:22.997642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:16:22.997642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:16:22.997642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:16:22.997642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:16:22.997642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:16:22.997642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:16:22.998245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:16:22.998245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:16:22.998245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:16:22.998245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:16:22.998245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:16:22.998245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:22.998245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:16:22.998245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:22.998245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:16:22.998245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:22.998245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:16:22.998245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:22.998245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:16:22.998245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:22.998245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:16:22.998245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:16:22.998245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:22.998245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:16:22.998245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:22.998688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:16:22.998688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:16:22.998688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:22.998688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:16:22.998688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:22.998688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:16:23.508513 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:16:23.508983 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:16:23.509169 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:16:23.510234 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:16:23.510581 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:16:23.510930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:23.511853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:16:23.511853 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:23.511853 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:16:23.511853 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:16:23.511853 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:23.511853 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:16:23.511853 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:23.511853 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:16:23.512930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:23.512930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:16:23.512930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:16:23.512930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:16:23.512930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:16:23.512930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:16:23.512930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:16:23.512930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:23.512930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:16:23.512930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:23.512930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:16:23.512930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:23.513468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:16:23.513468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:23.513468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:16:23.513468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:23.513468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:16:23.513468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:16:23.513468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:23.513468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:16:23.513468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:23.513468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:16:23.513468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:16:23.513468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:23.513468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:16:23.513889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:23.513889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:16:23.514257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:23.514257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:16:23.514257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:16:23.514257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:16:23.514257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:16:23.514257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:16:23.514257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:16:23.514257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:16:23.514257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:16:23.514257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:16:23.514257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:16:23.514257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:16:23.514257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:16:23.514257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:16:23.514257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:16:23.514257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:16:23.514861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:16:23.514861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:16:23.514861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:23.514861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:16:23.514861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:23.514861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:16:23.514861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:23.514861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:16:23.514861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:23.514861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:16:23.514861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:23.514861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:16:23.514861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:16:23.514861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:23.514861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:16:23.514861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:23.514861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:16:23.514861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:16:23.514861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:23.515388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:16:23.515388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:23.515388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:16:23.517094 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:16:23.517349 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:16:23.517536 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:16:23.517679 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:16:23.518014 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:16:23.518211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:23.518712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:16:23.518712 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:23.518712 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:16:23.518712 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:16:23.518712 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:23.518712 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:16:23.518712 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:23.518712 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:16:23.519574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:23.519574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:16:23.519574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:16:23.519574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:16:23.519574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:23.519574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:16:23.519574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:23.519574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:16:23.519574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:23.519574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:16:23.519574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:23.519574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:16:23.519574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:23.519574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:16:23.519574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:16:23.520392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:23.520392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:16:23.520392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:23.520392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:16:23.520392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:16:23.520392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:23.520392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:16:23.520392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:23.520392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:16:23.520875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:23.520875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:16:23.520875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:16:23.520875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:16:23.520875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:16:23.520875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:16:23.520875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:16:23.520875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:16:23.520875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:16:23.520875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:16:23.520875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:23.520875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:16:23.521435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:23.521435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:16:23.521435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:23.521435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:16:23.521435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:23.521435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:16:23.521435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:23.521435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:16:23.521435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:16:23.521435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:23.521435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:16:23.521435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:23.521435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:16:23.521435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:16:23.522170 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:23.522170 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:16:23.522170 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:23.522170 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:16:23.522170 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:23.522170 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:16:23.522170 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:16:23.522170 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:16:23.522170 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:16:23.522170 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:16:23.522170 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:16:23.522170 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:16:23.522170 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:16:23.522170 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:16:23.522601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:16:23.522601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:16:23.522601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:16:23.522601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:16:23.522601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:16:23.522601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:16:23.522601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:16:23.522601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:16:23.522601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:16:23.522601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:16:23.522601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:16:23.522601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:23.522601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:16:23.522601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:23.522601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:16:23.522601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:23.522601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:16:23.522601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:23.522601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:16:23.523713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:23.523713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:16:23.523713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:16:23.523713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:23.523713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:16:23.523713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:23.523713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:16:23.523713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:16:23.523713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:23.523713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:16:23.523713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:23.523713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:16:23.524791 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:16:23.526132 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:16:23.526311 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:16:23.526367 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:16:23.526954 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:16:23.527376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:23.528264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:16:23.528264 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:23.528264 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:16:23.528264 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:16:23.528264 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:23.528264 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:16:23.528264 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:23.528264 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:16:23.528917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:23.528917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:16:23.528917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:16:23.528917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:16:23.528917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:23.528917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:16:23.528917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:23.528917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:16:23.528917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:23.528917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:16:23.528917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:23.528917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:16:23.528917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:23.528917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:16:23.528917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:16:23.529341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:23.529341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:16:23.529341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:23.529341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:16:23.529341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:16:23.529341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:23.529341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:16:23.529341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:23.529341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:16:23.529644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:23.529644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:16:23.529644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:16:23.529644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:16:23.529644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:16:23.529644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:16:23.529644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:16:23.529644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:16:23.529644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:16:23.529644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:23.529644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:16:23.529955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:23.529955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:16:23.529955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:23.529955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:16:23.529955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:23.529955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:16:23.529955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:23.529955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:16:23.529955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:16:23.529955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:23.529955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:16:23.529955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:23.529955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:16:23.529955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:16:23.530315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:23.530315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:16:23.530315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:23.530315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:16:23.530427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:23.530427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:16:23.530427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:16:23.530427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:16:23.530427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:16:23.530427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:16:23.530427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:16:23.530427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:16:23.530427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:16:23.530427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:16:23.530427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:16:23.530427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:16:23.530427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:16:23.530427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:16:23.530427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:16:23.530427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:16:23.530427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:16:23.530875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:16:23.530875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:16:23.530875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:16:23.530875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:23.530875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:16:23.530875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:23.530875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:16:23.530875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:23.530875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:16:23.530875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:23.530875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:16:23.530875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:23.530875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:16:23.530875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:16:23.530875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:23.530875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:16:23.530875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:23.530875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:16:23.530875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:16:23.531378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:23.531378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:16:23.531378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:23.531378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:16:23.540225 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:16:23.540688 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:16:23.540836 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:16:23.541053 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:16:23.541620 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:16:23.542146 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:16:23.542413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:23.543213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:16:23.543213 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:23.543213 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:16:23.543213 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:16:23.543213 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:23.543213 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:16:23.543213 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:23.543213 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:16:23.544258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:23.544258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:16:23.544258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:16:23.544258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:16:23.544258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:16:23.544258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:16:23.544258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:16:23.544258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:23.544258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:16:23.544258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:23.544258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:16:23.544258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:23.544604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:16:23.544604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:23.544604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:16:23.544604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:23.544604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:16:23.544604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:16:23.544604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:23.544604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:16:23.544604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:23.544604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:16:23.544604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:16:23.544604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:23.544604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:16:23.544970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:23.544970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:16:23.545393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:23.545393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:16:23.545393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:16:23.545393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:16:23.545393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:16:23.545393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:16:23.545393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:16:23.545393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:16:23.545393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:16:23.545393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:16:23.545393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:16:23.545393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:16:23.545393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:16:23.545393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:16:23.545393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:16:23.545393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:16:23.545892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:16:23.545892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:16:23.545892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:23.545892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:16:23.545892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:23.545892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:16:23.545892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:23.545892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:16:23.545892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:23.545892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:16:23.545892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:23.545892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:16:23.545892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:16:23.545892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:23.545892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:16:23.545892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:23.545892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:16:23.545892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:16:23.545892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:23.546402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:16:23.546402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:23.546402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:16:27.950034 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:16:27.950792 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:16:27.950792 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:16:27.951824 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:16:27.994855 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:16:27.995399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:27.996626 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:16:27.996626 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:27.996626 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:16:27.996626 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:16:27.996626 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:27.996626 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:16:27.996626 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:27.996626 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:16:27.997623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.046 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:27.997623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:16:27.997623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:16:27.997623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:16:27.997623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:16:27.997623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:16:27.997623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:16:27.997623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:27.997623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:16:27.997623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:27.997623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:16:27.997623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:28.015826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:16:28.015826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:28.015826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:16:28.015826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:28.015826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:16:28.015826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:16:28.015826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:28.015826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:16:28.015826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:28.015826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:16:28.015826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:16:28.015826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:28.015826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:16:28.016424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:28.016424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:16:28.016424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:28.016424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:16:28.016424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:16:28.016424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:16:28.016424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:16:28.016424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:16:28.016424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:16:28.016424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:16:28.016424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:16:28.016424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:16:28.016424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:16:28.016424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:16:28.016424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:16:28.017244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:16:28.017244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:16:28.017244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:16:28.017244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:16:28.017244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:16:28.017244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:28.017244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:16:28.017244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:28.017244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:16:28.017244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:28.017244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:16:28.017244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:28.017244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:16:28.017244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:28.017244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:16:28.017244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:16:28.017244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:28.017244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:16:28.017244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:28.018276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:16:28.018276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:16:28.018276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:28.018276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:16:28.018276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:28.018276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:16:28.509580 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:16:28.510811 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:16:28.510811 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:16:28.512258 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:16:28.512837 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:16:28.513115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:28.514221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:16:28.514221 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:28.514221 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:16:28.514221 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:16:28.514221 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:28.514221 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:16:28.514221 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:28.514221 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:16:28.515435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:28.515435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:16:28.515435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:16:28.515435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:16:28.515435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:16:28.515435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:16:28.515435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:16:28.515435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:28.515435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:16:28.515435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:28.515435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:16:28.515435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:28.516135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:16:28.516135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:28.516135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:16:28.516135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:28.516135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:16:28.516135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:16:28.516135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:28.516135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:16:28.516135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:28.516135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:16:28.516135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:16:28.516135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:28.516135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:16:28.517523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:28.517523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:16:28.517523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:28.517523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:16:28.517523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:16:28.517523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:16:28.517523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:16:28.517523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:16:28.517523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:16:28.517523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:16:28.517523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:16:28.517523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:16:28.517523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:16:28.517523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:16:28.517523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:16:28.518780 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:16:28.518780 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:16:28.518780 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:16:28.518780 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:16:28.518780 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:16:28.518780 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:28.518780 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:16:28.518780 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:28.518780 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:16:28.518780 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:28.518780 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:16:28.518780 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:28.518780 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:16:28.518780 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:28.518780 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:16:28.518780 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:16:28.518780 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:28.518780 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:16:28.518780 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:28.519782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:16:28.519782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:16:28.519782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:28.519782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:16:28.519782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:28.519782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:16:28.519782 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:16:28.519782 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:16:28.519782 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:16:28.519782 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:16:28.520202 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:16:28.520202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:28.520202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:16:28.520202 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:28.520202 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:16:28.520202 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:16:28.520202 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:28.520202 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:16:28.520202 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:28.520202 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:16:28.521306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:28.521306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:16:28.521306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:16:28.521306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:16:28.521306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:28.521306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:16:28.521306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:28.521306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:16:28.521306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:28.521306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:16:28.521306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:28.521306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:16:28.521306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:28.521306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:16:28.521306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:16:28.522763 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:28.522763 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:16:28.522763 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:28.522763 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:16:28.522763 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:16:28.522763 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:28.522763 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:16:28.522763 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:28.522763 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:16:28.522763 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:28.522763 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:16:28.522763 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:16:28.522763 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:16:28.522763 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:16:28.523402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:16:28.523402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:16:28.523402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:16:28.523402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:16:28.523402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:16:28.523402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:28.523402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:16:28.523402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:28.523402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:16:28.523402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:28.523402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:16:28.523402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:28.523402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:16:28.523816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:28.523816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:16:28.523816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:16:28.523816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:28.523816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:16:28.523816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:28.523816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:16:28.523816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:16:28.523816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:28.523816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:16:28.523816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:28.523816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:16:28.524173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:28.524173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:16:28.524173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:16:28.524173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:16:28.524173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:16:28.524173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:16:28.524173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:16:28.524173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:16:28.524173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:16:28.524173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:16:28.524173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:16:28.524173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:16:28.524173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:16:28.524173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:16:28.524173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:16:28.524173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:16:28.524672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:16:28.524672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:16:28.524672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:16:28.524672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:16:28.524672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:16:28.524672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:28.524672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:16:28.524672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:28.524672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:16:28.524672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:28.524672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:16:28.524672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:28.524672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:16:28.524672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:28.524672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:16:28.524672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:16:28.524672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:28.524672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:16:28.525380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:28.525380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:16:28.525380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:16:28.525380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:28.525380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:16:28.525380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:28.525380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:16:28.526276 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:16:28.526665 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:16:28.526810 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:16:28.526976 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:16:28.527495 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:16:28.527864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:28.528674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:16:28.528674 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:28.528674 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:16:28.528674 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:16:28.528674 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:28.528674 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:16:28.528674 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:28.528674 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:16:28.529871 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:28.529871 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:16:28.529871 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:16:28.529871 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:16:28.529871 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:28.529871 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:16:28.529871 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:28.529871 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:16:28.529871 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:28.529871 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:16:28.529871 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:28.529871 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:16:28.529871 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:28.529871 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:16:28.529871 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:16:28.530333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:28.530333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:16:28.530333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:28.530333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:16:28.530333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:16:28.530333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:28.530333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:16:28.530333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:28.530333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:16:28.531172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:28.531172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:16:28.531172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:16:28.531172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:16:28.531172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:16:28.531172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:16:28.531172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:16:28.531172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:16:28.531172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:16:28.531172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:28.531172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:16:28.531596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:28.531596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:16:28.531596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:28.531596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:16:28.531596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:28.531596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:16:28.531596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:28.531596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:16:28.531596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:16:28.531596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:28.531596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:16:28.531596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:28.531596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:16:28.531596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:16:28.532249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:28.532249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:16:28.532249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:28.532249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:16:28.532858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:28.532858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:16:28.532858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:16:28.532858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:16:28.532858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:16:28.532858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:16:28.532858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:16:28.532858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:16:28.532858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:16:28.532858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:16:28.532858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:16:28.532858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:16:28.532858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:16:28.532858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:16:28.532858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:16:28.532858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:16:28.532858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:16:28.533394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:16:28.533394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:16:28.533394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:16:28.533394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:28.533394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:16:28.533394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:28.533394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:16:28.533394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:28.533394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:16:28.533394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:28.533394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:16:28.533394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:28.533394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:16:28.533394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:16:28.533394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:28.533394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:16:28.533394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:28.533394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:16:28.533394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:16:28.533951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:28.533951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:16:28.533951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:28.533951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:16:28.540757 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:16:28.541615 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:16:28.542147 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:16:28.542147 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:16:28.542641 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:16:28.542992 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:16:28.543254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:28.544007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:16:28.544007 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:28.544007 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:16:28.544007 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:16:28.544007 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:28.544007 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:16:28.544007 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:28.544007 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:16:28.545528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:28.545528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:16:28.545528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:16:28.545528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:16:28.545528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:16:28.545528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:16:28.545528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:16:28.545528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:28.545528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:16:28.545528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:28.545528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:16:28.545528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:28.545951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:16:28.545951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:28.545951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:16:28.545951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:28.545951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:16:28.545951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:16:28.545951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:28.545951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:16:28.545951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:28.545951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:16:28.545951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:16:28.545951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:28.545951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:16:28.546298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:28.546298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:16:28.553214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:28.553214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:16:28.553214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:16:28.553214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:16:28.553214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:16:28.553214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:16:28.553214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:16:28.553214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:16:28.553214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:16:28.553214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:16:28.553214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:16:28.553214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:16:28.553214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:16:28.553214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:16:28.553214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:16:28.553214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:16:28.554137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:16:28.554137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:16:28.554137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:28.554137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:16:28.554137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:28.554137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:16:28.554137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:28.554137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:16:28.554137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:28.554137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:16:28.554137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:28.554137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:16:28.554137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:16:28.554137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:28.554137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:16:28.554137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:28.554137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:16:28.554137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:16:28.554137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:28.555428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:16:28.555428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:28.555428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:16:32.950656 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:16:32.951414 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:16:32.951414 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:16:32.952114 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:16:32.994853 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:16:32.995221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:32.995946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:16:32.995946 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:32.995946 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:16:32.995946 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:16:32.995946 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:32.995946 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:16:32.995946 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:32.995946 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:16:32.996679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.045 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:32.996679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:16:32.996679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:16:32.996679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:16:32.996679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:16:32.996679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:16:32.996679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:16:32.996679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:32.996679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:16:32.996679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:32.996679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:16:32.996679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:32.997107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:16:32.997107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:32.997107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:16:32.997107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:32.997107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:16:32.997107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:16:32.997107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:32.997107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:16:32.997107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:32.997107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:16:32.997107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:16:32.997107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:32.997107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:16:32.997818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:32.997818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:16:32.997818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:32.997818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:16:32.997818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:16:32.997818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:16:32.997818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:16:32.997818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:16:32.997818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:16:32.997818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:16:32.997818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:16:32.997818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:16:32.997818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:16:32.997818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:16:32.997818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:16:32.998588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:16:32.998588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:16:32.998588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:16:32.998588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:16:32.998588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:16:32.998588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:32.998588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:16:32.998588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:32.998588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:16:32.998588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:32.998588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:16:32.998588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:32.998588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:16:32.998588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:32.998588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:16:32.998588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:16:32.998588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:32.998588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:16:32.998588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:32.999654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:16:32.999654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:16:32.999654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:32.999654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:16:32.999654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:32.999654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:16:33.510272 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:16:33.510541 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:16:33.510596 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:16:33.512631 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:16:33.512631 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:16:33.512631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:33.512883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:16:33.512883 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:33.512883 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:16:33.512883 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:16:33.512883 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:33.512883 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:16:33.512883 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:33.512883 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:16:33.514628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:33.514628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:16:33.514628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:16:33.514628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:16:33.514628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:16:33.514628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:16:33.514628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:16:33.514628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:33.514628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:16:33.514628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:33.514628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:16:33.514628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:33.515033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:16:33.515033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:33.515033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:16:33.515033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:33.515033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:16:33.515033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:16:33.515033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:33.515033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:16:33.515033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:33.515033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:16:33.515033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:16:33.515033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:33.515033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:16:33.516571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:33.516571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:16:33.516571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:33.516571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:16:33.516571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:16:33.516571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:16:33.516571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:16:33.516571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:16:33.516571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:16:33.516571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:16:33.516571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:16:33.516571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:16:33.516571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:16:33.516571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:16:33.516571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:16:33.519065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:16:33.519065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:16:33.519065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:16:33.519065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:16:33.519065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:16:33.519065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:33.519065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:16:33.519065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:33.519065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:16:33.519065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:33.519065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:16:33.519065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:33.519065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:16:33.519065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:33.519065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:16:33.519065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:16:33.519065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:33.519065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:16:33.519065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:33.519828 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:16:33.519828 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:16:33.519828 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:33.519828 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:16:33.519828 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:33.519828 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:16:33.519828 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:16:33.522893 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:16:33.522893 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:16:33.522893 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:16:33.523069 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:16:33.523344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:33.524075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:16:33.524075 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:33.524075 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:16:33.524075 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:16:33.524075 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:33.524075 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:16:33.524075 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:33.524075 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:16:33.525328 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:33.525328 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:16:33.525328 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:16:33.525328 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:16:33.525328 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:33.525328 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:16:33.525328 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:33.525328 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:16:33.525328 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:33.525328 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:16:33.525328 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:33.525328 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:16:33.525328 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:33.525328 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:16:33.525328 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:16:33.525790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:33.525790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:16:33.525790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:33.525790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:16:33.525790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:16:33.525790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:33.525790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:16:33.525790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:33.525790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:16:33.526135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:33.526135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:16:33.526135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:16:33.526135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:16:33.526135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:16:33.526135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:16:33.526135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:16:33.526135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:16:33.526135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:16:33.526135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:16:33.526135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:33.526135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:16:33.526550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:33.526550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:16:33.526550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:33.526550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:16:33.526550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:33.526550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:16:33.526550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:33.526550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:16:33.526550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:16:33.526550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:33.526550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:16:33.526550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:33.526550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:16:33.526550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:16:33.526988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:33.526988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:16:33.526988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:33.526988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:16:33.527608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:33.527608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:16:33.527608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:16:33.527608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:16:33.527608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:16:33.527608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:16:33.527608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:16:33.527608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:16:33.527608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:16:33.527608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:16:33.527608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:16:33.527608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:16:33.527608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:16:33.527608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:16:33.527608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:16:33.527608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:16:33.528104 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:16:33.528104 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:16:33.528104 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:16:33.528104 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:16:33.528104 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:16:33.528104 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:33.528104 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:16:33.528104 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:33.528104 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:16:33.528104 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:33.528104 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:16:33.528104 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:33.528104 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:16:33.528104 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:33.528104 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:16:33.528104 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:16:33.528104 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:33.528104 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:16:33.528609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:33.528609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:16:33.528609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:16:33.528609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:33.528609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:16:33.528609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:33.528609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:16:33.528609 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:16:33.528609 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:16:33.528609 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:16:33.529026 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:16:33.529026 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:16:33.529174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:33.530111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:16:33.530111 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:33.530111 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:16:33.530111 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:16:33.530111 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:33.530111 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:16:33.530111 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:33.530111 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:16:33.530972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:33.530972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:16:33.530972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:16:33.530972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:16:33.530972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:33.530972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:16:33.530972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:33.530972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:16:33.530972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:33.530972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:16:33.530972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:33.530972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:16:33.530972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:33.530972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:16:33.530972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:16:33.531419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:33.531419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:16:33.531419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:33.531419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:16:33.531419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:16:33.531419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:33.531419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:16:33.531419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:33.531419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:16:33.531810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:33.531810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:16:33.531810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:16:33.531810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:16:33.531810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:16:33.531810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:16:33.531810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:16:33.531810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:16:33.531810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:16:33.531810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:33.531810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:16:33.532170 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:33.532170 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:16:33.532170 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:33.532170 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:16:33.532170 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:33.532170 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:16:33.532170 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:33.532170 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:16:33.532170 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:16:33.532170 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:33.532170 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:16:33.532170 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:33.532170 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:16:33.532170 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:16:33.532663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:33.532663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:16:33.532663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:33.532663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:16:33.532663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:33.532663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:16:33.532663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:16:33.532663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:16:33.532663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:16:33.532663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:16:33.532663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:16:33.532663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:16:33.532663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:16:33.532663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:16:33.535058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:16:33.535058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:16:33.535058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:16:33.535058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:16:33.535058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:16:33.535058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:16:33.535058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:16:33.535058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:16:33.535058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:16:33.535058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:16:33.535058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:33.535058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:16:33.535058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:33.535058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:16:33.535058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:33.535058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:16:33.535058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:33.535058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:16:33.535058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:33.536044 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:16:33.536044 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:16:33.536044 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:33.536044 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:16:33.536044 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:33.536044 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:16:33.536044 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:16:33.536044 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:33.536044 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:16:33.536044 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:33.536044 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:16:33.541438 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:16:33.541851 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:16:33.542067 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:16:33.542345 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:16:33.542809 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:16:33.543079 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:16:33.543240 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:33.543901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:16:33.543901 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:33.543901 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:16:33.543901 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:16:33.543901 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:33.543901 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:16:33.543901 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:33.543901 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:16:33.545362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:33.545362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:16:33.545362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:16:33.545362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:16:33.545362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:16:33.545362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:16:33.545362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:16:33.545362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:33.545362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:16:33.545362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:33.545362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:16:33.545362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:33.545786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:16:33.545786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:33.545786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:16:33.545786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:33.545786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:16:33.545786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:16:33.545786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:33.545786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:16:33.545786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:33.545786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:16:33.545786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:16:33.545786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:33.545786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:16:33.546316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:33.546316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:16:33.547677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:33.547677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:16:33.547677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:16:33.547677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:16:33.547677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:16:33.547677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:16:33.547677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:16:33.547677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:16:33.547677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:16:33.547677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:16:33.547677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:16:33.547677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:16:33.547677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:16:33.547677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:16:33.547677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:16:33.547677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:16:33.548434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:16:33.548434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:16:33.548434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:33.548434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:16:33.548434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:33.548434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:16:33.548434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:33.548434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:16:33.548434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:33.548434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:16:33.548434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:33.548434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:16:33.548434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:16:33.548434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:33.548434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:16:33.548434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:33.548434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:16:33.548434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:16:33.548434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:33.549291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:16:33.549291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:33.549291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:16:37.951140 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:16:37.951850 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:16:37.951850 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:16:37.952984 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:16:37.992004 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:16:37.992337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:37.992590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:16:37.992590 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:37.992590 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:16:37.992590 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:16:37.992590 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:37.992590 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:16:37.992590 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:37.992590 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:16:37.993603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:37.993603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:16:37.993603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:16:37.993603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:16:37.993603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:16:37.993603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:16:37.993603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:16:37.993603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:37.993603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:16:37.993603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:37.993603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:16:37.993603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:37.994460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:16:37.994460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:37.994460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:16:37.994460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:37.994460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:16:37.994460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:16:37.994460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:37.994460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:16:37.994460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:37.994460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:16:37.994460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:16:37.994460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:37.994460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:16:37.994948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:37.994948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:16:37.994948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:37.994948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:16:37.994948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:16:37.994948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:16:37.994948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:16:37.994948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:16:37.994948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:16:37.994948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:16:37.994948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:16:37.994948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:16:37.994948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:16:37.994948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:16:37.994948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:16:37.995610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:16:37.995610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:16:37.995610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:16:37.995610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:16:37.995610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:16:37.995610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:37.995610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:16:37.995610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:37.995610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:16:37.995610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:37.995610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:16:37.995610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:37.995610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:16:37.995610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:37.995610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:16:37.995610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:16:37.995610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:37.995610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:16:37.995610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:37.996384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:16:37.996384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:16:37.996384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:37.996384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:16:37.996384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:37.996384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:16:38.510929 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:16:38.511361 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:16:38.511483 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:16:38.512360 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:16:38.512654 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:16:38.512935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:38.513709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:16:38.513709 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:38.513709 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:16:38.513709 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:16:38.513709 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:38.513709 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:16:38.513709 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:38.513709 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:16:38.514894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:38.514894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:16:38.514894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:16:38.514894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:16:38.514894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:16:38.514894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:16:38.514894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:16:38.514894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:38.514894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:16:38.514894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:38.514894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:16:38.514894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:38.515286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:16:38.515286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:38.515286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:16:38.515286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:38.515286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:16:38.515286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:16:38.515286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:38.515286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:16:38.515286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:38.515286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:16:38.515286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:16:38.515286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:38.515286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:16:38.515659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:38.515659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:16:38.516313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:38.516313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:16:38.516313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:16:38.516313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:16:38.516313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:16:38.516313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:16:38.516313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:16:38.516313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:16:38.516313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:16:38.516313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:16:38.516313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:16:38.516313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:16:38.516313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:16:38.516313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:16:38.516313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:16:38.516313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:16:38.516898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:16:38.516898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:16:38.516898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:38.516898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:16:38.516898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:38.516898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:16:38.516898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:38.516898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:16:38.516898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:38.516898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:16:38.516898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:38.516898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:16:38.516898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:16:38.516898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:38.516898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:16:38.516898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:38.516898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:16:38.516898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:16:38.516898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:38.517423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:16:38.517423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:38.517423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:16:38.522104 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:16:38.522572 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:16:38.522675 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:16:38.522827 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:16:38.523323 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:16:38.523647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:38.524430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:16:38.524430 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:38.524430 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:16:38.524430 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:16:38.524430 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:38.524430 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:16:38.524430 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:38.524430 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:16:38.525656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:38.525656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:16:38.525656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:16:38.525656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:16:38.525656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:38.525656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:16:38.525656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:38.525656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:16:38.525656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:38.525656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:16:38.525656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:38.525656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:16:38.525656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:38.525656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:16:38.525656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:16:38.526148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:38.526148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:16:38.526148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:38.526148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:16:38.526148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:16:38.526148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:38.526148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:16:38.526148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:38.526148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:16:38.526630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:38.526630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:16:38.526630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:16:38.526630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:16:38.526630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:16:38.526630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:16:38.526630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:16:38.526630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:16:38.526630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:16:38.526630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:16:38.526630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:38.526630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:16:38.527137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:38.527137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:16:38.527137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:38.527137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:16:38.527137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:38.527137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:16:38.527137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:38.527137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:16:38.527137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:16:38.527137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:38.527137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:16:38.527137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:38.527137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:16:38.527137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:16:38.527483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:38.527483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:16:38.527483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:38.527483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:16:38.527483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:38.527483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:16:38.527483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:16:38.527483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:16:38.527483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:16:38.527483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:16:38.527483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:16:38.527483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:16:38.527483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:16:38.527483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:16:38.527935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:16:38.527935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:16:38.527935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:16:38.527935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:16:38.527935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:16:38.527935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:16:38.527935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:16:38.527935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:16:38.527935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:16:38.527935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:16:38.527935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:16:38.527935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:38.527935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:16:38.527935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:38.527935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:16:38.527935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:38.527935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:16:38.527935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:38.527935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:16:38.528422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:38.528422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:16:38.528422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:16:38.528422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:38.528422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:16:38.528422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:38.528422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:16:38.528422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:16:38.528422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:38.528422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:16:38.528422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:38.528422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:16:38.528422 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:16:38.529094 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:16:38.529931 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:16:38.530063 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:16:38.530588 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:16:38.531001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:38.531759 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:16:38.531759 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:38.531759 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:16:38.531759 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:16:38.531759 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:38.531759 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:16:38.531759 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:38.531759 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:16:38.532708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:38.532708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:16:38.532708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:16:38.532708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:16:38.532708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:38.532708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:16:38.532708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:38.532708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:16:38.532708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:38.532708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:16:38.532708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:38.532708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:16:38.532708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:38.532708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:16:38.532708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:16:38.533139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:38.533139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:16:38.533139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:38.533139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:16:38.533139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:16:38.533139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:38.533139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:16:38.533139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:38.533139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:16:38.533571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:38.533571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:16:38.533571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:16:38.533571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:16:38.533571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:16:38.533571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:16:38.533571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:16:38.533571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:16:38.533571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:16:38.533571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:38.533571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:16:38.533940 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:38.533940 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:16:38.533940 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:38.533940 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:16:38.533940 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:38.533940 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:16:38.533940 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:38.533940 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:16:38.533940 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:16:38.533940 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:38.533940 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:16:38.533940 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:38.533940 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:16:38.533940 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:16:38.534288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:38.534288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:16:38.534288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:38.534288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:16:38.535279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:38.535279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:16:38.535279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:16:38.535279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:16:38.535279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:16:38.535279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:16:38.535279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:16:38.535279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:16:38.535279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:16:38.535279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:16:38.535279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:16:38.535279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:16:38.535279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:16:38.535279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:16:38.535279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:16:38.535279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:16:38.535279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:16:38.535904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:16:38.535904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:16:38.535904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:16:38.535904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:38.535904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:16:38.535904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:38.535904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:16:38.535904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:38.535904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:16:38.535904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:38.535904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:16:38.535904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:38.535904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:16:38.535904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:16:38.535904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:38.535904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:16:38.535904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:38.535904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:16:38.535904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:16:38.536377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:38.536377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:16:38.536377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:38.536377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:16:38.541872 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:16:38.542371 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:16:38.542494 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:16:38.542732 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:16:38.543075 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:16:38.543322 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:16:38.543494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:38.544142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:16:38.544142 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:38.544142 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:16:38.544142 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:16:38.544142 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:38.544142 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:16:38.544142 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:38.544142 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:16:38.545398 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:38.545398 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:16:38.545398 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:16:38.545398 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:16:38.545398 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:16:38.545398 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:16:38.545398 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:16:38.545398 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:38.545398 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:16:38.545398 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:38.545398 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:16:38.545398 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:38.546512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:16:38.546512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:38.546512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:16:38.546512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:38.546512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:16:38.546512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:16:38.546512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:38.546512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:16:38.546512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:38.546512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:16:38.546512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:16:38.546512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:38.546512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:16:38.547124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:38.547124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:16:38.547124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:38.547124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:16:38.547124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:16:38.547124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:16:38.547124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:16:38.547124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:16:38.547124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:16:38.547124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:16:38.547124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:16:38.547124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:16:38.547124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:16:38.547124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:16:38.547124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:16:38.548075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:16:38.548075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:16:38.548075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:16:38.548075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:16:38.548075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:16:38.548075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:38.548075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:16:38.548075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:38.548075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:16:38.548075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:38.548075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:16:38.548075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:38.548075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:16:38.548075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:38.548075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:16:38.548075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:16:38.548075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:38.548075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:16:38.548075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:38.549137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:16:38.549137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:16:38.549137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:38.549137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:16:38.549137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:38.549137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:16:42.951832 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:16:42.952919 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:16:42.952919 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:16:42.954114 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:16:42.993100 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:16:42.993532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:42.994466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:16:42.994466 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:42.994466 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:16:42.994466 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:16:42.994466 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:42.994466 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:16:42.994466 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:42.994466 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:16:42.995350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:42.995350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:16:42.995350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:16:42.995350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:16:42.995350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:16:42.995350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:16:42.995350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:16:42.995350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:42.995350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:16:42.995350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:42.995350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:16:42.995350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:42.995901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:16:42.995901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:42.995901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:16:42.995901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:42.995901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:16:42.995901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:16:42.995901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:42.995901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:16:42.995901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:42.995901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:16:42.995901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:16:42.995901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:42.995901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:16:42.996372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:42.996372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:16:42.996372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:42.996372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:16:42.996372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:16:42.996372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:16:42.996372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:16:42.996372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:16:42.996372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:16:42.996372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:16:42.996372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:16:42.996372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:16:42.996372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:16:42.996372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:16:42.996372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:16:42.996939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:16:42.996939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:16:42.996939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:16:42.996939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:16:42.996939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:16:42.996939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:42.996939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:16:42.996939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:42.996939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:16:42.996939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:42.996939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:16:42.996939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:42.996939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:16:42.996939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:42.996939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:16:42.996939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:16:42.996939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:42.996939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:16:42.996939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:42.997600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:16:42.997600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:16:42.997600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:42.997600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:16:42.997600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:42.997600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:16:43.511491 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:16:43.512119 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:16:43.512331 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:16:43.513875 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:16:43.514799 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:16:43.514982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:43.516100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:16:43.516100 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:43.516100 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:16:43.516100 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:16:43.516100 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:43.516100 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:16:43.516100 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:43.516100 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:16:43.517877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:43.517877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:16:43.517877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:16:43.517877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:16:43.517877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:16:43.517877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:16:43.517877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:16:43.517877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:43.517877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:16:43.517877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:43.517877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:16:43.517877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:43.518250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:16:43.518250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:43.518250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:16:43.518250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:43.518250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:16:43.518250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:16:43.518250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:43.518250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:16:43.518250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:43.518250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:16:43.518250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:16:43.518250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:43.518250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:16:43.518622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:43.518622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:16:43.519403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:43.519403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:16:43.519403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:16:43.519403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:16:43.519403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:16:43.519403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:16:43.519403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:16:43.519403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:16:43.519403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:16:43.519403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:16:43.519403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:16:43.519403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:16:43.519403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:16:43.519403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:16:43.519403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:16:43.519403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:16:43.519876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:16:43.519876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:16:43.519876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:43.519876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:16:43.519876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:43.519876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:16:43.519876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:43.519876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:16:43.519876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:43.519876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:16:43.519876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:43.519876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:16:43.519876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:16:43.519876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:43.519876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:16:43.519876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:43.519876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:16:43.519876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:16:43.519876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:43.520334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:16:43.520334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:43.520334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:16:43.522650 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:16:43.522983 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:16:43.523113 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:16:43.523231 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:16:43.524015 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:16:43.524311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:43.525009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:16:43.525009 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:43.525009 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:16:43.525009 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:16:43.525009 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:43.525009 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:16:43.525009 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:43.525009 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:16:43.525754 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:43.525754 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:16:43.525754 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:16:43.525754 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:16:43.525754 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:43.525754 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:16:43.525754 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:43.525754 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:16:43.525754 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:43.525754 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:16:43.525754 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:43.525754 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:16:43.525754 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:43.525754 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:16:43.525754 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:16:43.526247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:43.526247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:16:43.526247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:43.526247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:16:43.526247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:16:43.526247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:43.526247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:16:43.526247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:43.526247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:16:43.526628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:43.526628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:16:43.526628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:16:43.526628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:16:43.526628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:16:43.526628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:16:43.526628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:16:43.526628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:16:43.526628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:16:43.526628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:16:43.526628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:43.526628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:16:43.527126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:43.527126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:16:43.527126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:43.527126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:16:43.527126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:43.527126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:16:43.527126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:43.527126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:16:43.527126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:16:43.527126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:43.527126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:16:43.527126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:43.527126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:16:43.527126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:16:43.527674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:43.527674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:16:43.527674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:43.527674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:16:43.527674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:43.527674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:16:43.527674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:16:43.527674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:16:43.527674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:16:43.527674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:16:43.527674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:16:43.527674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:16:43.527674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:16:43.527674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:16:43.528989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:16:43.528989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:16:43.528989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:16:43.528989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:16:43.528989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:16:43.528989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:16:43.528989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:16:43.528989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:16:43.528989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:16:43.528989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:16:43.528989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:16:43.528989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:43.528989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:16:43.528989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:43.528989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:16:43.528989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:43.528989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:16:43.528989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:43.528989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:16:43.529734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:43.529734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:16:43.529734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:16:43.529734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:43.529734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:16:43.529734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:43.529734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:16:43.529734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:16:43.529734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:43.529734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:16:43.529734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:43.529734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:16:43.529734 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:16:43.529734 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:16:43.530302 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:16:43.530302 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:16:43.530569 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:16:43.530835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:43.531445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:16:43.531445 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:43.531445 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:16:43.531445 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:16:43.531445 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:43.531445 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:16:43.531445 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:43.531445 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:16:43.532238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:43.532238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:16:43.532238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:16:43.532238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:16:43.532238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:43.532238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:16:43.532238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:43.532238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:16:43.532238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:43.532238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:16:43.532238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:43.532238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:16:43.532238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:43.532238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:16:43.532238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:16:43.533327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:43.533327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:16:43.533327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:43.533327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:16:43.533327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:16:43.533327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:43.533327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:16:43.533327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:43.533327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:16:43.533327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:43.533327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:16:43.533327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:16:43.533327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:16:43.533327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:16:43.534629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:16:43.534629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:16:43.534629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:16:43.534629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:16:43.534629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:43.534629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:16:43.534629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:43.534629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:16:43.534629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:43.534629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:16:43.534629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:43.534629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:16:43.534629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:43.535124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:16:43.535124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:16:43.535124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:43.535124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:16:43.535124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:43.535124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:16:43.535124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:16:43.535124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:43.535124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:16:43.535124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:43.535124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:16:43.535124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:43.535507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:16:43.535507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:16:43.535507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:16:43.535507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:16:43.535507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:16:43.535507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:16:43.535507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:16:43.535507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:16:43.535507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:16:43.535507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:16:43.535507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:16:43.535507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:16:43.535507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:16:43.535507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:16:43.535507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:16:43.535507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:16:43.535507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:16:43.535507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:16:43.535507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:16:43.536102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:43.536102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:16:43.536102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:43.536102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:16:43.536102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:43.536102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:16:43.536102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:43.536102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:16:43.536102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:43.536102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:16:43.536102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:16:43.536102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:43.536102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:16:43.536102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:43.536102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:16:43.536102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:16:43.536102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:43.536102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:16:43.536102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:43.536102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:16:43.542529 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:16:43.542877 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:16:43.542983 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:16:43.543181 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:16:43.543622 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:16:43.544006 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:16:43.544063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:43.544642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:16:43.544642 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:43.544642 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:16:43.544642 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:16:43.544642 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:43.544642 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:16:43.544642 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:43.544642 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:16:43.545757 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:43.545757 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:16:43.545757 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:16:43.545757 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:16:43.545757 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:16:43.545757 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:16:43.545757 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:16:43.545757 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:43.545757 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:16:43.545757 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:43.545757 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:16:43.545757 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:43.546375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:16:43.546375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:43.546375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:16:43.546375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:43.546375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:16:43.546375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:16:43.546375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:43.546375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:16:43.546375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:43.546375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:16:43.546375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:16:43.546375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:43.546375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:16:43.546968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:43.546968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:16:43.547406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:43.547406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:16:43.547406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:16:43.547406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:16:43.547406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:16:43.547406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:16:43.547406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:16:43.547406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:16:43.547406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:16:43.547406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:16:43.547406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:16:43.547406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:16:43.547406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:16:43.547406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:16:43.547406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:16:43.547406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:16:43.547974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:16:43.547974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:16:43.547974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:43.547974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:16:43.547974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:43.547974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:16:43.547974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:43.547974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:16:43.547974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:43.547974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:16:43.547974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:43.547974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:16:43.547974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:16:43.547974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:43.547974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:16:43.547974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:43.547974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:16:43.547974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:16:43.547974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:43.548445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:16:43.548445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:43.548445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:16:47.952601 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:16:47.953486 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:16:47.953486 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:16:47.954732 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:16:47.996677 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:16:47.997051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:47.997907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:16:47.997907 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:47.997907 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:16:47.997907 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:16:47.997907 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:47.997907 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:16:47.997907 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:47.997907 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:16:47.999267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.045 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:47.999267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:16:47.999267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:16:47.999267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:16:47.999267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:16:47.999267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:16:47.999267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:16:47.999267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:47.999267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:16:47.999267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:47.999267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:16:47.999267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:48.000062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:16:48.000062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:48.000062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:16:48.000062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:48.000062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:16:48.000062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:16:48.000062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:48.000062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:16:48.000062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:48.000062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:16:48.000062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:16:48.000062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:48.000062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:16:48.000620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:48.000620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:16:48.000620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:48.000620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:16:48.000620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:16:48.000620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:16:48.000620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:16:48.000620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:16:48.000620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:16:48.000620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:16:48.000620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:16:48.000620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:16:48.000620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:16:48.000620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:16:48.000620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:16:48.001300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:16:48.001300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:16:48.001300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:16:48.001300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:16:48.001300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:16:48.001300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:48.001300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:16:48.001300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:48.001300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:16:48.001300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:48.001300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:16:48.001300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:48.001300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:16:48.001300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:48.001300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:16:48.001300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:16:48.001300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:48.001300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:16:48.001300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:48.002207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:16:48.002207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:16:48.002207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:48.002207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:16:48.002207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:48.002207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:16:48.512171 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:16:48.512558 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:16:48.512679 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:16:48.513429 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:16:48.513694 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:16:48.513882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:48.514523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:16:48.514523 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:48.514523 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:16:48.514523 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:16:48.514523 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:48.514523 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:16:48.514523 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:48.514523 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:16:48.515931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:48.515931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:16:48.515931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:16:48.515931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:16:48.515931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:16:48.515931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:16:48.515931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:16:48.515931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:48.515931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:16:48.515931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:48.515931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:16:48.515931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:48.516788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:16:48.516788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:48.516788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:16:48.516788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:48.516788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:16:48.516788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:16:48.516788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:48.516788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:16:48.516788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:48.516788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:16:48.516788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:16:48.516788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:48.516788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:16:48.517657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:48.517657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:16:48.518106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:48.518106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:16:48.518106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:16:48.518106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:16:48.518106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:16:48.518106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:16:48.518106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:16:48.518106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:16:48.518106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:16:48.518106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:16:48.518106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:16:48.518106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:16:48.518106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:16:48.518106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:16:48.518106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:16:48.518106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:16:48.519004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:16:48.519004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:16:48.519004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:48.519004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:16:48.519004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:48.519004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:16:48.519004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:48.519004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:16:48.519004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:48.519004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:16:48.519004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:48.519004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:16:48.519004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:16:48.519004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:48.519004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:16:48.519004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:48.519004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:16:48.519004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:16:48.519004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:48.519969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:16:48.519969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:48.519969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:16:48.523046 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:16:48.523337 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:16:48.523443 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:16:48.523570 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:16:48.523921 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:16:48.524182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:48.524724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:16:48.524724 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:48.524724 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:16:48.524724 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:16:48.524724 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:48.524724 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:16:48.524724 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:48.524724 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:16:48.525493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:48.525493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:16:48.525493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:16:48.525493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:16:48.525493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:48.525493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:16:48.525493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:48.525493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:16:48.525493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:48.525493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:16:48.525493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:48.525493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:16:48.525493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:48.525493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:16:48.525493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:16:48.526238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:48.526238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:16:48.526238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:48.526238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:16:48.526238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:16:48.526238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:48.526238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:16:48.526238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:48.526238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:16:48.526646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:48.526646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:16:48.526646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:16:48.526646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:16:48.526646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:16:48.526646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:16:48.526646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:16:48.526646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:16:48.526646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:16:48.526646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:16:48.526646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:48.526646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:16:48.527255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:48.527255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:16:48.527255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:48.527255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:16:48.527255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:48.527255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:16:48.527255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:48.527255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:16:48.527255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:16:48.527255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:48.527255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:16:48.527255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:48.527255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:16:48.527255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:16:48.527853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:48.527853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:16:48.527853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:48.527853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:16:48.527853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:48.527853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:16:48.527853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:16:48.527853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:16:48.527853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:16:48.527853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:16:48.527853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:16:48.527853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:16:48.527853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:16:48.527853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:16:48.528487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:16:48.528487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:16:48.528487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:16:48.528487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:16:48.528487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:16:48.528487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:16:48.528487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:16:48.528487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:16:48.528487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:16:48.528487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:16:48.528487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:16:48.528487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:48.528487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:16:48.528487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:48.528487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:16:48.528487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:48.528487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:16:48.528487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:48.528487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:16:48.529284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:48.529284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:16:48.529284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:16:48.529284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:48.529284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:16:48.529284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:48.529284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:16:48.529284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:16:48.529284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:48.529284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:16:48.529284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:48.529284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:16:48.529683 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:16:48.529969 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:16:48.530382 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:16:48.530569 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:16:48.531012 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:16:48.531362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:48.532087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:16:48.532087 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:48.532087 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:16:48.532087 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:16:48.532087 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:48.532087 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:16:48.532087 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:48.532087 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:16:48.532910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:48.532910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:16:48.532910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:16:48.532910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:16:48.532910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:48.532910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:16:48.532910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:48.532910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:16:48.532910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:48.532910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:16:48.532910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:48.532910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:16:48.532910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:48.532910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:16:48.532910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:16:48.533548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:48.533548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:16:48.533548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:48.533548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:16:48.533548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:16:48.533548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:48.533548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:16:48.533548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:48.533548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:16:48.533927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:48.533927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:16:48.533927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:16:48.533927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:16:48.533927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:16:48.533927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:16:48.533927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:16:48.533927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:16:48.533927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:16:48.533927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:48.533927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:16:48.534604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:48.534604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:16:48.534604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:48.534604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:16:48.534604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:48.534604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:16:48.534604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:48.534604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:16:48.534604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:16:48.534604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:48.534604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:16:48.534604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:48.534604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:16:48.534604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:16:48.535193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:48.535193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:16:48.535193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:48.535193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:16:48.535193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:48.535193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:16:48.535193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:16:48.535193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:16:48.535193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:16:48.535193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:16:48.535193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:16:48.535193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:16:48.535193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:16:48.535193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:16:48.535796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:16:48.535796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:16:48.535796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:16:48.535796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:16:48.535796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:16:48.535796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:16:48.535796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:16:48.535796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:16:48.535796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:16:48.535796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:16:48.535796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:48.535796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:16:48.535796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:48.535796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:16:48.535796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:48.535796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:16:48.535796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:48.535796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:16:48.535796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:48.536589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:16:48.536589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:16:48.536589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:48.536589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:16:48.536589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:48.536589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:16:48.536589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:16:48.536589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:48.536589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:16:48.536589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:48.536589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:16:48.543110 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:16:48.543316 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:16:48.543440 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:16:48.543603 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:16:48.544072 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:16:48.544364 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:16:48.544558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:48.545224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:16:48.545224 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:48.545224 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:16:48.545224 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:16:48.545224 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:48.545224 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:16:48.545224 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:48.545224 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:16:48.546934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:48.546934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:16:48.546934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:16:48.546934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:16:48.546934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:16:48.546934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:16:48.546934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:16:48.546934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:48.546934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:16:48.546934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:48.546934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:16:48.546934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:48.547304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:16:48.547304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:48.547304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:16:48.547304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:48.547304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:16:48.547304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:16:48.547304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:48.547304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:16:48.547304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:48.547304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:16:48.547304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:16:48.547304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:48.547304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:16:48.547641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:48.547641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:16:48.548957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:48.548957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:16:48.548957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:16:48.548957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:16:48.548957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:16:48.548957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:16:48.548957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:16:48.548957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:16:48.548957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:16:48.548957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:16:48.548957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:16:48.548957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:16:48.548957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:16:48.548957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:16:48.548957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:16:48.548957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:16:48.549395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:16:48.549395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:16:48.549395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:48.549395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:16:48.549395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:48.549395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:16:48.549395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:48.549395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:16:48.549395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:48.549395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:16:48.549395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:48.549395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:16:48.549395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:16:48.549395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:48.549395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:16:48.549395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:48.549395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:16:48.549395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:16:48.549395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:48.549889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:16:48.549889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:48.549889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:16:52.953307 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:16:52.957867 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:16:52.957867 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:16:52.957867 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:16:52.961155 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG dbcounter [-] [106314] Writing DB stats neutron:SELECT=260 {{(pid=106314) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 09 09:16:53.001363 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:16:53.001787 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:53.004113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:16:53.004113 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:53.004113 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:16:53.004113 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:16:53.004113 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:53.004113 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:16:53.004113 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:53.004113 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:16:53.005105 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.048 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:53.005105 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:16:53.005105 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:16:53.005105 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:16:53.005105 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:16:53.005105 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:16:53.005105 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:16:53.005105 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:53.005105 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:16:53.005105 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:53.005105 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:16:53.005105 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:53.005902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:16:53.005902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:53.005902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:16:53.005902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:53.005902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:16:53.005902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:16:53.005902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:53.005902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:16:53.005902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:53.005902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:16:53.005902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:16:53.005902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:53.005902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:16:53.006579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:53.006579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:16:53.006579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:53.006579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:16:53.006579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:16:53.006579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:16:53.006579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:16:53.006579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:16:53.006579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:16:53.006579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:16:53.006579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:16:53.006579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:16:53.006579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:16:53.006579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:16:53.006579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:16:53.007469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:16:53.007469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:16:53.007469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:16:53.007469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:16:53.007469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:16:53.007469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:53.007469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:16:53.007469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:53.007469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:16:53.007469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:53.007469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:16:53.007469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:53.007469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:16:53.007469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:53.007469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:16:53.007469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:16:53.007469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:53.007469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:16:53.007469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:53.009034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:16:53.009034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:16:53.009034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:53.009034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:16:53.009034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:53.009034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:16:53.512767 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:16:53.515786 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:16:53.515786 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:16:53.515917 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:16:53.516252 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:16:53.516469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:53.517161 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:16:53.517161 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:53.517161 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:16:53.517161 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:16:53.517161 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:53.517161 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:16:53.517161 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:53.517161 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:16:53.520033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:53.520033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:16:53.520033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:16:53.520033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:16:53.520033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:16:53.520033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:16:53.520033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:16:53.520033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:53.520033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:16:53.520033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:53.520033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:16:53.520033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:53.520446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:16:53.520446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:53.520446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:16:53.520446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:53.520446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:16:53.520446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:16:53.520446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:53.520446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:16:53.520446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:53.520446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:16:53.520446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:16:53.520446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:53.520446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:16:53.520711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:53.520711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:16:53.523525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:53.523525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:16:53.523525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:16:53.523525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:16:53.523525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:16:53.523525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:16:53.523525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:16:53.523525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:16:53.523525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:16:53.523525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:16:53.523525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:16:53.523525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:16:53.523525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:16:53.523525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:16:53.523525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:16:53.523525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:16:53.524248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:16:53.524248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:16:53.524248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:53.524248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:16:53.524248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:53.524248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:16:53.524248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:53.524248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:16:53.524248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:53.524248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:16:53.524248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:53.524248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:16:53.524248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:16:53.524248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:53.524248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:16:53.524248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:53.524248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:16:53.524248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:16:53.524248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:53.524817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:16:53.524817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:53.524817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:16:53.524817 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:16:53.524817 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:16:53.524817 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:16:53.525005 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:16:53.526896 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:16:53.527076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:53.528388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:16:53.528388 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:53.528388 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:16:53.528388 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:16:53.528388 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:53.528388 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:16:53.528388 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:53.528388 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:16:53.528935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:53.528935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:16:53.528935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:16:53.528935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:16:53.528935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:53.528935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:16:53.528935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:53.528935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:16:53.528935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:53.528935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:16:53.528935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:53.528935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:16:53.528935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:53.528935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:16:53.528935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:16:53.529639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:53.529639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:16:53.529639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:53.529639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:16:53.529639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:16:53.529639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:53.529639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:16:53.529639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:53.529639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:16:53.530438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:53.530438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:16:53.530438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:16:53.530438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:16:53.530438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:16:53.530438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:16:53.530438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:16:53.530438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:16:53.530438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:16:53.530438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:16:53.530438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:53.530438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:16:53.530903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:53.530903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:16:53.530903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:53.530903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:16:53.530903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:53.530903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:16:53.530903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:53.530903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:16:53.530903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:16:53.530903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:53.530903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:16:53.530903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:53.530903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:16:53.530903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:16:53.531878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:53.531878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:16:53.531878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:53.531878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:16:53.531878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:53.531878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:16:53.531878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:16:53.531878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:16:53.531878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:16:53.531878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:16:53.531878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:16:53.531878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:16:53.531878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:16:53.531878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:16:53.534903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:16:53.534903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:16:53.534903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:16:53.534903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:16:53.534903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:16:53.534903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:16:53.534903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:16:53.534903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:16:53.534903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:16:53.534903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:16:53.534903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:16:53.534903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:53.534903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:16:53.534903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:53.534903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:16:53.534903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:53.534903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:16:53.534903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:53.534903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:16:53.535824 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:53.535824 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:16:53.535824 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:16:53.535824 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:53.535824 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:16:53.535824 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:53.535824 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:16:53.535824 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:16:53.535824 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:53.535824 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:16:53.535824 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:53.535824 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:16:53.535824 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:16:53.535824 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:16:53.536244 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:16:53.536244 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:16:53.536244 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:16:53.536244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:53.536244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:16:53.536244 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:53.536244 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:16:53.536244 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:16:53.536244 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:53.536244 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:16:53.536244 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:53.536244 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:16:53.536601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:53.536601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:16:53.536601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:16:53.536601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:16:53.536601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:53.536601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:16:53.536601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:53.536601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:16:53.536601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:53.536601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:16:53.536601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:53.536601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:16:53.536601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:53.536601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:16:53.536601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:16:53.538369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:53.538369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:16:53.538369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:53.538369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:16:53.538369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:16:53.538369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:53.538369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:16:53.538369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:53.538369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:16:53.538369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:53.538369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:16:53.538369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:16:53.538369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:16:53.538369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:16:53.538835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:16:53.538835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:16:53.538835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:16:53.538835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:16:53.538835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:53.538835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:16:53.538835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:53.538835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:16:53.538835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:53.538835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:16:53.538835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:53.538835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:16:53.538835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:53.539368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:16:53.539368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:16:53.539368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:53.539368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:16:53.539368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:53.539368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:16:53.539368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:16:53.539368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:53.539368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:16:53.539368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:53.539368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:16:53.542094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:53.542094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:16:53.542094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:16:53.542094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:16:53.542094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:16:53.542094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:16:53.542094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:16:53.542094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:16:53.542094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:16:53.542094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:16:53.542094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:16:53.542094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:16:53.542094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:16:53.542094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:16:53.542094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:16:53.542094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:16:53.542094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:16:53.542440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:16:53.542440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:16:53.542440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:16:53.542440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:53.542440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:16:53.542440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:53.542440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:16:53.542440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:53.542440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:16:53.542440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:53.542440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:16:53.542440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:53.542440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:16:53.542440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:16:53.542440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:53.542440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:16:53.542440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:53.542440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:16:53.542440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:16:53.542796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:53.542796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:16:53.542796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:53.542796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:16:53.543463 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:16:53.545083 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:16:53.545083 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:16:53.545083 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:16:53.545083 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:16:53.545083 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:16:53.545083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:53.546553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:16:53.546553 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:53.546553 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:16:53.546553 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:16:53.546553 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:53.546553 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:16:53.546553 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:53.546553 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:16:53.549614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:53.549614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:16:53.549614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:16:53.549614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:16:53.549614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:16:53.549614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:16:53.549614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:16:53.549614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:53.549614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:16:53.549614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:53.549614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:16:53.549614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:53.550393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:16:53.550393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:53.550393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:16:53.550393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:53.550393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:16:53.550393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:16:53.550393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:53.550393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:16:53.550393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:53.550393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:16:53.550393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:16:53.550393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:53.550393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:16:53.552847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:53.552847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:16:53.552847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:53.552847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:16:53.552847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:16:53.552847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:16:53.552847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:16:53.552847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:16:53.552847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:16:53.552847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:16:53.552847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:16:53.552847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:16:53.552847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:16:53.552847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:16:53.552847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:16:53.553274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:16:53.553274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:16:53.553274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:16:53.553274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:16:53.553274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:16:53.553274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:53.553274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:16:53.553274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:53.553274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:16:53.553274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:53.553274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:16:53.553274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:53.553274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:16:53.553274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:53.553274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:16:53.553274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:16:53.553274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:53.553274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:16:53.553274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:53.553673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:16:53.553673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:16:53.553673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:53.553673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:16:53.553673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:53.553673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:16:57.956922 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:16:57.961823 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:16:57.961823 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:16:57.961823 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:16:57.991631 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:16:57.991986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:57.993817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:16:57.993817 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:57.993817 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:16:57.993817 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:16:57.993817 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:57.993817 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:16:57.993817 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:57.993817 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:16:57.993817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.032 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:57.993817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:16:57.993817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:16:57.993817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:16:57.993817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:16:57.993817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:16:57.993817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:16:57.994426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:57.994426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:16:57.994426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:57.994426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:16:57.994426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:57.994426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:16:57.994426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:57.994426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:16:57.994426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:57.994426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:16:57.994426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:16:57.994426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:57.994426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:16:57.994426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:57.994870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:16:57.994870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:16:57.994870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:57.994870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:16:57.994870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:57.994870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:16:57.994870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:57.994870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:16:57.994870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:16:57.994870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:16:57.994870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:16:57.994870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:16:57.994870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:16:57.994870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:16:57.995264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:16:57.995264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:16:57.995264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:16:57.995264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:16:57.995264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:16:57.995264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:16:57.995264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:16:57.995264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:16:57.995264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:16:57.995264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:16:57.995264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:57.995264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:16:57.995264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:57.995264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:16:57.995264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:57.995264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:16:57.995264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:57.995264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:16:57.995264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:57.995264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:16:57.995264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:16:57.995891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:57.995891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:16:57.995891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:57.995891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:16:57.995891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:16:57.995891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:57.995891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:16:57.995891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:57.995891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:16:58.513606 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:16:58.514322 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:16:58.514640 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:16:58.515396 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:16:58.515727 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:16:58.516027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:58.516897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:16:58.516897 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:58.516897 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:16:58.516897 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:16:58.516897 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:58.516897 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:16:58.516897 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:58.516897 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:16:58.518212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:58.518212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:16:58.518212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:16:58.518212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:16:58.518212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:16:58.518212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:16:58.518212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:16:58.518212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:58.518212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:16:58.518212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:58.518212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:16:58.518212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:58.518922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:16:58.518922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:58.518922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:16:58.518922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:58.518922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:16:58.518922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:16:58.518922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:58.518922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:16:58.518922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:58.518922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:16:58.518922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:16:58.518922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:58.518922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:16:58.519537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:58.519537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:16:58.519648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:58.519648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:16:58.519648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:16:58.519648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:16:58.519648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:16:58.519648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:16:58.519648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:16:58.519648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:16:58.519648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:16:58.519648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:16:58.519648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:16:58.519648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:16:58.519648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:16:58.519648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:16:58.519648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:16:58.519648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:16:58.520445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:16:58.520445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:16:58.520445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:58.520445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:16:58.520445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:58.520445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:16:58.520445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:58.520445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:16:58.520445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:58.520445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:16:58.520445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:58.520445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:16:58.520445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:16:58.520445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:58.520445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:16:58.520445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:58.520445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:16:58.520445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:16:58.520445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:58.521215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:16:58.521215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:58.521215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:16:58.524919 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:16:58.525309 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:16:58.525409 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:16:58.525541 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:16:58.526091 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:16:58.526382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:58.527190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:16:58.527190 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:58.527190 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:16:58.527190 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:16:58.527190 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:58.527190 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:16:58.527190 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:58.527190 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:16:58.528043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:58.528043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:16:58.528043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:16:58.528043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:16:58.528043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:58.528043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:16:58.528043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:58.528043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:16:58.528043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:58.528043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:16:58.528043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:58.528043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:16:58.528043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:58.528043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:16:58.528043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:16:58.528490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:58.528490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:16:58.528490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:58.528490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:16:58.528490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:16:58.528490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:58.528490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:16:58.528490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:58.528490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:16:58.528797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:58.528797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:16:58.528797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:16:58.528797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:16:58.528797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:16:58.528797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:16:58.528797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:16:58.528797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:16:58.528797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:16:58.528797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:16:58.528797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:58.528797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:16:58.529177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:58.529177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:16:58.529177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:58.529177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:16:58.529177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:58.529177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:16:58.529177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:58.529177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:16:58.529177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:16:58.529177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:58.529177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:16:58.529177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:58.529177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:16:58.529177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:16:58.529598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:58.529598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:16:58.529598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:58.529598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:16:58.529727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:58.529727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:16:58.529727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:16:58.529727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:16:58.529727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:16:58.529727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:16:58.529727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:16:58.529727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:16:58.529727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:16:58.529727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:16:58.529727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:16:58.529727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:16:58.529727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:16:58.529727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:16:58.529727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:16:58.529727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:16:58.530396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:16:58.530396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:16:58.530396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:16:58.530396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:16:58.530396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:16:58.530396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:58.530396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:16:58.530396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:58.530396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:16:58.530396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:58.530396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:16:58.530396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:58.530396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:16:58.530396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:58.530396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:16:58.530396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:16:58.530396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:58.530396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:16:58.531137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:58.531137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:16:58.531137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:16:58.531137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:58.531137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:16:58.531137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:58.531137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:16:58.531962 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:16:58.532280 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:16:58.532367 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:16:58.532481 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:16:58.532986 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:16:58.533367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:58.534059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:16:58.534059 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:58.534059 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:16:58.534059 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:16:58.534059 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:58.534059 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:16:58.534059 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:58.534059 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:16:58.534865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:58.534865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:16:58.534865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:16:58.534865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:16:58.534865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:58.534865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:16:58.534865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:58.534865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:16:58.534865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:58.534865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:16:58.534865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:58.534865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:16:58.534865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:58.534865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:16:58.534865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:16:58.535301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:58.535301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:16:58.535301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:58.535301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:16:58.535301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:16:58.535301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:58.535301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:16:58.535301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:58.535301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:16:58.535615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:58.535615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:16:58.535615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:16:58.535615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:16:58.535615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:16:58.535615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:16:58.535615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:16:58.535615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:16:58.535615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:16:58.535615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:58.535615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:16:58.535986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:58.535986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:16:58.535986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:58.535986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:16:58.535986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:58.535986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:16:58.535986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:58.535986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:16:58.535986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:16:58.535986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:58.535986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:16:58.535986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:58.535986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:16:58.535986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:16:58.536417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:58.536417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:16:58.536417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:58.536417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:16:58.536600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:58.536600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:16:58.536600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:16:58.536600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:16:58.536600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:16:58.536600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:16:58.536600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:16:58.536600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:16:58.536600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:16:58.536600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:16:58.536600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:16:58.536600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:16:58.536600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:16:58.536600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:16:58.536600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:16:58.536600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:16:58.536600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:16:58.537184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:16:58.537184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:16:58.537184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:16:58.537184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:58.537184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:16:58.537184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:58.537184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:16:58.537184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:58.537184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:16:58.537184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:58.537184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:16:58.537184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:58.537184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:16:58.537184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:16:58.537184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:58.537184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:16:58.537184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:58.537184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:16:58.537184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:16:58.537758 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:58.537758 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:16:58.537758 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:58.537758 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:16:58.544183 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:16:58.544990 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:16:58.545230 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:16:58.545618 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:16:58.546113 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:16:58.546487 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:16:58.546816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:58.547678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:16:58.547678 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:58.547678 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:16:58.547678 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:16:58.547678 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:58.547678 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:16:58.547678 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:58.547678 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:16:58.549220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:58.549220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:16:58.549220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:16:58.549220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:16:58.549220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:16:58.549220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:16:58.549220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:16:58.549220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:58.549220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:16:58.549220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:58.549220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:16:58.549220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:58.549573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:16:58.549573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:58.549573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:16:58.549573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:58.549573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:16:58.549573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:16:58.549573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:58.549573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:16:58.549573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:58.549573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:16:58.549573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:16:58.549573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:58.549573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:16:58.549966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:58.549966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:16:58.551100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:58.551100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:16:58.551100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:16:58.551100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:16:58.551100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:16:58.551100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:16:58.551100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:16:58.551100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:16:58.551100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:16:58.551100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:16:58.551100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:16:58.551100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:16:58.551100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:16:58.551100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:16:58.551100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:16:58.551100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:16:58.551593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:16:58.551593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:16:58.551593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:58.551593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:16:58.551593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:16:58.551593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:16:58.551593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:16:58.551593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:16:58.551593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:16:58.551593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:16:58.551593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:16:58.551593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:16:58.551593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:16:58.551593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:16:58.551593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:16:58.551593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:16:58.551593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:16:58.551593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:16:58.551593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:16:58.552131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:16:58.552131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:16:58.552131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:17:02.960274 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:17:02.961132 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:17:02.961132 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:17:02.962106 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:17:03.000254 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:17:03.000599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:03.001267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:17:03.001267 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:03.001267 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:17:03.001267 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:17:03.001267 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:03.001267 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:17:03.001267 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:03.001267 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:17:03.002074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:03.002074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:17:03.002074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:17:03.002074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:17:03.002074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:17:03.002074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:17:03.002074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:17:03.002074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:03.002074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:17:03.002074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:03.002074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:17:03.002074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:03.002614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:17:03.002614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:03.002614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:17:03.002614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:03.002614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:17:03.002614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:17:03.002614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:03.002614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:17:03.002614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:03.002614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:17:03.002614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:17:03.002614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:03.002614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:17:03.003206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:03.003206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:17:03.003206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:03.003206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:17:03.003206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:17:03.003206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:17:03.003206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:17:03.003206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:17:03.003206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:17:03.003206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:17:03.003206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:17:03.003206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:17:03.003206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:17:03.003206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:17:03.003206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:17:03.003833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:17:03.003833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:17:03.003833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:17:03.003833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:17:03.003833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:17:03.003833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:03.003833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:17:03.003833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:03.003833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:17:03.003833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:03.003833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:17:03.003833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:03.003833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:17:03.003833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:03.003833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:17:03.003833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:17:03.003833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:03.003833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:17:03.003833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:03.004627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:17:03.004627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:17:03.004627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:03.004627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:17:03.004627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:03.004627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:17:03.514341 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:17:03.515525 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:17:03.515734 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:17:03.517914 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:17:03.518134 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:17:03.518323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:03.518992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:17:03.518992 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:03.518992 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:17:03.518992 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:17:03.518992 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:03.518992 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:17:03.518992 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:03.518992 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:17:03.520826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:03.520826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:17:03.520826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:17:03.520826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:17:03.520826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:17:03.520826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:17:03.520826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:17:03.520826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:03.520826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:17:03.520826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:03.520826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:17:03.520826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:03.521420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:17:03.521420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:03.521420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:17:03.521420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:03.521420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:17:03.521420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:17:03.521420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:03.521420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:17:03.521420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:03.521420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:17:03.521420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:17:03.521420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:03.521420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:17:03.521937 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:03.521937 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:17:03.522335 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:03.522335 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:17:03.522335 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:17:03.522335 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:17:03.522335 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:17:03.522335 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:17:03.522335 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:17:03.522335 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:17:03.522335 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:17:03.522335 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:17:03.522335 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:17:03.522335 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:17:03.522335 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:17:03.522335 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:17:03.522335 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:17:03.522335 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:17:03.523057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:17:03.523057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:17:03.523057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:03.523057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:17:03.523057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:03.523057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:17:03.523057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:03.523057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:17:03.523057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:03.523057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:17:03.523057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:03.523057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:17:03.523057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:17:03.523057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:03.523057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:17:03.523057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:03.523057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:17:03.523057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:17:03.523057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:03.523823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:17:03.523823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:03.523823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:17:03.525455 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:17:03.525988 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:17:03.526152 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:17:03.526279 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:17:03.526767 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:17:03.527002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:03.527683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:17:03.527683 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:03.527683 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:17:03.527683 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:17:03.527683 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:03.527683 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:17:03.527683 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:03.527683 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:17:03.528306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:03.528306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:17:03.528306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:17:03.528306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:17:03.528306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:03.528306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:17:03.528306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:03.528306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:17:03.528306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:03.528306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:17:03.528306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:03.528306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:17:03.528306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:03.528306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:17:03.528306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:17:03.529206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:03.529206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:17:03.529206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:03.529206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:17:03.529206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:17:03.529206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:03.529206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:17:03.529206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:03.529206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:17:03.529206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:03.529206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:17:03.529206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:17:03.529206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:17:03.529206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:17:03.530207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:17:03.530207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:17:03.530207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:17:03.530207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:17:03.530207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:17:03.530207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:03.530207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:17:03.530207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:03.530207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:17:03.530207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:03.530207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:17:03.530207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:03.530207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:17:03.530840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:03.530840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:17:03.530840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:17:03.530840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:03.530840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:17:03.530840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:03.530840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:17:03.530840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:17:03.530840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:03.530840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:17:03.530840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:03.530840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:17:03.531465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:03.531465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:17:03.531465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:17:03.531465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:17:03.531465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:17:03.531465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:17:03.531465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:17:03.531465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:17:03.531465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:17:03.531465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:17:03.531465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:17:03.531465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:17:03.531465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:17:03.531465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:17:03.531465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:17:03.531465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:17:03.533945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:17:03.533945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:17:03.533945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:17:03.533945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:17:03.533945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:17:03.533945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:03.533945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:17:03.533945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:03.533945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:17:03.533945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:03.533945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:17:03.533945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:03.533945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:17:03.533945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:03.533945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:17:03.533945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:17:03.533945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:03.533945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:17:03.534963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:03.534963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:17:03.534963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:17:03.534963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:03.534963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:17:03.534963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:03.534963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:17:03.534963 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:17:03.534963 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:17:03.534963 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:17:03.535483 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:17:03.535483 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:17:03.535483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:03.535483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:17:03.535483 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:03.535483 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:17:03.535483 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:17:03.535483 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:03.535483 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:17:03.535483 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:03.535483 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:17:03.535483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:03.536154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:17:03.536154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:17:03.536154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:17:03.536154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:03.536154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:17:03.536154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:03.536154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:17:03.536154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:03.536154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:17:03.536154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:03.536154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:17:03.536154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:03.536154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:17:03.536154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:17:03.536154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:03.536154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:17:03.536154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:03.539720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:17:03.539720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:17:03.539720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:03.539720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:17:03.539720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:03.539720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:17:03.539720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:03.539720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:17:03.539720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:17:03.539720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:17:03.539720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:17:03.539720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:17:03.539720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:17:03.540767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:17:03.540767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:17:03.540767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:03.540767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:17:03.540767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:03.540767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:17:03.540767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:03.540767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:17:03.540767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:03.540767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:17:03.540767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:03.540767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:17:03.540767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:17:03.541392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:03.541392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:17:03.541392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:03.541392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:17:03.541392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:17:03.541392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:03.541392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:17:03.541392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:03.541392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:17:03.541392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:03.541392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:17:03.541392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:17:03.541776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:17:03.541776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:17:03.541776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:17:03.541776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:17:03.541776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:17:03.541776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:17:03.541776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:17:03.541776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:17:03.541776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:17:03.541776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:17:03.541776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:17:03.541776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:17:03.541776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:17:03.541776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:17:03.541776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:17:03.541776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:17:03.541776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:17:03.541776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:03.541776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:17:03.542308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:03.542308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:17:03.542308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:03.542308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:17:03.542308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:03.542308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:17:03.542308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:03.542308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:17:03.542308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:17:03.542308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:03.542308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:17:03.542308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:03.542308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:17:03.542308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:17:03.542308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:03.542308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:17:03.542308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:03.542308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:17:03.544779 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:17:03.545735 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:17:03.545891 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:17:03.546133 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:17:03.546494 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:17:03.546728 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:17:03.546899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:03.548493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:17:03.548493 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:03.548493 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:17:03.548493 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:17:03.548493 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:03.548493 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:17:03.548493 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:03.548493 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:17:03.549318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:03.549318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:17:03.549318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:17:03.549318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:17:03.549318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:17:03.549318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:17:03.549318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:17:03.549318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:03.549318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:17:03.549318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:03.549318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:17:03.549318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:03.549660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:17:03.549660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:03.549660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:17:03.549660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:03.549660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:17:03.549660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:17:03.549660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:03.549660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:17:03.549660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:03.549660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:17:03.549660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:17:03.549660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:03.549660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:17:03.549993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:03.549993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:17:03.550311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:03.550311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:17:03.550311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:17:03.550311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:17:03.550311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:17:03.550311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:17:03.550311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:17:03.550311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:17:03.550311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:17:03.550311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:17:03.550311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:17:03.550311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:17:03.550311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:17:03.550311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:17:03.550311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:17:03.550311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:17:03.550696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:17:03.550696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:17:03.550696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:03.550696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:17:03.550696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:03.550696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:17:03.550696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:03.550696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:17:03.550696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:03.550696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:17:03.550696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:03.550696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:17:03.550696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:17:03.550696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:03.550696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:17:03.550696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:03.550696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:17:03.550696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:17:03.550696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:03.551219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:17:03.551219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:03.551219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:17:07.960884 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:17:07.962084 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:17:07.962084 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:17:07.963388 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:17:08.000364 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:17:08.000841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:08.001949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:17:08.001949 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:08.001949 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:17:08.001949 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:17:08.001949 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:08.001949 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:17:08.001949 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:08.001949 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:17:08.003252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:08.003252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:17:08.003252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:17:08.003252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:17:08.003252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:17:08.003252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:17:08.003252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:17:08.003252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:08.003252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:17:08.003252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:08.003252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:17:08.003252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:08.003950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:17:08.003950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:08.003950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:17:08.003950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:08.003950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:17:08.003950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:17:08.003950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:08.003950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:17:08.003950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:08.003950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:17:08.003950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:17:08.003950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:08.003950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:17:08.004564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:08.004564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:17:08.004867 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:08.004867 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:17:08.004867 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:17:08.004867 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:17:08.004867 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:17:08.004867 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:17:08.004867 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:17:08.004867 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:17:08.004867 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:17:08.004867 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:17:08.004867 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:17:08.004867 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:17:08.004867 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:17:08.004867 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:17:08.004867 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:17:08.004867 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:17:08.005396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:17:08.005396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:17:08.005396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:08.005396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:17:08.005396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:08.005396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:17:08.005396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:08.005396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:17:08.005396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:08.005396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:17:08.005396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:08.005396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:17:08.005396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:17:08.005396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:08.005396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:17:08.005396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:08.005396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:17:08.005396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:17:08.005396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:08.006027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:17:08.006027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:08.006027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:17:08.515512 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:17:08.516025 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:17:08.516251 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:17:08.517707 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:17:08.518137 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:17:08.518299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:08.519198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:17:08.519198 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:08.519198 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:17:08.519198 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:17:08.519198 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:08.519198 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:17:08.519198 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:08.519198 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:17:08.520298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:08.520298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:17:08.520298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:17:08.520298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:17:08.520298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:17:08.520298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:17:08.520298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:17:08.520298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:08.520298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:17:08.520298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:08.520298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:17:08.520298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:08.521198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:17:08.521198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:08.521198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:17:08.521198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:08.521198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:17:08.521198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:17:08.521198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:08.521198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:17:08.521198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:08.521198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:17:08.521198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:17:08.521198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:08.521198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:17:08.521542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:08.521542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:17:08.521542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:08.521542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:17:08.521542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:17:08.521542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:17:08.521542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:17:08.521542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:17:08.521542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:17:08.521542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:17:08.521542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:17:08.521542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:17:08.521542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:17:08.521542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:17:08.521542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:17:08.521998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:17:08.521998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:17:08.521998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:17:08.521998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:17:08.521998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:17:08.521998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:08.521998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:17:08.521998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:08.521998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:17:08.521998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:08.521998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:17:08.521998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:08.521998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:17:08.521998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:08.521998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:17:08.521998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:17:08.521998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:08.521998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:17:08.521998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:08.522507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:17:08.522507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:17:08.522507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:08.522507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:17:08.522507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:08.522507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:17:08.525859 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:17:08.526218 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:17:08.526370 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:17:08.526536 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:17:08.528513 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:17:08.528786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:08.529890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:17:08.529890 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:08.529890 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:17:08.529890 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:17:08.529890 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:08.529890 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:17:08.529890 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:08.529890 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:17:08.530997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:08.530997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:17:08.530997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:17:08.530997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:17:08.530997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:08.530997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:17:08.530997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:08.530997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:17:08.530997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:08.530997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:17:08.530997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:08.530997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:17:08.530997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:08.530997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:17:08.530997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:17:08.531666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:08.531666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:17:08.531666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:08.531666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:17:08.531666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:17:08.531666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:08.531666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:17:08.531666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:08.531666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:17:08.532081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:08.532081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:17:08.532081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:17:08.532081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:17:08.532081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:17:08.532081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:17:08.532081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:17:08.532081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:17:08.532081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:17:08.532081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:17:08.532081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:08.532081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:17:08.532580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:08.532580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:17:08.532580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:08.532580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:17:08.532580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:08.532580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:17:08.532580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:08.532580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:17:08.532580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:17:08.532580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:08.532580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:17:08.532580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:08.532580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:17:08.532580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:17:08.533191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:08.533191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:17:08.533191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:08.533191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:17:08.533356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:08.533356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:17:08.533356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:17:08.533356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:17:08.533356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:17:08.533356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:17:08.533356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:17:08.533356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:17:08.533356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:17:08.533356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:17:08.533356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:17:08.533356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:17:08.533356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:17:08.533356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:17:08.533356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:17:08.533356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:17:08.534011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:17:08.534011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:17:08.534011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:17:08.534011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:17:08.534011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:17:08.534011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:08.534011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:17:08.534011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:08.534011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:17:08.534011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:08.534011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:17:08.534011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:08.534011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:17:08.534011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:08.534011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:17:08.534011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:17:08.534011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:08.534011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:17:08.534479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:08.534479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:17:08.534479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:17:08.534479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:08.534479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:17:08.534479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:08.534479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:17:08.534479 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:17:08.534479 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:17:08.534479 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:17:08.534699 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:17:08.534699 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:17:08.534699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:08.535700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:17:08.535700 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:08.535700 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:17:08.535700 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:17:08.535700 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:08.535700 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:17:08.535700 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:08.535700 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:17:08.536568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:08.536568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:17:08.536568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:17:08.536568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:17:08.536568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:08.536568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:17:08.536568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:08.536568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:17:08.536568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:08.536568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:17:08.536568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:08.536568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:17:08.536568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:08.536568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:17:08.536568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:17:08.536880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:08.536880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:17:08.536880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:08.536880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:17:08.536880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:17:08.536880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:08.536880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:17:08.536880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:08.536880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:17:08.537597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:08.537597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:17:08.537597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:17:08.537597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:17:08.537597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:17:08.537597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:17:08.537597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:17:08.537597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:17:08.537597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:17:08.537597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:08.537597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:17:08.537869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:08.537869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:17:08.537869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:08.537869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:17:08.537869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:08.537869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:17:08.537869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:08.537869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:17:08.537869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:17:08.537869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:08.537869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:17:08.537869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:08.537869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:17:08.537869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:17:08.538133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:08.538133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:17:08.538133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:08.538133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:17:08.538673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:08.538673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:17:08.538673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:17:08.538673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:17:08.538673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:17:08.538673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:17:08.538673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:17:08.538673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:17:08.538673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:17:08.538673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:17:08.538673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:17:08.538673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:17:08.538673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:17:08.538673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:17:08.538673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:17:08.538673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:17:08.538673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:17:08.539016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:17:08.539016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:17:08.539016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:17:08.539016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:08.539016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:17:08.539016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:08.539016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:17:08.539016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:08.539016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:17:08.539016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:08.539016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:17:08.539016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:08.539016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:17:08.539016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:17:08.539016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:08.539016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:17:08.539016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:08.539016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:17:08.539016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:17:08.539411 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:08.539411 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:17:08.539411 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:08.539411 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:17:08.545670 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:17:08.546203 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:17:08.546463 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:17:08.546666 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:17:08.547207 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:17:08.547575 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:17:08.547858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:08.548920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:17:08.548920 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:08.548920 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:17:08.548920 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:17:08.548920 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:08.548920 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:17:08.548920 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:08.548920 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:17:08.550439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:08.550439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:17:08.550439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:17:08.550439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:17:08.550439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:17:08.550439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:17:08.550439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:17:08.550439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:08.550439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:17:08.550439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:08.550439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:17:08.550439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:08.551296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:17:08.551296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:08.551296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:17:08.551296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:08.551296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:17:08.551296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:17:08.551296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:08.551296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:17:08.551296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:08.551296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:17:08.551296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:17:08.551296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:08.551296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:17:08.552069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:08.552069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:17:08.552069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:08.552069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:17:08.552069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:17:08.552069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:17:08.552069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:17:08.552069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:17:08.552069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:17:08.552069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:17:08.552069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:17:08.552069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:17:08.552069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:17:08.552069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:17:08.552069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:17:08.552907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:17:08.552907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:17:08.552907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:17:08.552907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:17:08.552907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:17:08.552907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:08.552907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:17:08.552907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:08.552907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:17:08.552907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:08.552907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:17:08.552907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:08.552907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:17:08.552907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:08.552907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:17:08.552907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:17:08.552907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:08.552907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:17:08.552907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:08.553814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:17:08.553814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:17:08.553814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:08.553814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:17:08.553814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:08.553814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:17:12.961589 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:17:12.962411 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:17:12.962411 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:17:12.963608 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:17:13.002704 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:17:13.002864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:13.003354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:17:13.003354 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:13.003354 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:17:13.003354 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:17:13.003354 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:13.003354 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:17:13.003354 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:13.003354 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:17:13.004183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:13.004183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:17:13.004183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:17:13.004183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:17:13.004183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:17:13.004183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:17:13.004183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:17:13.004183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:13.004183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:17:13.004183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:13.004183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:17:13.004183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:13.004958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:17:13.004958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:13.004958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:17:13.004958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:13.004958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:17:13.004958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:17:13.004958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:13.004958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:17:13.004958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:13.004958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:17:13.004958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:17:13.004958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:13.004958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:17:13.005537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:13.005537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:17:13.005537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:13.005537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:17:13.005537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:17:13.005537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:17:13.005537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:17:13.005537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:17:13.005537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:17:13.005537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:17:13.005537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:17:13.005537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:17:13.005537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:17:13.005537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:17:13.005537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:17:13.006278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:17:13.006278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:17:13.006278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:17:13.006278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:17:13.006278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:17:13.006278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:13.006278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:17:13.006278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:13.006278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:17:13.006278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:13.006278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:17:13.006278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:13.006278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:17:13.006278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:13.006278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:17:13.006278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:17:13.006278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:13.006278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:17:13.006278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:13.007167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:17:13.007167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:17:13.007167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:13.007167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:17:13.007167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:13.007167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:17:13.515973 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:17:13.516512 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:17:13.516601 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:17:13.517472 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:17:13.517701 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:17:13.517890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:13.518499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:17:13.518499 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:13.518499 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:17:13.518499 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:17:13.518499 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:13.518499 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:17:13.518499 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:13.518499 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:17:13.519551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:13.519551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:17:13.519551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:17:13.519551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:17:13.519551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:17:13.519551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:17:13.519551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:17:13.519551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:13.519551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:17:13.519551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:13.519551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:17:13.519551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:13.520274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:17:13.520274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:13.520274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:17:13.520274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:13.520274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:17:13.520274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:17:13.520274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:13.520274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:17:13.520274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:13.520274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:17:13.520274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:17:13.520274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:13.520274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:17:13.520996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:13.520996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:17:13.520996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:13.520996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:17:13.520996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:17:13.520996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:17:13.520996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:17:13.520996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:17:13.520996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:17:13.520996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:17:13.520996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:17:13.520996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:17:13.520996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:17:13.520996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:17:13.520996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:17:13.521672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:17:13.521672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:17:13.521672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:17:13.521672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:17:13.521672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:17:13.521672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:13.521672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:17:13.521672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:13.521672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:17:13.521672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:13.521672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:17:13.521672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:13.521672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:17:13.521672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:13.521672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:17:13.521672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:17:13.521672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:13.521672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:17:13.521672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:13.522731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:17:13.522731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:17:13.522731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:13.522731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:17:13.522731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:13.522731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:17:13.526400 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:17:13.526775 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:17:13.526830 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:17:13.526998 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:17:13.527426 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:17:13.527784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:13.528593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:17:13.528593 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:13.528593 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:17:13.528593 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:17:13.528593 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:13.528593 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:17:13.528593 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:13.528593 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:17:13.529467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:13.529467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:17:13.529467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:17:13.529467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:17:13.529467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:13.529467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:17:13.529467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:13.529467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:17:13.529467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:13.529467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:17:13.529467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:13.529467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:17:13.529467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:13.529467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:17:13.529467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:17:13.530005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:13.530005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:17:13.530005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:13.530005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:17:13.530005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:17:13.530005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:13.530005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:17:13.530005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:13.530005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:17:13.530599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:13.530599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:17:13.530599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:17:13.530599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:17:13.530599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:17:13.530599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:17:13.530599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:17:13.530599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:17:13.530599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:17:13.530599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:17:13.530599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:13.530599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:17:13.531030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:13.531030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:17:13.531030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:13.531030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:17:13.531030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:13.531030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:17:13.531030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:13.531030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:17:13.531030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:17:13.531030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:13.531030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:17:13.531030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:13.531030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:17:13.531030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:17:13.531425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:13.531425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:17:13.531425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:13.531425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:17:13.532041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:13.532041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:17:13.532041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:17:13.532041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:17:13.532041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:17:13.532041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:17:13.532041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:17:13.532041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:17:13.532041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:17:13.532041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:17:13.532041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:17:13.532041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:17:13.532041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:17:13.532041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:17:13.532041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:17:13.532041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:17:13.533160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:17:13.533160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:17:13.533160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:17:13.533160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:17:13.533160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:17:13.533160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:13.533160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:17:13.533160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:13.533160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:17:13.533160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:13.533160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:17:13.533160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:13.533160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:17:13.533160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:13.533160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:17:13.533160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:17:13.533160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:13.533160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:17:13.533533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:13.533533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:17:13.533533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:17:13.533533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:13.533533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:17:13.533533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:13.533533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:17:13.533907 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:17:13.535012 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:17:13.535210 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:17:13.535278 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:17:13.535644 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:17:13.535940 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:13.536510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:17:13.536510 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:13.536510 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:17:13.536510 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:17:13.536510 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:13.536510 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:17:13.536510 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:13.536510 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:17:13.537367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:13.537367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:17:13.537367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:17:13.537367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:17:13.537367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:13.537367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:17:13.537367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:13.537367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:17:13.537367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:13.537367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:17:13.537367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:13.537367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:17:13.537367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:13.537367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:17:13.537367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:17:13.537821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:13.537821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:17:13.537821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:13.537821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:17:13.537821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:17:13.537821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:13.537821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:17:13.537821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:13.537821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:17:13.538305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:13.538305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:17:13.538305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:17:13.538305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:17:13.538305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:17:13.538305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:17:13.538305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:17:13.538305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:17:13.538305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:17:13.538305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:13.538305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:17:13.538670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:13.538670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:17:13.538670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:13.538670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:17:13.538670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:13.538670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:17:13.538670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:13.538670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:17:13.538670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:17:13.538670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:13.538670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:17:13.538670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:13.538670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:17:13.538670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:17:13.539143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:13.539143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:17:13.539143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:13.539143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:17:13.539397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:13.539397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:17:13.539397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:17:13.539397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:17:13.539397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:17:13.539397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:17:13.539397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:17:13.539397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:17:13.539397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:17:13.539397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:17:13.539397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:17:13.539397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:17:13.539397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:17:13.539397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:17:13.539397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:17:13.539397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:17:13.539397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:17:13.539884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:17:13.539884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:17:13.539884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:17:13.539884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:13.539884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:17:13.539884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:13.539884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:17:13.539884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:13.539884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:17:13.539884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:13.539884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:17:13.539884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:13.539884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:17:13.539884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:17:13.539884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:13.539884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:17:13.539884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:13.539884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:17:13.539884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:17:13.540399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:13.540399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:17:13.540399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:13.540399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:17:13.546085 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:17:13.546372 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:17:13.546495 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:17:13.546658 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:17:13.546992 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:17:13.547279 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:17:13.547553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:13.548108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:17:13.548108 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:13.548108 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:17:13.548108 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:17:13.548108 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:13.548108 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:17:13.548108 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:13.548108 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:17:13.548837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:13.548837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:17:13.548837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:17:13.548837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:17:13.548837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:17:13.548837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:17:13.548837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:17:13.548837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:13.548837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:17:13.548837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:13.548837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:17:13.548837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:13.549183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:17:13.549183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:13.549183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:17:13.549183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:13.549183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:17:13.549183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:17:13.549183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:13.549183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:17:13.549183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:13.549183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:17:13.549183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:17:13.549183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:13.549183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:17:13.549583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:13.549583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:17:13.549910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:13.549910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:17:13.549910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:17:13.549910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:17:13.549910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:17:13.549910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:17:13.549910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:17:13.549910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:17:13.549910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:17:13.549910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:17:13.549910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:17:13.549910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:17:13.549910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:17:13.549910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:17:13.549910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:17:13.549910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:17:13.550217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:17:13.550217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:17:13.550217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:13.550217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:17:13.550217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:13.550217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:17:13.550217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:13.550217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:17:13.550217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:13.550217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:17:13.550217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:13.550217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:17:13.550217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:17:13.550217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:13.550217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:17:13.550217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:13.550217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:17:13.550217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:17:13.550217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:13.550855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:17:13.550855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:13.550855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:17:17.962220 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:17:17.966316 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:17:17.966316 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:17:17.966316 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:17:18.004227 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:17:18.004380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:18.005320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:17:18.005320 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:18.005320 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:17:18.005320 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:17:18.005320 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:18.005320 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:17:18.005320 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:18.005320 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:17:18.006572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:18.006572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:17:18.006572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:17:18.006572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:17:18.006572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:17:18.006572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:17:18.006572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:17:18.006572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:18.006572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:17:18.006572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:18.006572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:17:18.006572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:18.007459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:17:18.007459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:18.007459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:17:18.007459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:18.007459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:17:18.007459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:17:18.007459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:18.007459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:17:18.007459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:18.007459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:17:18.007459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:17:18.007459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:18.007459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:17:18.007872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:18.007872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:17:18.007872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:18.007872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:17:18.007872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:17:18.007872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:17:18.007872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:17:18.007872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:17:18.007872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:17:18.007872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:17:18.007872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:17:18.007872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:17:18.007872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:17:18.007872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:17:18.007872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:17:18.008332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:17:18.008332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:17:18.008332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:17:18.008332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:17:18.008332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:17:18.008332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:18.008332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:17:18.008332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:18.008332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:17:18.008332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:18.008332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:17:18.008332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:18.008332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:17:18.008332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:18.008332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:17:18.008332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:17:18.008332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:18.008332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:17:18.008332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:18.008940 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:17:18.008940 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:17:18.008940 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:18.008940 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:17:18.008940 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:18.008940 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:17:18.516716 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:17:18.517146 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:17:18.517251 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:17:18.518053 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:17:18.518324 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:17:18.518583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:18.519413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:17:18.519413 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:18.519413 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:17:18.519413 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:17:18.519413 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:18.519413 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:17:18.519413 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:18.519413 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:17:18.520415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:18.520415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:17:18.520415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:17:18.520415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:17:18.520415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:17:18.520415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:17:18.520415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:17:18.520415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:18.520415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:17:18.520415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:18.520415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:17:18.520415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:18.521708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:17:18.521708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:18.521708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:17:18.521708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:18.521708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:17:18.521708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:17:18.521708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:18.521708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:17:18.521708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:18.521708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:17:18.521708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:17:18.521708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:18.521708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:17:18.522380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:18.522380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:17:18.522380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:18.522380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:17:18.522380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:17:18.522380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:17:18.522380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:17:18.522380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:17:18.522380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:17:18.522380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:17:18.522380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:17:18.522380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:17:18.522380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:17:18.522380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:17:18.522380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:17:18.523697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:17:18.523697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:17:18.523697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:17:18.523697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:17:18.523697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:17:18.523697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:18.523697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:17:18.523697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:18.523697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:17:18.523697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:18.523697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:17:18.523697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:18.523697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:17:18.523697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:18.523697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:17:18.523697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:17:18.523697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:18.523697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:17:18.523697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:18.524639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:17:18.524639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:17:18.524639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:18.524639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:17:18.524639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:18.524639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:17:18.527136 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:17:18.527485 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:17:18.527485 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:17:18.527653 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:17:18.528062 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:17:18.528717 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:18.529019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:17:18.529019 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:18.529019 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:17:18.529019 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:17:18.529019 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:18.529019 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:17:18.529019 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:18.529019 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:17:18.529720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:18.529720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:17:18.529720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:17:18.529720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:17:18.529720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:18.529720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:17:18.529720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:18.529720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:17:18.529720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:18.529720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:17:18.529720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:18.529720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:17:18.529720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:18.529720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:17:18.529720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:17:18.530185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:18.530185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:17:18.530185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:18.530185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:17:18.530185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:17:18.530185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:18.530185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:17:18.530185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:18.530185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:17:18.530568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:18.530568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:17:18.530568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:17:18.530568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:17:18.530568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:17:18.530568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:17:18.530568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:17:18.530568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:17:18.530568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:17:18.530568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:17:18.530568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:18.530568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:17:18.530998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:18.530998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:17:18.530998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:18.530998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:17:18.530998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:18.530998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:17:18.530998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:18.530998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:17:18.530998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:17:18.530998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:18.530998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:17:18.530998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:18.530998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:17:18.530998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:17:18.531356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:18.531356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:17:18.531356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:18.531356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:17:18.531673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:18.531673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:17:18.531673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:17:18.531673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:17:18.531673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:17:18.531673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:17:18.531673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:17:18.531673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:17:18.531673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:17:18.531673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:17:18.531673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:17:18.531673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:17:18.531673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:17:18.531673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:17:18.531673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:17:18.531673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:17:18.532130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:17:18.532130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:17:18.532130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:17:18.532130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:17:18.532130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:17:18.532130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:18.532130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:17:18.532130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:18.532130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:17:18.532130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:18.532130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:17:18.532130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:18.532130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:17:18.532130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:18.532130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:17:18.532130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:17:18.532130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:18.532130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:17:18.533071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:18.533071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:17:18.533071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:17:18.533071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:18.533071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:17:18.533071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:18.533071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:17:18.535067 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:17:18.535573 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:17:18.535846 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:17:18.536109 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:17:18.536424 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:17:18.536674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:18.537391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:17:18.537391 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:18.537391 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:17:18.537391 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:17:18.537391 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:18.537391 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:17:18.537391 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:18.537391 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:17:18.538615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:18.538615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:17:18.538615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:17:18.538615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:17:18.538615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:18.538615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:17:18.538615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:18.538615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:17:18.538615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:18.538615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:17:18.538615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:18.538615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:17:18.538615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:18.538615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:17:18.538615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:17:18.539077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:18.539077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:17:18.539077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:18.539077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:17:18.539077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:17:18.539077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:18.539077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:17:18.539077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:18.539077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:17:18.540211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:18.540211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:17:18.540211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:17:18.540211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:17:18.540211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:17:18.540211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:17:18.540211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:17:18.540211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:17:18.540211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:17:18.540211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:18.540211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:17:18.540586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:18.540586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:17:18.540586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:18.540586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:17:18.540586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:18.540586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:17:18.540586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:18.540586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:17:18.540586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:17:18.540586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:18.540586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:17:18.540586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:18.540586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:17:18.540586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:17:18.540995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:18.540995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:17:18.540995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:18.540995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:17:18.542175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:18.542175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:17:18.542175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:17:18.542175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:17:18.542175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:17:18.542175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:17:18.542175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:17:18.542175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:17:18.542175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:17:18.542175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:17:18.542175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:17:18.542175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:17:18.542175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:17:18.542175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:17:18.542175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:17:18.542175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:17:18.542175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:17:18.542645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:17:18.542645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:17:18.542645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:17:18.542645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:18.542645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:17:18.542645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:18.542645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:17:18.542645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:18.542645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:17:18.542645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:18.542645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:17:18.542645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:18.542645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:17:18.542645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:17:18.542645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:18.542645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:17:18.542645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:18.542645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:17:18.542645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:17:18.543214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:18.543214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:17:18.543214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:18.543214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:17:18.546418 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:17:18.546630 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:17:18.546753 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:17:18.546958 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:17:18.547219 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:17:18.547417 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:17:18.547550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:18.548119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:17:18.548119 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:18.548119 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:17:18.548119 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:17:18.548119 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:18.548119 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:17:18.548119 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:18.548119 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:17:18.548978 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:18.548978 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:17:18.548978 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:17:18.548978 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:17:18.548978 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:17:18.548978 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:17:18.548978 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:17:18.548978 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:18.548978 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:17:18.548978 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:18.548978 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:17:18.548978 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:18.550546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:17:18.550546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:18.550546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:17:18.550546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:18.550546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:17:18.550546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:17:18.550546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:18.550546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:17:18.550546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:18.550546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:17:18.550546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:17:18.550546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:18.550546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:17:18.550970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:18.550970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:17:18.550970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:18.550970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:17:18.550970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:17:18.550970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:17:18.550970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:17:18.550970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:17:18.550970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:17:18.550970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:17:18.550970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:17:18.550970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:17:18.550970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:17:18.550970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:17:18.550970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:17:18.551430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:17:18.551430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:17:18.551430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:17:18.551430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:17:18.551430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:17:18.551430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:18.551430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:17:18.551430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:18.551430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:17:18.551430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:18.551430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:17:18.551430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:18.551430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:17:18.551430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:18.551430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:17:18.551430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:17:18.551430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:18.551430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:17:18.551430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:18.552004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:17:18.552004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:17:18.552004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:18.552004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:17:18.552004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:18.552004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:17:22.963151 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:17:22.964224 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:17:22.964224 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:17:22.965558 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:17:23.007222 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:17:23.007472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:23.008221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:17:23.008221 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:23.008221 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:17:23.008221 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:17:23.008221 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:23.008221 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:17:23.008221 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:23.008221 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:17:23.009460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.044 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:23.009460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:17:23.009460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:17:23.009460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:17:23.009460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:17:23.009460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:17:23.009460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:17:23.009460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:23.009460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:17:23.009460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:23.009460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:17:23.009460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:23.010121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:17:23.010121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:23.010121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:17:23.010121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:23.010121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:17:23.010121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:17:23.010121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:23.010121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:17:23.010121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:23.010121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:17:23.010121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:17:23.010121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:23.010121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:17:23.010635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:23.010635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:17:23.010635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:23.010635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:17:23.010635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:17:23.010635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:17:23.010635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:17:23.010635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:17:23.010635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:17:23.010635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:17:23.010635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:17:23.010635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:17:23.010635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:17:23.010635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:17:23.010635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:17:23.011289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:17:23.011289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:17:23.011289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:17:23.011289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:17:23.011289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:17:23.011289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:23.011289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:17:23.011289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:23.011289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:17:23.011289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:23.011289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:17:23.011289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:23.011289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:17:23.011289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:23.011289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:17:23.011289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:17:23.011289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:23.011289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:17:23.011289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:23.012031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:17:23.012031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:17:23.012031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:23.012031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:17:23.012031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:23.012031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:17:23.517819 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:17:23.518373 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:17:23.518592 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:17:23.520242 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:17:23.520521 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:17:23.521702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:23.522543 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:17:23.522543 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:23.522543 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:17:23.522543 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:17:23.522543 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:23.522543 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:17:23.522543 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:23.522543 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:17:23.524252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:23.524252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:17:23.524252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:17:23.524252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:17:23.524252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:17:23.524252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:17:23.524252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:17:23.524252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:23.524252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:17:23.524252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:23.524252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:17:23.524252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:23.524905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:17:23.524905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:23.524905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:17:23.524905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:23.524905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:17:23.524905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:17:23.524905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:23.524905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:17:23.524905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:23.524905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:17:23.524905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:17:23.524905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:23.524905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:17:23.525619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:23.525619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:17:23.526270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:23.526270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:17:23.526270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:17:23.526270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:17:23.526270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:17:23.526270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:17:23.526270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:17:23.526270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:17:23.526270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:17:23.526270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:17:23.526270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:17:23.526270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:17:23.526270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:17:23.526270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:17:23.526270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:17:23.526270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:17:23.527204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:17:23.527204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:17:23.527204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:23.527204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:17:23.527204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:23.527204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:17:23.527204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:23.527204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:17:23.527204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:23.527204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:17:23.527204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:23.527204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:17:23.527204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:17:23.527204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:23.527204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:17:23.527204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:23.527204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:17:23.527204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:17:23.527204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:23.528204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:17:23.528204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:23.528204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:17:23.528204 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:17:23.528204 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:17:23.528204 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:17:23.528204 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:17:23.529614 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:17:23.529614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:23.529614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:17:23.529614 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:23.529614 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:17:23.529614 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:17:23.529614 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:23.529614 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:17:23.529614 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:23.529614 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:17:23.530839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:23.530839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:17:23.530839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:17:23.530839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:17:23.530839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:23.530839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:17:23.530839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:23.530839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:17:23.530839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:23.530839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:17:23.530839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:23.530839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:17:23.530839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:23.530839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:17:23.530839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:17:23.532055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:23.532055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:17:23.532055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:23.532055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:17:23.532055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:17:23.532055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:23.532055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:17:23.532055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:23.532055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:17:23.532055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:23.532055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:17:23.532055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:17:23.532055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:17:23.532055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:17:23.533569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:17:23.533569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:17:23.533569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:17:23.533569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:17:23.533569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:17:23.533569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:23.533569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:17:23.533569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:23.533569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:17:23.533569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:23.533569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:17:23.533569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:23.533569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:17:23.534167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:23.534167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:17:23.534167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:17:23.534167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:23.534167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:17:23.534167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:23.534167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:17:23.534167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:17:23.534167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:23.534167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:17:23.534167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:23.534167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:17:23.534515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:23.534515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:17:23.534515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:17:23.534515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:17:23.534515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:17:23.534515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:17:23.534515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:17:23.534515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:17:23.534515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:17:23.534515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:17:23.534515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:17:23.534515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:17:23.534515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:17:23.534515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:17:23.534515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:17:23.534515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:17:23.534881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:17:23.534881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:17:23.534881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:17:23.534881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:17:23.534881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:17:23.534881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:23.534881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:17:23.534881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:23.534881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:17:23.534881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:23.534881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:17:23.534881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:23.534881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:17:23.534881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:23.534881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:17:23.534881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:17:23.534881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:23.534881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:17:23.535253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:23.535253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:17:23.535253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:17:23.535253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:23.535253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:17:23.535253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:23.535253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:17:23.535504 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:17:23.535807 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:17:23.535913 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:17:23.536051 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:17:23.536415 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:17:23.536646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:23.537386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:17:23.537386 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:23.537386 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:17:23.537386 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:17:23.537386 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:23.537386 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:17:23.537386 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:23.537386 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:17:23.538068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:23.538068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:17:23.538068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:17:23.538068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:17:23.538068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:23.538068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:17:23.538068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:23.538068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:17:23.538068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:23.538068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:17:23.538068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:23.538068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:17:23.538068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:23.538068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:17:23.538068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:17:23.538358 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:23.538358 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:17:23.538358 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:23.538358 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:17:23.538358 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:17:23.538358 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:23.538358 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:17:23.538358 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:23.538358 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:17:23.538958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:23.538958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:17:23.538958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:17:23.538958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:17:23.538958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:17:23.538958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:17:23.538958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:17:23.538958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:17:23.538958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:17:23.538958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:23.538958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:17:23.539202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:23.539202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:17:23.539202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:23.539202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:17:23.539202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:23.539202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:17:23.539202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:23.539202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:17:23.539202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:17:23.539202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:23.539202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:17:23.539202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:23.539202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:17:23.539202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:17:23.539465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:23.539465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:17:23.539465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:23.539465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:17:23.539977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:23.539977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:17:23.539977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:17:23.539977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:17:23.539977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:17:23.539977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:17:23.539977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:17:23.539977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:17:23.539977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:17:23.539977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:17:23.539977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:17:23.539977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:17:23.539977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:17:23.539977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:17:23.539977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:17:23.539977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:17:23.539977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:17:23.540306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:17:23.540306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:17:23.540306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:17:23.540306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:23.540306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:17:23.540306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:23.540306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:17:23.540306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:23.540306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:17:23.540306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:23.540306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:17:23.540306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:23.540306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:17:23.540306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:17:23.540306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:23.540306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:17:23.540306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:23.540306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:17:23.540306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:17:23.540674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:23.540674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:17:23.540674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:23.540674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:17:23.546836 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:17:23.547243 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:17:23.547380 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:17:23.547619 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:17:23.548148 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:17:23.548592 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:17:23.548885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:23.549696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:17:23.549696 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:23.549696 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:17:23.549696 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:17:23.549696 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:23.549696 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:17:23.549696 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:23.549696 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:17:23.550665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:23.550665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:17:23.550665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:17:23.550665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:17:23.550665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:17:23.550665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:17:23.550665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:17:23.550665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:23.550665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:17:23.550665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:23.550665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:17:23.550665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:23.551319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:17:23.551319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:23.551319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:17:23.551319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:23.551319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:17:23.551319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:17:23.551319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:23.551319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:17:23.551319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:23.551319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:17:23.551319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:17:23.551319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:23.551319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:17:23.552004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:23.552004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:17:23.552004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:23.552004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:17:23.552004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:17:23.552004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:17:23.552004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:17:23.552004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:17:23.552004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:17:23.552004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:17:23.552004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:17:23.552004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:17:23.552004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:17:23.552004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:17:23.552004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:17:23.554602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:17:23.554602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:17:23.554602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:17:23.554602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:17:23.554602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:17:23.554602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:23.554602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:17:23.554602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:23.554602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:17:23.554602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:23.554602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:17:23.554602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:23.554602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:17:23.554602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:23.554602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:17:23.554602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:17:23.554602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:23.554602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:17:23.554602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:23.555908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:17:23.555908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:17:23.555908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:23.555908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:17:23.555908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:23.555908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:17:27.963923 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:17:27.964593 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:17:27.964593 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:17:27.965488 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:17:28.004075 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:17:28.004435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:28.005122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:17:28.005122 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:28.005122 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:17:28.005122 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:17:28.005122 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:28.005122 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:17:28.005122 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:28.005122 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:17:28.006259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:28.006259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:17:28.006259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:17:28.006259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:17:28.006259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:17:28.006259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:17:28.006259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:17:28.006259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:28.006259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:17:28.006259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:28.006259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:17:28.006259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:28.007347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:17:28.007347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:28.007347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:17:28.007347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:28.007347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:17:28.007347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:17:28.007347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:28.007347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:17:28.007347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:28.007347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:17:28.007347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:17:28.007347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:28.007347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:17:28.008321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:28.008321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:17:28.008321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:28.008321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:17:28.008321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:17:28.008321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:17:28.008321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:17:28.008321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:17:28.008321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:17:28.008321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:17:28.008321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:17:28.008321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:17:28.008321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:17:28.008321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:17:28.008321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:17:28.009417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:17:28.009417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:17:28.009417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:17:28.009417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:17:28.009417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:17:28.009417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:28.009417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:17:28.009417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:28.009417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:17:28.009417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:28.009417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:17:28.009417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:28.009417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:17:28.009417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:28.009417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:17:28.009417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:17:28.009417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:28.009417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:17:28.009417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:28.010841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:17:28.010841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:17:28.010841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:28.010841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:17:28.010841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:28.010841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:17:28.518444 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:17:28.519089 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:17:28.519305 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:17:28.520511 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:17:28.521289 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:17:28.521289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:28.522164 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:17:28.522164 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:28.522164 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:17:28.522164 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:17:28.522164 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:28.522164 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:17:28.522164 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:28.522164 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:17:28.523769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:28.523769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:17:28.523769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:17:28.523769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:17:28.523769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:17:28.523769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:17:28.523769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:17:28.523769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:28.523769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:17:28.523769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:28.523769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:17:28.523769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:28.524525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:17:28.524525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:28.524525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:17:28.524525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:28.524525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:17:28.524525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:17:28.524525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:28.524525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:17:28.524525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:28.524525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:17:28.524525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:17:28.524525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:28.524525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:17:28.525302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:28.525302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:17:28.525525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:28.525525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:17:28.525525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:17:28.525525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:17:28.525525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:17:28.525525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:17:28.525525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:17:28.525525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:17:28.525525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:17:28.525525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:17:28.525525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:17:28.525525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:17:28.525525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:17:28.525525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:17:28.525525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:17:28.525525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:17:28.526634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:17:28.526634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:17:28.526634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:28.526634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:17:28.526634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:28.526634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:17:28.526634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:28.526634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:17:28.526634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:28.526634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:17:28.526634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:28.526634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:17:28.526634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:17:28.526634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:28.526634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:17:28.526634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:28.526634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:17:28.526634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:17:28.526634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:28.527806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:17:28.527806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:28.527806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:17:28.527806 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:17:28.528219 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:17:28.528419 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:17:28.528606 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:17:28.529133 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:17:28.529439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:28.530316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:17:28.530316 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:28.530316 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:17:28.530316 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:17:28.530316 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:28.530316 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:17:28.530316 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:28.530316 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:17:28.531326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:28.531326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:17:28.531326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:17:28.531326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:17:28.531326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:28.531326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:17:28.531326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:28.531326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:17:28.531326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:28.531326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:17:28.531326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:28.531326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:17:28.531326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:28.531326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:17:28.531326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:17:28.531775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:28.531775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:17:28.531775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:28.531775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:17:28.531775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:17:28.531775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:28.531775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:17:28.531775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:28.531775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:17:28.532263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:28.532263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:17:28.532263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:17:28.532263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:17:28.532263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:17:28.532263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:17:28.532263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:17:28.532263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:17:28.532263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:17:28.532263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:17:28.532263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:28.532263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:17:28.532556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:28.532556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:17:28.532556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:28.532556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:17:28.532556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:28.532556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:17:28.532556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:28.532556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:17:28.532556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:17:28.532556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:28.532556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:17:28.532556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:28.532556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:17:28.532556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:17:28.532850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:28.532850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:17:28.532850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:28.532850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:17:28.533396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:28.533396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:17:28.533396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:17:28.533396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:17:28.533396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:17:28.533396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:17:28.533396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:17:28.533396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:17:28.533396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:17:28.533396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:17:28.533396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:17:28.533396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:17:28.533396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:17:28.533396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:17:28.533396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:17:28.533396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:17:28.533789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:17:28.533789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:17:28.533789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:17:28.533789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:17:28.533789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:17:28.533789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:28.533789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:17:28.533789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:28.533789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:17:28.533789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:28.533789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:17:28.533789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:28.533789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:17:28.533789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:28.533789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:17:28.533789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:17:28.533789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:28.533789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:17:28.534274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:28.534274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:17:28.534274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:17:28.534274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:28.534274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:17:28.534274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:28.534274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:17:28.536017 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:17:28.536434 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:17:28.536503 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:17:28.536642 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:17:28.537300 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:17:28.537766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:28.538590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:17:28.538590 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:28.538590 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:17:28.538590 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:17:28.538590 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:28.538590 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:17:28.538590 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:28.538590 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:17:28.539388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:28.539388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:17:28.539388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:17:28.539388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:17:28.539388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:28.539388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:17:28.539388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:28.539388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:17:28.539388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:28.539388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:17:28.539388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:28.539388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:17:28.539388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:28.539388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:17:28.539388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:17:28.539805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:28.539805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:17:28.539805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:28.539805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:17:28.539805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:17:28.539805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:28.539805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:17:28.539805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:28.539805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:17:28.540362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:28.540362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:17:28.540362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:17:28.540362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:17:28.540362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:17:28.540362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:17:28.540362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:17:28.540362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:17:28.540362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:17:28.540362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:28.540362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:17:28.540718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:28.540718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:17:28.540718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:28.540718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:17:28.540718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:28.540718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:17:28.540718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:28.540718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:17:28.540718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:17:28.540718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:28.540718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:17:28.540718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:28.540718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:17:28.540718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:17:28.541159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:28.541159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:17:28.541159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:28.541159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:17:28.541474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:28.541474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:17:28.541474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:17:28.541474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:17:28.541474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:17:28.541474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:17:28.541474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:17:28.541474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:17:28.541474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:17:28.541474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:17:28.541474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:17:28.541474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:17:28.541474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:17:28.541474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:17:28.541474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:17:28.541474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:17:28.541474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:17:28.542050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:17:28.542050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:17:28.542050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:17:28.542050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:28.542050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:17:28.542050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:28.542050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:17:28.542050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:28.542050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:17:28.542050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:28.542050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:17:28.542050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:28.542050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:17:28.542050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:17:28.542050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:28.542050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:17:28.542050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:28.542050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:17:28.542050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:17:28.542525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:28.542525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:17:28.542525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:28.542525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:17:28.547284 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:17:28.547614 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:17:28.547721 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:17:28.547914 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:17:28.548506 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:17:28.549365 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:17:28.549365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:28.550099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:17:28.550099 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:28.550099 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:17:28.550099 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:17:28.550099 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:28.550099 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:17:28.550099 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:28.550099 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:17:28.551022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:28.551022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:17:28.551022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:17:28.551022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:17:28.551022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:17:28.551022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:17:28.551022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:17:28.551022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:28.551022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:17:28.551022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:28.551022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:17:28.551022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:28.551424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:17:28.551424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:28.551424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:17:28.551424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:28.551424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:17:28.551424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:17:28.551424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:28.551424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:17:28.551424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:28.551424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:17:28.551424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:17:28.551424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:28.551424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:17:28.551780 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:28.551780 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:17:28.552030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:28.552030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:17:28.552030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:17:28.552030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:17:28.552030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:17:28.552030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:17:28.552030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:17:28.552030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:17:28.552030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:17:28.552030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:17:28.552030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:17:28.552030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:17:28.552030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:17:28.552030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:17:28.552030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:17:28.552030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:17:28.552637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:17:28.552637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:17:28.552637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:28.552637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:17:28.552637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:28.552637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:17:28.552637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:28.552637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:17:28.552637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:28.552637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:17:28.552637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:28.552637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:17:28.552637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:17:28.552637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:28.552637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:17:28.552637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:28.552637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:17:28.552637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:17:28.552637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:28.553318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:17:28.553318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:28.553318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:17:32.964721 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:17:32.965623 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:17:32.965623 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:17:32.966699 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:17:33.014991 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:17:33.015276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:33.015657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:17:33.015657 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:33.015657 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:17:33.015657 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:17:33.015657 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:33.015657 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:17:33.015657 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:33.015657 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:17:33.016514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.050 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:33.016514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:17:33.016514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:17:33.016514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:17:33.016514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:17:33.016514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:17:33.016514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:17:33.016514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:33.016514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:17:33.016514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:33.016514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:17:33.016514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:33.016975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:17:33.016975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:33.016975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:17:33.016975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:33.016975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:17:33.016975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:17:33.016975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:33.016975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:17:33.016975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:33.016975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:17:33.016975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:17:33.016975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:33.016975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:17:33.017381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:33.017381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:17:33.017463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:33.017463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:17:33.017463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:17:33.017463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:17:33.017463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:17:33.017463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:17:33.017463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:17:33.017463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:17:33.017463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:17:33.017463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:17:33.017463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:17:33.017463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:17:33.017463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:17:33.017463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:17:33.017463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:17:33.017463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:17:33.017943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:17:33.017943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:17:33.017943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:33.017943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:17:33.017943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:33.017943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:17:33.017943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:33.017943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:17:33.017943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:33.017943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:17:33.017943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:33.017943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:17:33.017943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:17:33.017943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:33.017943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:17:33.017943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:33.017943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:17:33.017943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:17:33.017943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:33.018457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:17:33.018457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:33.018457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:17:33.519104 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:17:33.519596 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:17:33.519884 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:17:33.520628 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:17:33.520855 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:17:33.521170 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:33.522316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:17:33.522316 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:33.522316 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:17:33.522316 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:17:33.522316 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:33.522316 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:17:33.522316 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:33.522316 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:17:33.523919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:33.523919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:17:33.523919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:17:33.523919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:17:33.523919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:17:33.523919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:17:33.523919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:17:33.523919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:33.523919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:17:33.523919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:33.523919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:17:33.523919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:33.524368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:17:33.524368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:33.524368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:17:33.524368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:33.524368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:17:33.524368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:17:33.524368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:33.524368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:17:33.524368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:33.524368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:17:33.524368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:17:33.524368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:33.524368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:17:33.524874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:33.524874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:17:33.526053 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:33.526053 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:17:33.526053 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:17:33.526053 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:17:33.526053 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:17:33.526053 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:17:33.526053 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:17:33.526053 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:17:33.526053 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:17:33.526053 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:17:33.526053 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:17:33.526053 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:17:33.526053 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:17:33.526053 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:17:33.526053 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:17:33.526053 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:17:33.526714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:17:33.526714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:17:33.526714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:33.526714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:17:33.526714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:33.526714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:17:33.526714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:33.526714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:17:33.526714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:33.526714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:17:33.526714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:33.526714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:17:33.526714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:17:33.526714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:33.526714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:17:33.526714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:33.526714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:17:33.526714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:17:33.526714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:33.527326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:17:33.527326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:33.527326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:17:33.528110 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:17:33.528389 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:17:33.528505 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:17:33.528632 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:17:33.529019 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:17:33.529254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:33.529811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:17:33.529811 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:33.529811 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:17:33.529811 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:17:33.529811 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:33.529811 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:17:33.529811 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:33.529811 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:17:33.530576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:33.530576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:17:33.530576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:17:33.530576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:17:33.530576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:33.530576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:17:33.530576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:33.530576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:17:33.530576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:33.530576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:17:33.530576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:33.530576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:17:33.530576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:33.530576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:17:33.530576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:17:33.531124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:33.531124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:17:33.531124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:33.531124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:17:33.531124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:17:33.531124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:33.531124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:17:33.531124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:33.531124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:17:33.531490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:33.531490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:17:33.531490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:17:33.531490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:17:33.531490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:17:33.531490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:17:33.531490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:17:33.531490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:17:33.531490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:17:33.531490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:17:33.531490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:33.531490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:17:33.531958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:33.531958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:17:33.531958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:33.531958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:17:33.531958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:33.531958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:17:33.531958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:33.531958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:17:33.531958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:17:33.531958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:33.531958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:17:33.531958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:33.531958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:17:33.531958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:17:33.532462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:33.532462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:17:33.532462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:33.532462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:17:33.532615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:33.532615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:17:33.532615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:17:33.532615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:17:33.532615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:17:33.532615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:17:33.532615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:17:33.532615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:17:33.532615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:17:33.532615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:17:33.532615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:17:33.532615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:17:33.532615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:17:33.532615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:17:33.532615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:17:33.532615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:17:33.533188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:17:33.533188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:17:33.533188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:17:33.533188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:17:33.533188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:17:33.533188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:33.533188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:17:33.533188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:33.533188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:17:33.533188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:33.533188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:17:33.533188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:33.533188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:17:33.533188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:33.533188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:17:33.533188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:17:33.533188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:33.533188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:17:33.533837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:33.533837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:17:33.533837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:17:33.533837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:33.533837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:17:33.533837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:33.533837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:17:33.536577 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:17:33.537554 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:17:33.537697 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:17:33.537946 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:17:33.538289 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:17:33.538516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:33.539128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:17:33.539128 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:33.539128 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:17:33.539128 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:17:33.539128 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:33.539128 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:17:33.539128 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:33.539128 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:17:33.540418 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:33.540418 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:17:33.540418 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:17:33.540418 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:17:33.540418 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:33.540418 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:17:33.540418 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:33.540418 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:17:33.540418 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:33.540418 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:17:33.540418 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:33.540418 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:17:33.540418 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:33.540418 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:17:33.540418 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:17:33.540986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:33.540986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:17:33.540986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:33.540986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:17:33.540986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:17:33.540986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:33.540986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:17:33.540986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:33.540986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:17:33.542164 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:33.542164 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:17:33.542164 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:17:33.542164 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:17:33.542164 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:17:33.542164 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:17:33.542164 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:17:33.542164 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:17:33.542164 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:17:33.542164 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:33.542164 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:17:33.542469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:33.542469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:17:33.542469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:33.542469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:17:33.542469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:33.542469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:17:33.542469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:33.542469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:17:33.542469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:17:33.542469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:33.542469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:17:33.542469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:33.542469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:17:33.542469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:17:33.542839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:33.542839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:17:33.542839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:33.542839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:17:33.544001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:33.544001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:17:33.544001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:17:33.544001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:17:33.544001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:17:33.544001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:17:33.544001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:17:33.544001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:17:33.544001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:17:33.544001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:17:33.544001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:17:33.544001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:17:33.544001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:17:33.544001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:17:33.544001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:17:33.544001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:17:33.544001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:17:33.544424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:17:33.544424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:17:33.544424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:17:33.544424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:33.544424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:17:33.544424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:33.544424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:17:33.544424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:33.544424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:17:33.544424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:33.544424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:17:33.544424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:33.544424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:17:33.544424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:17:33.544424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:33.544424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:17:33.544424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:33.544424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:17:33.544424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:17:33.544887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:33.544887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:17:33.544887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:33.544887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:17:33.547891 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:17:33.548349 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:17:33.548536 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:17:33.548849 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:17:33.549443 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:17:33.549856 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:17:33.550124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:33.550871 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:17:33.550871 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:33.550871 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:17:33.550871 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:17:33.550871 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:33.550871 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:17:33.550871 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:33.550871 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:17:33.587521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:33.587521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:17:33.587521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:17:33.587521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:17:33.587521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:17:33.587521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:17:33.587521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:17:33.587521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:33.587521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:17:33.587521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:33.587521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:17:33.587521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:33.588271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:17:33.588271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:33.588271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:17:33.588271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:33.588271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:17:33.588271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:17:33.588271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:33.588271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:17:33.588271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:33.588271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:17:33.588271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:17:33.588271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:33.588271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:17:33.589083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:33.589083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:17:33.589083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:33.589083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:17:33.589083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:17:33.589083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:17:33.589083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:17:33.589083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:17:33.589083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:17:33.589083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:17:33.589083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:17:33.589083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:17:33.589083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:17:33.589083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:17:33.589083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:17:33.590191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:17:33.590191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:17:33.590191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:17:33.590191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:17:33.590191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:17:33.590191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:33.590191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:17:33.590191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:33.590191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:17:33.590191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:33.590191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:17:33.590191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:33.590191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:17:33.590191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:33.590191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:17:33.590191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:17:33.590191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:33.590191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:17:33.590191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:33.591346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:17:33.591346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:17:33.591346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:33.591346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:17:33.591346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:33.591346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:17:37.965100 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:17:37.965770 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:17:37.965955 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:17:37.967274 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:17:38.004826 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:17:38.005784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:38.005784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:17:38.005784 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:38.005784 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:17:38.005784 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:17:38.005784 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:38.005784 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:17:38.005784 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:38.005784 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:17:38.006210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:38.006210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:17:38.006210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:17:38.006210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:17:38.006210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:17:38.006210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:17:38.006210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:17:38.006210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:38.006210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:17:38.006210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:38.006210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:17:38.006210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:38.006505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:17:38.006505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:38.006505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:17:38.006505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:38.006505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:17:38.006505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:17:38.006505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:38.006505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:17:38.006505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:38.006505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:17:38.006505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:17:38.006505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:38.006505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:17:38.006781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:38.006781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:17:38.007062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:38.007062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:17:38.007062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:17:38.007062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:17:38.007062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:17:38.007062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:17:38.007062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:17:38.007062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:17:38.007062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:17:38.007062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:17:38.007062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:17:38.007062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:17:38.007062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:17:38.007062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:17:38.007062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:17:38.007062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:17:38.007522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:17:38.007522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:17:38.007522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:38.007522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:17:38.007522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:38.007522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:17:38.007522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:38.007522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:17:38.007522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:38.007522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:17:38.007522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:38.007522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:17:38.007522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:17:38.007522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:38.007522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:17:38.007522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:38.007522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:17:38.007522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:17:38.007522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:38.008049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:17:38.008049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:38.008049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:17:38.519972 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:17:38.520398 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:17:38.520526 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:17:38.521661 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:17:38.522226 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:17:38.523953 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:38.524880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:17:38.524880 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:38.524880 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:17:38.524880 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:17:38.524880 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:38.524880 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:17:38.524880 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:38.524880 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:17:38.526006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:38.526006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:17:38.526006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:17:38.526006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:17:38.526006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:17:38.526006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:17:38.526006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:17:38.526006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:38.526006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:17:38.526006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:38.526006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:17:38.526006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:38.526547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:17:38.526547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:38.526547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:17:38.526547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:38.526547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:17:38.526547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:17:38.526547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:38.526547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:17:38.526547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:38.526547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:17:38.526547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:17:38.526547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:38.526547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:17:38.527088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:38.527088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:17:38.527088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:38.527088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:17:38.527088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:17:38.527088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:17:38.527088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:17:38.527088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:17:38.527088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:17:38.527088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:17:38.527088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:17:38.527088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:17:38.527088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:17:38.527088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:17:38.527088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:17:38.527694 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:17:38.527694 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:17:38.527694 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:17:38.527694 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:17:38.527694 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:17:38.527694 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:38.527694 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:17:38.527694 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:38.527694 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:17:38.527694 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:38.527694 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:17:38.527694 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:38.527694 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:17:38.527694 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:38.527694 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:17:38.527694 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:17:38.527694 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:38.527694 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:17:38.527694 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:38.528616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:17:38.528616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:17:38.528616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:38.528616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:17:38.528616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:38.528616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:17:38.528616 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:17:38.528917 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:17:38.528917 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:17:38.529190 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:17:38.530031 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:17:38.530324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:38.531090 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:17:38.531090 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:38.531090 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:17:38.531090 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:17:38.531090 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:38.531090 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:17:38.531090 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:38.531090 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:17:38.531881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:38.531881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:17:38.531881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:17:38.531881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:17:38.531881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:38.531881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:17:38.531881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:38.531881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:17:38.531881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:38.531881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:17:38.531881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:38.531881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:17:38.531881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:38.531881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:17:38.531881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:17:38.532450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:38.532450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:17:38.532450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:38.532450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:17:38.532450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:17:38.532450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:38.532450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:17:38.532450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:38.532450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:17:38.532682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:38.532682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:17:38.532682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:17:38.532682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:17:38.532682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:17:38.532682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:17:38.532682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:17:38.532682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:17:38.532682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:17:38.532682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:17:38.532682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:38.532682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:17:38.533024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:38.533024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:17:38.533024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:38.533024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:17:38.533024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:38.533024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:17:38.533024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:38.533024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:17:38.533024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:17:38.533024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:38.533024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:17:38.533024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:38.533024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:17:38.533024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:17:38.533434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:38.533434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:17:38.533434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:38.533434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:17:38.535020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:38.535020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:17:38.535020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:17:38.535020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:17:38.535020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:17:38.535020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:17:38.535020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:17:38.535020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:17:38.535020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:17:38.535020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:17:38.535020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:17:38.535020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:17:38.535020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:17:38.535020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:17:38.535020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:17:38.535020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:17:38.535444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:17:38.535444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:17:38.535444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:17:38.535444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:17:38.535444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:17:38.535444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:38.535444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:17:38.535444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:38.535444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:17:38.535444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:38.535444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:17:38.535444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:38.535444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:17:38.535444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:38.535444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:17:38.535444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:17:38.535444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:38.535444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:17:38.535920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:38.535920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:17:38.535920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:17:38.535920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:38.535920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:17:38.535920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:38.535920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:17:38.537315 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:17:38.537719 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:17:38.537891 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:17:38.538062 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:17:38.538948 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:17:38.539365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:38.540459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:17:38.540459 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:38.540459 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:17:38.540459 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:17:38.540459 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:38.540459 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:17:38.540459 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:38.540459 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:17:38.541608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:38.541608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:17:38.541608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:17:38.541608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:17:38.541608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:38.541608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:17:38.541608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:38.541608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:17:38.541608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:38.541608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:17:38.541608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:38.541608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:17:38.541608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:38.541608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:17:38.541608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:17:38.542509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:38.542509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:17:38.542509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:38.542509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:17:38.542509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:17:38.542509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:38.542509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:17:38.542509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:38.542509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:17:38.543513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:38.543513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:17:38.543513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:17:38.543513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:17:38.543513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:17:38.543513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:17:38.543513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:17:38.543513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:17:38.543513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:17:38.543513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:38.543513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:17:38.543957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:38.543957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:17:38.543957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:38.543957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:17:38.543957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:38.543957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:17:38.543957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:38.543957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:17:38.543957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:17:38.543957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:38.543957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:17:38.543957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:38.543957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:17:38.543957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:17:38.544973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:38.544973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:17:38.544973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:38.544973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:17:38.545848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:38.545848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:17:38.545848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:17:38.545848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:17:38.545848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:17:38.545848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:17:38.545848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:17:38.545848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:17:38.545848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:17:38.545848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:17:38.545848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:17:38.545848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:17:38.545848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:17:38.545848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:17:38.545848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:17:38.545848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:17:38.545848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:17:38.546399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:17:38.546399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:17:38.546399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:17:38.546399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:38.546399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:17:38.546399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:38.546399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:17:38.546399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:38.546399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:17:38.546399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:38.546399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:17:38.546399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:38.546399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:17:38.546399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:17:38.546399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:38.546399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:17:38.546399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:38.546399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:17:38.546399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:17:38.547942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:38.547942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:17:38.547942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:38.547942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:17:38.548516 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:17:38.549035 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:17:38.549315 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:17:38.549658 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:17:38.550341 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:17:38.551251 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:17:38.551417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:38.552144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:17:38.552144 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:38.552144 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:17:38.552144 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:17:38.552144 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:38.552144 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:17:38.552144 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:38.552144 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:17:38.553539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:38.553539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:17:38.553539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:17:38.553539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:17:38.553539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:17:38.553539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:17:38.553539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:17:38.553539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:38.553539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:17:38.553539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:38.553539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:17:38.553539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:38.554188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:17:38.554188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:38.554188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:17:38.554188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:38.554188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:17:38.554188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:17:38.554188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:38.554188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:17:38.554188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:38.554188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:17:38.554188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:17:38.554188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:38.554188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:17:38.554526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:38.554526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:17:38.554783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:38.554783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:17:38.554783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:17:38.554783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:17:38.554783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:17:38.554783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:17:38.554783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:17:38.554783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:17:38.554783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:17:38.554783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:17:38.554783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:17:38.554783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:17:38.554783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:17:38.554783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:17:38.554783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:17:38.554783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:17:38.555319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:17:38.555319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:17:38.555319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:38.555319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:17:38.555319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:38.555319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:17:38.555319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:38.555319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:17:38.555319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:38.555319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:17:38.555319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:38.555319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:17:38.555319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:17:38.555319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:38.555319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:17:38.555319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:38.555319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:17:38.555319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:17:38.555319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:38.555948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:17:38.555948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:38.555948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:17:42.966010 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:17:42.966934 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:17:42.966934 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:17:42.968163 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:17:43.005457 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:17:43.005994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:43.006868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:17:43.006868 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:43.006868 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:17:43.006868 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:17:43.006868 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:43.006868 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:17:43.006868 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:43.006868 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:17:43.007924 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:43.007924 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:17:43.007924 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:17:43.007924 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:17:43.007924 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:17:43.007924 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:17:43.007924 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:17:43.007924 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:43.007924 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:17:43.007924 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:43.007924 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:17:43.007924 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:43.008943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:17:43.008943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:43.008943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:17:43.008943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:43.008943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:17:43.008943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:17:43.008943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:43.008943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:17:43.008943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:43.008943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:17:43.008943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:17:43.008943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:43.008943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:17:43.009611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:43.009611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:17:43.009611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:43.009611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:17:43.009611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:17:43.009611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:17:43.009611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:17:43.009611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:17:43.009611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:17:43.009611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:17:43.009611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:17:43.009611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:17:43.009611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:17:43.009611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:17:43.009611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:17:43.010507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:17:43.010507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:17:43.010507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:17:43.010507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:17:43.010507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:17:43.010507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:43.010507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:17:43.010507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:43.010507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:17:43.010507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:43.010507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:17:43.010507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:43.010507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:17:43.010507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:43.010507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:17:43.010507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:17:43.010507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:43.010507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:17:43.010507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:43.011524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:17:43.011524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:17:43.011524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:43.011524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:17:43.011524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:43.011524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:17:43.520599 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:17:43.521267 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:17:43.521267 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:17:43.522240 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:17:43.522710 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:17:43.523039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:43.524161 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:17:43.524161 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:43.524161 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:17:43.524161 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:17:43.524161 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:43.524161 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:17:43.524161 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:43.524161 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:17:43.526384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:43.526384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:17:43.526384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:17:43.526384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:17:43.526384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:17:43.526384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:17:43.526384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:17:43.526384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:43.526384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:17:43.526384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:43.526384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:17:43.526384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:43.526846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:17:43.526846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:43.526846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:17:43.526846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:43.526846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:17:43.526846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:17:43.526846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:43.526846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:17:43.526846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:43.526846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:17:43.526846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:17:43.526846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:43.526846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:17:43.527153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:43.527153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:17:43.531934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:43.531934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:17:43.531934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:17:43.531934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:17:43.531934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:17:43.531934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:17:43.531934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:17:43.531934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:17:43.531934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:17:43.531934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:17:43.531934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:17:43.531934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:17:43.531934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:17:43.531934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:17:43.531934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:17:43.531934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:17:43.532602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:17:43.532602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:17:43.532602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:43.532602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:17:43.532602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:43.532602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:17:43.532602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:43.532602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:17:43.532602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:43.532602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:17:43.532602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:43.532602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:17:43.532602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:17:43.532602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:43.532602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:17:43.532602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:43.532602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:17:43.532602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:17:43.532602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:43.533339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:17:43.533339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:43.533339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:17:43.533339 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:17:43.533339 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:17:43.533339 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:17:43.533339 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:17:43.533339 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:17:43.534653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:43.534653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:17:43.534653 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:43.534653 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:17:43.534653 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:17:43.534653 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:43.534653 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:17:43.534653 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:43.534653 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:17:43.535842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:43.535842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:17:43.535842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:17:43.535842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:17:43.535842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:43.535842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:17:43.535842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:43.535842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:17:43.535842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:43.535842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:17:43.535842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:43.535842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:17:43.535842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:43.535842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:17:43.535842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:17:43.536516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:43.536516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:17:43.536516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:43.536516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:17:43.536516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:17:43.536516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:43.536516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:17:43.536516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:43.536516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:17:43.536516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:43.536516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:17:43.536516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:17:43.536516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:17:43.536516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:17:43.537145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:17:43.537145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:17:43.537145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:17:43.537145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:17:43.537145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:17:43.537145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:43.537145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:17:43.537145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:43.537145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:17:43.537145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:43.537145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:17:43.537145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:43.537145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:17:43.537652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:43.537652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:17:43.537652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:17:43.537652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:43.537652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:17:43.537652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:43.537652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:17:43.537652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:17:43.537652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:43.537652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:17:43.537652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:43.537652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:17:43.538250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:43.538250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:17:43.538250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:17:43.538250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:17:43.538250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:17:43.538250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:17:43.538250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:17:43.538250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:17:43.538250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:17:43.538250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:17:43.538250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:17:43.538250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:17:43.538250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:17:43.538250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:17:43.538250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:17:43.538250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:17:43.538887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:17:43.538887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:17:43.538887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:17:43.538887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:17:43.538887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:17:43.538887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:43.538887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:17:43.538887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:43.538887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:17:43.538887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:43.538887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:17:43.538887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:43.538887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:17:43.538887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:43.538887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:17:43.538887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:17:43.538887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:43.538887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:17:43.539539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:43.539539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:17:43.539539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:17:43.539539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:43.539539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:17:43.539539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:43.539539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:17:43.539539 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:17:43.539539 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:17:43.539539 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:17:43.539852 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:17:43.539852 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:17:43.539852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:43.539852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:17:43.539852 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:43.539852 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:17:43.539852 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:17:43.539852 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:43.539852 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:17:43.539852 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:43.539852 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:17:43.540134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:43.540134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:17:43.540134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:17:43.540134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:17:43.540134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:43.540134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:17:43.540134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:43.540134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:17:43.540134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:43.540134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:17:43.540134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:43.540134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:17:43.540134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:43.540134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:17:43.540134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:17:43.540491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:43.540491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:17:43.540491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:43.540491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:17:43.540491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:17:43.540491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:43.540491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:17:43.540491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:43.540491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:17:43.541070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:43.541070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:17:43.541070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:17:43.541070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:17:43.541070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:17:43.541070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:17:43.541070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:17:43.541070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:17:43.541070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:17:43.541070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:43.541070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:17:43.541388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:43.541388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:17:43.541388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:43.541388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:17:43.541388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:43.541388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:17:43.541388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:43.541388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:17:43.541388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:17:43.541388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:43.541388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:17:43.541388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:43.541388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:17:43.541388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:17:43.541729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:43.541729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:17:43.541729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:43.541729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:17:43.542316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:43.542316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:17:43.542316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:17:43.542316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:17:43.542316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:17:43.542316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:17:43.542316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:17:43.542316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:17:43.542316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:17:43.542316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:17:43.542316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:17:43.542316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:17:43.542316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:17:43.542316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:17:43.542316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:17:43.542316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:17:43.542316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:17:43.542781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:17:43.542781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:17:43.542781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:17:43.542781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:43.542781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:17:43.542781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:43.542781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:17:43.542781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:43.542781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:17:43.542781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:43.542781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:17:43.542781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:43.542781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:17:43.542781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:17:43.542781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:43.542781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:17:43.542781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:43.542781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:17:43.542781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:17:43.543316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:43.543316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:17:43.543316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:43.543316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:17:43.549013 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:17:43.549384 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:17:43.549494 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:17:43.549665 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:17:43.550270 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:17:43.550661 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:17:43.551118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:43.552176 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:17:43.552176 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:43.552176 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:17:43.552176 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:17:43.552176 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:43.552176 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:17:43.552176 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:43.552176 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:17:43.553251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:43.553251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:17:43.553251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:17:43.553251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:17:43.553251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:17:43.553251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:17:43.553251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:17:43.553251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:43.553251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:17:43.553251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:43.553251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:17:43.553251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:43.553690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:17:43.553690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:43.553690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:17:43.553690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:43.553690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:17:43.553690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:17:43.553690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:43.553690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:17:43.553690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:43.553690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:17:43.553690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:17:43.553690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:43.553690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:17:43.554036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:43.554036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:17:43.554315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:43.554315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:17:43.554315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:17:43.554315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:17:43.554315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:17:43.554315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:17:43.554315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:17:43.554315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:17:43.554315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:17:43.554315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:17:43.554315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:17:43.554315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:17:43.554315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:17:43.554315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:17:43.554315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:17:43.554315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:17:43.554796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:17:43.554796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:17:43.554796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:43.554796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:17:43.554796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:43.554796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:17:43.554796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:43.554796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:17:43.554796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:43.554796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:17:43.554796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:43.554796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:17:43.554796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:17:43.554796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:43.554796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:17:43.554796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:43.554796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:17:43.554796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:17:43.554796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:43.555351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:17:43.555351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:43.555351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:17:47.966681 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:17:47.967632 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:17:47.967632 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:17:47.968907 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:17:48.006425 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:17:48.006987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:48.009565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:17:48.009565 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:48.009565 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:17:48.009565 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:17:48.009565 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:48.009565 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:17:48.009565 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:48.009565 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:17:48.010826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:48.010826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:17:48.010826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:17:48.010826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:17:48.010826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:17:48.010826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:17:48.010826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:17:48.010826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:48.010826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:17:48.010826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:48.010826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:17:48.010826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:48.011233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:17:48.011233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:48.011233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:17:48.011233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:48.011233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:17:48.011233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:17:48.011233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:48.011233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:17:48.011233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:48.011233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:17:48.011233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:17:48.011233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:48.011233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:17:48.011586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:48.011586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:17:48.012210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:48.012210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:17:48.012210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:17:48.012210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:17:48.012210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:17:48.012210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:17:48.012210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:17:48.012210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:17:48.012210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:17:48.012210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:17:48.012210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:17:48.012210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:17:48.012210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:17:48.012210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:17:48.012210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:17:48.012210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:17:48.012648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:17:48.012648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:17:48.012648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:48.012648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:17:48.012648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:48.012648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:17:48.012648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:48.012648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:17:48.012648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:48.012648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:17:48.012648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:48.012648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:17:48.012648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:17:48.012648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:48.012648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:17:48.012648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:48.012648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:17:48.012648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:17:48.012648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:48.013215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:17:48.013215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:48.013215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:17:48.521218 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:17:48.521818 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:17:48.521990 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:17:48.523240 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:17:48.523586 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:17:48.523883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:48.524939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:17:48.524939 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:48.524939 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:17:48.524939 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:17:48.524939 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:48.524939 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:17:48.524939 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:48.524939 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:17:48.526062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:48.526062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:17:48.526062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:17:48.526062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:17:48.526062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:17:48.526062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:17:48.526062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:17:48.526062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:48.526062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:17:48.526062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:48.526062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:17:48.526062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:48.526381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:17:48.526381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:48.526381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:17:48.526381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:48.526381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:17:48.526381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:17:48.526381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:48.526381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:17:48.526381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:48.526381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:17:48.526381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:17:48.526381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:48.526381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:17:48.526687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:48.526687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:17:48.527372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:48.527372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:17:48.527372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:17:48.527372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:17:48.527372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:17:48.527372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:17:48.527372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:17:48.527372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:17:48.527372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:17:48.527372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:17:48.527372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:17:48.527372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:17:48.527372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:17:48.527372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:17:48.527372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:17:48.527372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:17:48.527795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:17:48.527795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:17:48.527795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:48.527795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:17:48.527795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:48.527795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:17:48.527795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:48.527795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:17:48.527795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:48.527795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:17:48.527795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:48.527795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:17:48.527795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:17:48.527795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:48.527795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:17:48.527795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:48.527795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:17:48.527795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:17:48.527795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:48.528230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:17:48.528230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:48.528230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:17:48.532574 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:17:48.532981 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:17:48.533137 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:17:48.533267 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:17:48.533622 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:17:48.533863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:48.534563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:17:48.534563 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:48.534563 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:17:48.534563 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:17:48.534563 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:48.534563 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:17:48.534563 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:48.534563 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:17:48.536316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:48.536316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:17:48.536316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:17:48.536316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:17:48.536316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:48.536316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:17:48.536316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:48.536316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:17:48.536316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:48.536316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:17:48.536316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:48.536316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:17:48.536316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:48.536316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:17:48.536316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:17:48.536602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:48.536602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:17:48.536602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:48.536602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:17:48.536602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:17:48.536602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:48.536602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:17:48.536602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:48.536602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:17:48.538084 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:48.538084 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:17:48.538084 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:17:48.538084 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:17:48.538084 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:17:48.538084 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:17:48.538084 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:17:48.538084 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:17:48.538084 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:17:48.538084 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:17:48.538084 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:48.538084 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:17:48.538345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:48.538345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:17:48.538345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:48.538345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:17:48.538345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:48.538345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:17:48.538345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:48.538345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:17:48.538345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:17:48.538345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:48.538345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:17:48.538345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:48.538345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:17:48.538345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:17:48.538593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:48.538593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:17:48.538593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:48.538593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:17:48.538593 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:17:48.540859 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:48.540859 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:17:48.540859 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:17:48.540859 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:17:48.540859 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:17:48.540859 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:17:48.540859 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:17:48.540859 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:17:48.540859 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:17:48.540859 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:17:48.540859 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:17:48.540859 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:17:48.540859 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:17:48.540859 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:17:48.540859 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:17:48.540859 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:17:48.541444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:17:48.541444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:17:48.541444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:17:48.541444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:17:48.541444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:17:48.541444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:48.541444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:17:48.541444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:48.541444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:17:48.541444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:48.541444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:17:48.541444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:48.541444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:17:48.541444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:48.541444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:17:48.541444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:17:48.541444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:48.541444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:17:48.542636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:48.542636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:17:48.542636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:17:48.542636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:48.542636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:17:48.542636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:48.542636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:17:48.542636 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:17:48.542636 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:17:48.542636 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:17:48.543051 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:17:48.543051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:48.543424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:17:48.543424 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:48.543424 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:17:48.543424 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:17:48.543424 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:48.543424 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:17:48.543424 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:48.543424 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:17:48.544384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:48.544384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:17:48.544384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:17:48.544384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:17:48.544384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:48.544384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:17:48.544384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:48.544384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:17:48.544384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:48.544384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:17:48.544384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:48.544384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:17:48.544384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:48.544384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:17:48.544384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:17:48.545038 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:48.545038 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:17:48.545038 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:48.545038 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:17:48.545038 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:17:48.545038 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:48.545038 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:17:48.545038 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:48.545038 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:17:48.545391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:48.545391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:17:48.545391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:17:48.545391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:17:48.545391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:17:48.545391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:17:48.545391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:17:48.545391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:17:48.545391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:17:48.545391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:48.545391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:17:48.545978 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:48.545978 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:17:48.545978 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:48.545978 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:17:48.545978 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:48.545978 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:17:48.545978 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:48.545978 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:17:48.545978 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:17:48.545978 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:48.545978 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:17:48.545978 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:48.545978 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:17:48.545978 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:17:48.546478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:48.546478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:17:48.546478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:48.546478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:17:48.546478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:48.546478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:17:48.546478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:17:48.546478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:17:48.546478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:17:48.546478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:17:48.546478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:17:48.546478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:17:48.546478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:17:48.546478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:17:48.547018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:17:48.547018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:17:48.547018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:17:48.547018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:17:48.547018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:17:48.547018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:17:48.547018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:17:48.547018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:17:48.547018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:17:48.547018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:17:48.547018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:48.547018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:17:48.547018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:48.547018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:17:48.547018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:48.547018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:17:48.547018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:48.547018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:17:48.547018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:48.547733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:17:48.547733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:17:48.547733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:48.547733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:17:48.547733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:48.547733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:17:48.547733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:17:48.547733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:48.547733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:17:48.547733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:48.547733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:17:48.549715 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:17:48.550102 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:17:48.550221 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:17:48.550378 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:17:48.550847 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:17:48.551218 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:17:48.551445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:48.552210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:17:48.552210 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:48.552210 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:17:48.552210 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:17:48.552210 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:48.552210 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:17:48.552210 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:48.552210 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:17:48.553249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:48.553249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:17:48.553249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:17:48.553249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:17:48.553249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:17:48.553249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:17:48.553249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:17:48.553249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:48.553249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:17:48.553249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:48.553249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:17:48.553249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:48.553578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:17:48.553578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:48.553578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:17:48.553578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:48.553578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:17:48.553578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:17:48.553578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:48.553578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:17:48.553578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:48.553578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:17:48.553578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:17:48.553578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:48.553578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:17:48.553916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:48.553916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:17:48.554525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:48.554525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:17:48.554525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:17:48.554525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:17:48.554525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:17:48.554525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:17:48.554525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:17:48.554525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:17:48.554525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:17:48.554525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:17:48.554525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:17:48.554525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:17:48.554525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:17:48.554525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:17:48.554525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:17:48.554525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:17:48.555019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:17:48.555019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:17:48.555019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:48.555019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:17:48.555019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:48.555019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:17:48.555019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:48.555019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:17:48.555019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:48.555019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:17:48.555019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:48.555019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:17:48.555019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:17:48.555019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:48.555019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:17:48.555019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:48.555019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:17:48.555019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:17:48.555019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:48.555629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:17:48.555629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:48.555629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:17:52.967426 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:17:52.968292 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:17:52.968292 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:17:52.968944 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:17:52.975194 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG dbcounter [-] [106314] Writing DB stats neutron:SELECT=241 {{(pid=106314) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 09 09:17:53.008264 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:17:53.008443 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:53.009045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:17:53.009045 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:53.009045 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:17:53.009045 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:17:53.009045 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:53.009045 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:17:53.009045 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:53.009045 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:17:53.010316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:53.010316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:17:53.010316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:17:53.010316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:17:53.010316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:17:53.010316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:17:53.010316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:17:53.010316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:53.010316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:17:53.010316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:53.010316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:17:53.010316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:53.010781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:17:53.010781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:53.010781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:17:53.010781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:53.010781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:17:53.010781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:17:53.010781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:53.010781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:17:53.010781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:53.010781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:17:53.010781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:17:53.010781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:53.010781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:17:53.011171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:53.011171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:17:53.011420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:53.011420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:17:53.011420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:17:53.011420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:17:53.011420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:17:53.011420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:17:53.011420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:17:53.011420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:17:53.011420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:17:53.011420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:17:53.011420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:17:53.011420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:17:53.011420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:17:53.011420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:17:53.011420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:17:53.011420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:17:53.011948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:17:53.011948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:17:53.011948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:53.011948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:17:53.011948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:53.011948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:17:53.011948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:53.011948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:17:53.011948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:53.011948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:17:53.011948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:53.011948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:17:53.011948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:17:53.011948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:53.011948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:17:53.011948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:53.011948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:17:53.011948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:17:53.011948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:53.012668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:17:53.012668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:53.012668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:17:53.522016 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:17:53.522555 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:17:53.522869 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:17:53.523849 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:17:53.524089 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:17:53.524283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:53.525105 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:17:53.525105 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:53.525105 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:17:53.525105 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:17:53.525105 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:53.525105 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:17:53.525105 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:53.525105 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:17:53.526734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:53.526734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:17:53.526734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:17:53.526734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:17:53.526734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:17:53.526734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:17:53.526734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:17:53.526734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:53.526734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:17:53.526734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:53.526734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:17:53.526734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:53.527261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:17:53.527261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:53.527261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:17:53.527261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:53.527261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:17:53.527261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:17:53.527261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:53.527261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:17:53.527261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:53.527261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:17:53.527261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:17:53.527261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:53.527261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:17:53.527708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:53.527708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:17:53.528148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:53.528148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:17:53.528148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:17:53.528148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:17:53.528148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:17:53.528148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:17:53.528148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:17:53.528148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:17:53.528148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:17:53.528148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:17:53.528148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:17:53.528148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:17:53.528148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:17:53.528148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:17:53.528148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:17:53.528148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:17:53.528643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:17:53.528643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:17:53.528643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:53.528643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:17:53.528643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:53.528643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:17:53.528643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:53.528643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:17:53.528643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:53.528643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:17:53.528643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:53.528643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:17:53.528643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:17:53.528643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:53.528643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:17:53.528643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:53.528643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:17:53.528643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:17:53.528643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:53.529274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:17:53.529274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:53.529274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:17:53.533273 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:17:53.533622 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:17:53.533736 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:17:53.533951 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:17:53.534313 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:17:53.534581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:53.535130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:17:53.535130 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:53.535130 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:17:53.535130 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:17:53.535130 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:53.535130 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:17:53.535130 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:53.535130 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:17:53.535959 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:53.535959 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:17:53.535959 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:17:53.535959 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:17:53.535959 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:53.535959 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:17:53.535959 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:53.535959 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:17:53.535959 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:53.535959 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:17:53.535959 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:53.535959 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:17:53.535959 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:53.535959 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:17:53.535959 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:17:53.536426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:53.536426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:17:53.536426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:53.536426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:17:53.536426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:17:53.536426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:53.536426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:17:53.536426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:53.536426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:17:53.537117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:53.537117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:17:53.537117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:17:53.537117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:17:53.537117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:17:53.537117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:17:53.537117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:17:53.537117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:17:53.537117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:17:53.537117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:17:53.537117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:53.537117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:17:53.537508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:53.537508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:17:53.537508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:53.537508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:17:53.537508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:53.537508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:17:53.537508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:53.537508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:17:53.537508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:17:53.537508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:53.537508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:17:53.537508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:53.537508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:17:53.537508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:17:53.538020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:53.538020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:17:53.538020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:53.538020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:17:53.538436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:53.538436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:17:53.538436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:17:53.538436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:17:53.538436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:17:53.538436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:17:53.538436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:17:53.538436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:17:53.538436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:17:53.538436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:17:53.538436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:17:53.538436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:17:53.538436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:17:53.538436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:17:53.538436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:17:53.538436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:17:53.538932 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:17:53.538932 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:17:53.538932 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:17:53.538932 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:17:53.538932 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:17:53.538932 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:53.538932 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:17:53.538932 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:53.538932 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:17:53.538932 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:53.538932 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:17:53.538932 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:53.538932 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:17:53.538932 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:53.538932 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:17:53.538932 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:17:53.538932 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:53.538932 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:17:53.539565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:53.539565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:17:53.539565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:17:53.539565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:53.539565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:17:53.539565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:53.539565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:17:53.541601 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:17:53.542070 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:17:53.542191 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:17:53.542320 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:17:53.542918 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:17:53.543316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:53.544153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:17:53.544153 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:53.544153 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:17:53.544153 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:17:53.544153 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:53.544153 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:17:53.544153 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:53.544153 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:17:53.545065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:53.545065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:17:53.545065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:17:53.545065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:17:53.545065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:53.545065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:17:53.545065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:53.545065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:17:53.545065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:53.545065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:17:53.545065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:53.545065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:17:53.545065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:53.545065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:17:53.545065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:17:53.545609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:53.545609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:17:53.545609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:53.545609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:17:53.545609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:17:53.545609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:53.545609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:17:53.545609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:53.545609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:17:53.546196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:53.546196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:17:53.546196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:17:53.546196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:17:53.546196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:17:53.546196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:17:53.546196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:17:53.546196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:17:53.546196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:17:53.546196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:53.546196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:17:53.546560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:53.546560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:17:53.546560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:53.546560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:17:53.546560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:53.546560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:17:53.546560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:53.546560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:17:53.546560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:17:53.546560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:53.546560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:17:53.546560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:53.546560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:17:53.546560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:17:53.547026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:53.547026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:17:53.547026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:53.547026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:17:53.547305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:53.547305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:17:53.547305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:17:53.547305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:17:53.547305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:17:53.547305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:17:53.547305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:17:53.547305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:17:53.547305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:17:53.547305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:17:53.547305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:17:53.547305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:17:53.547305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:17:53.547305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:17:53.547305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:17:53.547305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:17:53.547305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:17:53.548065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:17:53.548065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:17:53.548065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:17:53.548065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:53.548065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:17:53.548065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:53.548065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:17:53.548065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:53.548065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:17:53.548065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:53.548065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:17:53.548065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:53.548065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:17:53.548065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:17:53.548065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:53.548065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:17:53.548065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:53.548065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:17:53.548065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:17:53.548636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:53.548636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:17:53.548636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:53.548636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:17:53.550258 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:17:53.550558 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:17:53.550672 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:17:53.550858 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:17:53.551774 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:17:53.551774 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:17:53.551774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:53.552261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:17:53.552261 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:53.552261 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:17:53.552261 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:17:53.552261 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:53.552261 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:17:53.552261 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:53.552261 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:17:53.553163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:53.553163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:17:53.553163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:17:53.553163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:17:53.553163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:17:53.553163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:17:53.553163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:17:53.553163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:53.553163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:17:53.553163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:53.553163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:17:53.553163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:53.553530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:17:53.553530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:53.553530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:17:53.553530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:53.553530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:17:53.553530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:17:53.553530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:53.553530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:17:53.553530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:53.553530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:17:53.553530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:17:53.553530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:53.553530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:17:53.553907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:53.553907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:17:53.554303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:53.554303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:17:53.554303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:17:53.554303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:17:53.554303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:17:53.554303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:17:53.554303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:17:53.554303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:17:53.554303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:17:53.554303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:17:53.554303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:17:53.554303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:17:53.554303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:17:53.554303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:17:53.554303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:17:53.554303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:17:53.554791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:17:53.554791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:17:53.554791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:53.554791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:17:53.554791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:53.554791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:17:53.554791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:53.554791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:17:53.554791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:53.554791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:17:53.554791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:53.554791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:17:53.554791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:17:53.554791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:53.554791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:17:53.554791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:53.554791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:17:53.554791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:17:53.554791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:53.555322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:17:53.555322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:53.555322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:17:57.967983 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:17:57.968559 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:17:57.968559 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:17:57.969053 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:17:58.000971 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:17:58.003788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:58.003788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:17:58.003788 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:58.003788 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:17:58.003788 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:17:58.003788 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:58.003788 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:17:58.003788 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:58.003788 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:17:58.003788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:58.003788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:17:58.003788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:17:58.003788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:17:58.003788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:17:58.003788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:17:58.004559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:17:58.004559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:58.004559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:17:58.004559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:58.004559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:17:58.004559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:58.004559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:17:58.004559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:58.004559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:17:58.004559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:58.004559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:17:58.004559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:17:58.004559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:58.004559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:17:58.004872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:58.004872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:17:58.004872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:17:58.004872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:58.004872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:17:58.004872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:58.004872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:17:58.004872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:58.004872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:17:58.004872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:17:58.004872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:17:58.004872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:17:58.004872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:17:58.005133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:17:58.005133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:17:58.005133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:17:58.005133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:17:58.005133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:17:58.005133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:17:58.005133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:17:58.005133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:17:58.005133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:17:58.005133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:17:58.005133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:17:58.005133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:17:58.005133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:58.005133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:17:58.005133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:58.005133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:17:58.005133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:58.005133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:17:58.005133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:58.005133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:17:58.005614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:58.005614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:17:58.005614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:17:58.005614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:58.005614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:17:58.005614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:58.005614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:17:58.005614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:17:58.005614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:58.005614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:17:58.005614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:58.005614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:17:58.522697 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:17:58.523313 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:17:58.523460 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:17:58.524593 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:17:58.524962 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:17:58.525215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:58.526162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:17:58.526162 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:58.526162 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:17:58.526162 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:17:58.526162 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:58.526162 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:17:58.526162 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:58.526162 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:17:58.529714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:58.529714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:17:58.529714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:17:58.529714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:17:58.529714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:17:58.529714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:17:58.529714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:17:58.529714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:58.529714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:17:58.529714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:58.529714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:17:58.529714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:58.530021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:17:58.530021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:58.530021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:17:58.530021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:58.530021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:17:58.530021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:17:58.530021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:58.530021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:17:58.530021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:58.530021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:17:58.530021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:17:58.530021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:58.530021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:17:58.530270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:58.530270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:17:58.530270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:58.530270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:17:58.530270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:17:58.530270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:17:58.530270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:17:58.530270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:17:58.530270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:17:58.530270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:17:58.530270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:17:58.530270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:17:58.530270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:17:58.530270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:17:58.530270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:17:58.530568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:17:58.530568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:17:58.530568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:17:58.530568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:17:58.530568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:17:58.530568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:58.530568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:17:58.530568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:58.530568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:17:58.530568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:58.530568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:17:58.530568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:58.530568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:17:58.530568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:58.530568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:17:58.530568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:17:58.530568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:58.530568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:17:58.530568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:58.530936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:17:58.530936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:17:58.530936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:58.530936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:17:58.530936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:58.530936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:17:58.533832 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:17:58.534193 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:17:58.534309 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:17:58.534480 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:17:58.535156 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:17:58.535379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:58.536032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:17:58.536032 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:58.536032 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:17:58.536032 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:17:58.536032 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:58.536032 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:17:58.536032 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:58.536032 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:17:58.537081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:58.537081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:17:58.537081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:17:58.537081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:17:58.537081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:58.537081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:17:58.537081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:58.537081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:17:58.537081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:58.537081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:17:58.537081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:58.537081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:17:58.537081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:58.537081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:17:58.537081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:17:58.537425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:58.537425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:17:58.537425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:58.537425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:17:58.537425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:17:58.537425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:58.537425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:17:58.537425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:58.537425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:17:58.538156 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:58.538156 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:17:58.538156 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:17:58.538156 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:17:58.538156 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:17:58.538156 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:17:58.538156 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:17:58.538156 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:17:58.538156 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:17:58.538156 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:17:58.538156 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:58.538156 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:17:58.538486 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:58.538486 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:17:58.538486 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:58.538486 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:17:58.538486 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:58.538486 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:17:58.538486 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:58.538486 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:17:58.538486 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:17:58.538486 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:58.538486 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:17:58.538486 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:58.538486 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:17:58.538486 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:17:58.538775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:58.538775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:17:58.538775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:58.538775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:17:58.539716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:58.539716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:17:58.539716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:17:58.539716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:17:58.539716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:17:58.539716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:17:58.539716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:17:58.539716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:17:58.539716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:17:58.539716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:17:58.539716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:17:58.539716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:17:58.539716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:17:58.539716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:17:58.539716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:17:58.539716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:17:58.540115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:17:58.540115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:17:58.540115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:17:58.540115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:17:58.540115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:17:58.540115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:58.540115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:17:58.540115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:58.540115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:17:58.540115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:58.540115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:17:58.540115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:58.540115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:17:58.540115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:58.540115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:17:58.540115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:17:58.540115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:58.540115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:17:58.540533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:58.540533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:17:58.540533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:17:58.540533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:58.540533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:17:58.540533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:58.540533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:17:58.542188 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:17:58.542528 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:17:58.542683 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:17:58.542891 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:17:58.543400 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:17:58.543664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:58.544253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:17:58.544253 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:58.544253 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:17:58.544253 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:17:58.544253 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:58.544253 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:17:58.544253 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:58.544253 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:17:58.545250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:58.545250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:17:58.545250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:17:58.545250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:17:58.545250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:58.545250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:17:58.545250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:58.545250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:17:58.545250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:58.545250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:17:58.545250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:58.545250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:17:58.545250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:58.545250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:17:58.545250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:17:58.546087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:58.546087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:17:58.546087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:58.546087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:17:58.546087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:17:58.546087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:58.546087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:17:58.546087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:58.546087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:17:58.546594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:58.546594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:17:58.546594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:17:58.546594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:17:58.546594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:17:58.546594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:17:58.546594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:17:58.546594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:17:58.546594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:17:58.546594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:58.546594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:17:58.547089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:58.547089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:17:58.547089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:58.547089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:17:58.547089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:58.547089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:17:58.547089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:58.547089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:17:58.547089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:17:58.547089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:58.547089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:17:58.547089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:58.547089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:17:58.547089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:17:58.547677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:58.547677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:17:58.547677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:58.547677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:17:58.548312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:58.548312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:17:58.548312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:17:58.548312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:17:58.548312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:17:58.548312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:17:58.548312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:17:58.548312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:17:58.548312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:17:58.548312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:17:58.548312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:17:58.548312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:17:58.548312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:17:58.548312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:17:58.548312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:17:58.548312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:17:58.548312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:17:58.548658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:17:58.548658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:17:58.548658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:17:58.548658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:58.548658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:17:58.548658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:58.548658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:17:58.548658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:58.548658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:17:58.548658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:58.548658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:17:58.548658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:58.548658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:17:58.548658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:17:58.548658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:58.548658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:17:58.548658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:58.548658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:17:58.548658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:17:58.549013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:58.549013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:17:58.549013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:58.549013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:17:58.550583 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:17:58.550883 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:17:58.551005 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:17:58.551167 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:17:58.551668 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:17:58.551909 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:17:58.552029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:58.552672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:17:58.552672 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:58.552672 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:17:58.552672 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:17:58.552672 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:58.552672 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:17:58.552672 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:58.552672 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:17:58.554219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:58.554219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:17:58.554219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:17:58.554219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:17:58.554219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:17:58.554219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:17:58.554219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:17:58.554219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:58.554219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:17:58.554219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:58.554219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:17:58.554219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:58.554445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:17:58.554445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:58.554445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:17:58.554445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:58.554445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:17:58.554445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:17:58.554445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:58.554445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:17:58.554445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:58.554445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:17:58.554445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:17:58.554445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:58.554445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:17:58.554689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:58.554689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:17:58.556878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:58.556878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:17:58.556878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:17:58.556878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:17:58.556878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:17:58.556878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:17:58.556878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:17:58.556878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:17:58.556878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:17:58.556878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:17:58.556878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:17:58.556878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:17:58.556878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:17:58.556878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:17:58.556878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:17:58.556878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:17:58.557180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:17:58.557180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:17:58.557180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:58.557180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:17:58.557180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:17:58.557180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:17:58.557180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:17:58.557180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:17:58.557180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:17:58.557180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:17:58.557180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:17:58.557180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:17:58.557180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:17:58.557180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:17:58.557180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:17:58.557180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:17:58.557180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:17:58.557180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:17:58.557180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:17:58.557521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:17:58.557521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:17:58.557521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:18:02.968717 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:18:02.969279 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:18:02.972784 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:18:02.972784 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:18:03.001942 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:18:03.002379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:03.003212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:18:03.003212 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:03.003212 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:18:03.003212 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:18:03.003212 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:03.003212 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:18:03.003212 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:03.003212 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:18:03.004012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:03.004012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:18:03.004012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:18:03.004012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:18:03.004012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:18:03.004012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:18:03.004012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:18:03.004012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:03.004012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:18:03.004012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:03.004012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:18:03.004012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:03.004271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:18:03.004271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:03.004271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:18:03.004271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:03.004271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:18:03.004271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:18:03.004271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:03.004271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:18:03.004271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:03.004271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:18:03.004271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:18:03.004271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:03.004271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:18:03.004915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:03.004915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:18:03.004915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:03.004915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:18:03.004915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:18:03.004915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:18:03.004915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:18:03.004915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:18:03.004915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:18:03.004915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:18:03.004915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:18:03.004915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:18:03.004915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:18:03.004915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:18:03.004915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:18:03.005323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:18:03.005323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:18:03.005323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:18:03.005323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:18:03.005323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:18:03.005323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:03.005323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:18:03.005323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:03.005323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:18:03.005323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:03.005323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:18:03.005323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:03.005323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:18:03.005323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:03.005323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:18:03.005323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:18:03.005323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:03.005323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:18:03.005323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:03.005670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:18:03.005670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:18:03.005670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:03.005670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:18:03.005670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:03.005670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:18:03.523705 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:18:03.524651 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:18:03.524651 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:18:03.526184 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:18:03.526694 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:18:03.526891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:03.528409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:18:03.528409 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:03.528409 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:18:03.528409 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:18:03.528409 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:03.528409 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:18:03.528409 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:03.528409 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:18:03.529844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:03.529844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:18:03.529844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:18:03.529844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:18:03.529844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:18:03.529844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:18:03.529844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:18:03.529844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:03.529844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:18:03.529844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:03.529844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:18:03.529844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:03.530120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:18:03.530120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:03.530120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:18:03.530120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:03.530120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:18:03.530120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:18:03.530120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:03.530120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:18:03.530120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:03.530120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:18:03.530120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:18:03.530120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:03.530120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:18:03.530384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:03.530384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:18:03.531256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:03.531256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:18:03.531256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:18:03.531256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:18:03.531256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:18:03.531256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:18:03.531256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:18:03.531256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:18:03.531256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:18:03.531256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:18:03.531256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:18:03.531256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:18:03.531256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:18:03.531256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:18:03.531256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:18:03.531256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:18:03.531603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:18:03.531603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:18:03.531603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:03.531603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:18:03.531603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:03.531603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:18:03.531603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:03.531603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:18:03.531603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:03.531603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:18:03.531603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:03.531603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:18:03.531603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:18:03.531603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:03.531603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:18:03.531603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:03.531603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:18:03.531603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:18:03.531603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:03.532030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:18:03.532030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:03.532030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:18:03.534348 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:18:03.534766 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:18:03.534912 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:18:03.535119 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:18:03.535668 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:18:03.536713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:03.537467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:18:03.537467 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:03.537467 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:18:03.537467 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:18:03.537467 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:03.537467 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:18:03.537467 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:03.537467 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:18:03.538683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:03.538683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:18:03.538683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:18:03.538683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:18:03.538683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:03.538683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:18:03.538683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:03.538683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:18:03.538683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:03.538683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:18:03.538683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:03.538683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:18:03.538683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:03.538683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:18:03.538683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:18:03.539592 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:03.539592 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:18:03.539592 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:03.539592 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:18:03.539592 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:18:03.539592 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:03.539592 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:18:03.539592 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:03.539592 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:18:03.540047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:03.540047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:18:03.540047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:18:03.540047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:18:03.540047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:18:03.540047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:18:03.540047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:18:03.540047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:18:03.540047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:18:03.540047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:18:03.540047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:03.540047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:18:03.540683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:03.540683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:18:03.540683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:03.540683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:18:03.540683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:03.540683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:18:03.540683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:03.540683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:18:03.540683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:18:03.540683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:03.540683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:18:03.540683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:03.540683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:18:03.540683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:18:03.541359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:03.541359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:18:03.541359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:03.541359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:18:03.541604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:03.541604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:18:03.541604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:18:03.541604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:18:03.541604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:18:03.541604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:18:03.541604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:18:03.541604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:18:03.541604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:18:03.541604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:18:03.541604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:18:03.541604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:18:03.541604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:18:03.541604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:18:03.541604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:18:03.541604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:18:03.542277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:18:03.542277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:18:03.542277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:18:03.542277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:18:03.542277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:18:03.542277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:03.542277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:18:03.542277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:03.542277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:18:03.542277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:03.542277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:18:03.542277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:03.542277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:18:03.542277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:03.542277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:18:03.542277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:18:03.542277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:03.542277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:18:03.543027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:03.543027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:18:03.543027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:18:03.543027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:03.543027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:18:03.543027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:03.543027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:18:03.543027 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:18:03.543407 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:18:03.543407 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:18:03.543523 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:18:03.544296 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:18:03.544583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:03.545316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:18:03.545316 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:03.545316 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:18:03.545316 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:18:03.545316 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:03.545316 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:18:03.545316 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:03.545316 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:18:03.546058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:03.546058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:18:03.546058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:18:03.546058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:18:03.546058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:03.546058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:18:03.546058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:03.546058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:18:03.546058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:03.546058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:18:03.546058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:03.546058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:18:03.546058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:03.546058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:18:03.546058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:18:03.546552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:03.546552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:18:03.546552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:03.546552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:18:03.546552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:18:03.546552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:03.546552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:18:03.546552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:03.546552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:18:03.546863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:03.546863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:18:03.546863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:18:03.546863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:18:03.546863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:18:03.546863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:18:03.546863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:18:03.546863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:18:03.546863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:18:03.546863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:03.546863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:18:03.547332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:03.547332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:18:03.547332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:03.547332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:18:03.547332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:03.547332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:18:03.547332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:03.547332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:18:03.547332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:18:03.547332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:03.547332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:18:03.547332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:03.547332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:18:03.547332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:18:03.547788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:03.547788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:18:03.547788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:03.547788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:18:03.548462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:03.548462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:18:03.548462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:18:03.548462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:18:03.548462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:18:03.548462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:18:03.548462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:18:03.548462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:18:03.548462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:18:03.548462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:18:03.548462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:18:03.548462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:18:03.548462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:18:03.548462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:18:03.548462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:18:03.548462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:18:03.548462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:18:03.549146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:18:03.549146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:18:03.549146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:18:03.549146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:03.549146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:18:03.549146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:03.549146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:18:03.549146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:03.549146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:18:03.549146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:03.549146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:18:03.549146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:03.549146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:18:03.549146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:18:03.549146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:03.549146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:18:03.549146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:03.549146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:18:03.549146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:18:03.549814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:03.549814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:18:03.549814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:03.549814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:18:03.551116 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:18:03.551537 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:18:03.551695 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:18:03.552008 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:18:03.552655 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:18:03.552987 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:18:03.553172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:03.554142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:18:03.554142 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:03.554142 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:18:03.554142 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:18:03.554142 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:03.554142 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:18:03.554142 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:03.554142 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:18:03.555470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:03.555470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:18:03.555470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:18:03.555470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:18:03.555470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:18:03.555470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:18:03.555470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:18:03.555470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:03.555470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:18:03.555470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:03.555470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:18:03.555470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:03.556120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:18:03.556120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:03.556120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:18:03.556120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:03.556120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:18:03.556120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:18:03.556120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:03.556120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:18:03.556120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:03.556120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:18:03.556120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:18:03.556120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:03.556120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:18:03.556903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:03.556903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:18:03.557513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:03.557513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:18:03.557513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:18:03.557513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:18:03.557513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:18:03.557513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:18:03.557513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:18:03.557513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:18:03.557513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:18:03.557513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:18:03.557513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:18:03.557513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:18:03.557513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:18:03.557513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:18:03.557513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:18:03.557513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:18:03.558287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:18:03.558287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:18:03.558287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:03.558287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:18:03.558287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:03.558287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:18:03.558287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:03.558287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:18:03.558287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:03.558287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:18:03.558287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:03.558287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:18:03.558287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:18:03.558287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:03.558287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:18:03.558287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:03.558287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:18:03.558287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:18:03.558287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:03.559177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:18:03.559177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:03.559177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:18:07.969358 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:18:07.970310 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:18:07.970310 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:18:07.970993 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:18:08.007589 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:18:08.007812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:08.008655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:18:08.008655 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:08.008655 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:18:08.008655 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:18:08.008655 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:08.008655 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:18:08.008655 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:08.008655 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:18:08.009548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:08.009548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:18:08.009548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:18:08.009548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:18:08.009548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:18:08.009548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:18:08.009548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:18:08.009548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:08.009548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:18:08.009548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:08.009548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:18:08.009548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:08.010029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:18:08.010029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:08.010029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:18:08.010029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:08.010029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:18:08.010029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:18:08.010029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:08.010029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:18:08.010029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:08.010029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:18:08.010029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:18:08.010029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:08.010029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:18:08.010419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:08.010419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:18:08.010492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:08.010492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:18:08.010492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:18:08.010492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:18:08.010492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:18:08.010492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:18:08.010492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:18:08.010492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:18:08.010492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:18:08.010492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:18:08.010492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:18:08.010492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:18:08.010492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:18:08.010492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:18:08.010492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:18:08.010492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:18:08.011009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:18:08.011009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:18:08.011009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:08.011009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:18:08.011009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:08.011009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:18:08.011009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:08.011009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:18:08.011009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:08.011009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:18:08.011009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:08.011009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:18:08.011009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:18:08.011009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:08.011009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:18:08.011009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:08.011009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:18:08.011009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:18:08.011009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:08.011560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:18:08.011560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:08.011560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:18:08.524261 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:18:08.524777 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:18:08.525023 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:18:08.526106 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:18:08.526286 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:18:08.526395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:08.527181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:18:08.527181 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:08.527181 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:18:08.527181 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:18:08.527181 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:08.527181 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:18:08.527181 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:08.527181 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:18:08.528903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:08.528903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:18:08.528903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:18:08.528903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:18:08.528903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:18:08.528903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:18:08.528903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:18:08.528903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:08.528903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:18:08.528903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:08.528903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:18:08.528903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:08.529615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:18:08.529615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:08.529615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:18:08.529615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:08.529615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:18:08.529615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:18:08.529615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:08.529615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:18:08.529615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:08.529615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:18:08.529615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:18:08.529615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:08.529615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:18:08.530537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:08.530537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:18:08.531341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:08.531341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:18:08.531341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:18:08.531341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:18:08.531341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:18:08.531341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:18:08.531341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:18:08.531341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:18:08.531341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:18:08.531341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:18:08.531341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:18:08.531341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:18:08.531341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:18:08.531341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:18:08.531341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:18:08.531341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:18:08.532158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:18:08.532158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:18:08.532158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:08.532158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:18:08.532158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:08.532158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:18:08.532158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:08.532158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:18:08.532158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:08.532158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:18:08.532158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:08.532158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:18:08.532158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:18:08.532158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:08.532158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:18:08.532158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:08.532158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:18:08.532158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:18:08.532158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:08.533059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:18:08.533059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:08.533059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:18:08.534853 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:18:08.535121 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:18:08.535212 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:18:08.535362 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:18:08.535669 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:18:08.535841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:08.536290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:18:08.536290 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:08.536290 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:18:08.536290 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:18:08.536290 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:08.536290 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:18:08.536290 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:08.536290 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:18:08.537113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:08.537113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:18:08.537113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:18:08.537113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:18:08.537113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:08.537113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:18:08.537113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:08.537113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:18:08.537113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:08.537113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:18:08.537113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:08.537113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:18:08.537113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:08.537113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:18:08.537113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:18:08.537561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:08.537561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:18:08.537561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:08.537561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:18:08.537561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:18:08.537561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:08.537561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:18:08.537561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:08.537561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:18:08.538064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:08.538064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:18:08.538064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:18:08.538064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:18:08.538064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:18:08.538064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:18:08.538064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:18:08.538064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:18:08.538064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:18:08.538064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:18:08.538064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:08.538064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:18:08.538424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:08.538424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:18:08.538424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:08.538424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:18:08.538424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:08.538424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:18:08.538424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:08.538424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:18:08.538424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:18:08.538424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:08.538424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:18:08.538424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:08.538424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:18:08.538424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:18:08.538828 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:08.538828 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:18:08.538828 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:08.538828 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:18:08.539293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:08.539293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:18:08.539293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:18:08.539293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:18:08.539293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:18:08.539293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:18:08.539293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:18:08.539293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:18:08.539293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:18:08.539293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:18:08.539293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:18:08.539293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:18:08.539293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:18:08.539293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:18:08.539293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:18:08.539293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:18:08.539735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:18:08.539735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:18:08.539735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:18:08.539735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:18:08.539735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:18:08.539735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:08.539735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:18:08.539735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:08.539735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:18:08.539735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:08.539735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:18:08.539735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:08.539735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:18:08.539735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:08.539735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:18:08.539735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:18:08.539735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:08.539735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:18:08.540353 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:08.540353 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:18:08.540353 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:18:08.540353 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:08.540353 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:18:08.540353 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:08.540353 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:18:08.543086 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:18:08.543363 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:18:08.543479 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:18:08.543614 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:18:08.543947 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:18:08.544118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:08.544564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:18:08.544564 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:08.544564 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:18:08.544564 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:18:08.544564 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:08.544564 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:18:08.544564 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:08.544564 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:18:08.545367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:08.545367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:18:08.545367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:18:08.545367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:18:08.545367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:08.545367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:18:08.545367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:08.545367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:18:08.545367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:08.545367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:18:08.545367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:08.545367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:18:08.545367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:08.545367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:18:08.545367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:18:08.545777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:08.545777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:18:08.545777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:08.545777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:18:08.545777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:18:08.545777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:08.545777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:18:08.545777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:08.545777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:18:08.546362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:08.546362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:18:08.546362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:18:08.546362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:18:08.546362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:18:08.546362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:18:08.546362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:18:08.546362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:18:08.546362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:18:08.546362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:08.546362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:18:08.546688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:08.546688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:18:08.546688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:08.546688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:18:08.546688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:08.546688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:18:08.546688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:08.546688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:18:08.546688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:18:08.546688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:08.546688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:18:08.546688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:08.546688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:18:08.546688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:18:08.547094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:08.547094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:18:08.547094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:08.547094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:18:08.547433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:08.547433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:18:08.547433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:18:08.547433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:18:08.547433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:18:08.547433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:18:08.547433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:18:08.547433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:18:08.547433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:18:08.547433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:18:08.547433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:18:08.547433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:18:08.547433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:18:08.547433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:18:08.547433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:18:08.547433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:18:08.547433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:18:08.547944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:18:08.547944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:18:08.547944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:18:08.547944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:08.547944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:18:08.547944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:08.547944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:18:08.547944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:08.547944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:18:08.547944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:08.547944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:18:08.547944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:08.547944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:18:08.547944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:18:08.547944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:08.547944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:18:08.547944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:08.547944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:18:08.547944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:18:08.548451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:08.548451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:18:08.548451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:08.548451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:18:08.551555 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:18:08.551899 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:18:08.552066 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:18:08.552279 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:18:08.552828 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:18:08.553187 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:18:08.553410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:08.554270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:18:08.554270 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:08.554270 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:18:08.554270 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:18:08.554270 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:08.554270 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:18:08.554270 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:08.554270 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:18:08.555269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:08.555269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:18:08.555269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:18:08.555269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:18:08.555269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:18:08.555269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:18:08.555269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:18:08.555269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:08.555269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:18:08.555269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:08.555269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:18:08.555269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:08.555635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:18:08.555635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:08.555635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:18:08.555635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:08.555635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:18:08.555635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:18:08.555635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:08.555635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:18:08.555635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:08.555635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:18:08.555635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:18:08.555635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:08.555635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:18:08.556020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:08.556020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:18:08.556303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:08.556303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:18:08.556303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:18:08.556303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:18:08.556303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:18:08.556303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:18:08.556303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:18:08.556303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:18:08.556303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:18:08.556303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:18:08.556303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:18:08.556303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:18:08.556303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:18:08.556303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:18:08.556303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:18:08.556303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:18:08.556732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:18:08.556732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:18:08.556732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:08.556732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:18:08.556732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:08.556732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:18:08.556732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:08.556732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:18:08.556732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:08.556732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:18:08.556732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:08.556732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:18:08.556732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:18:08.556732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:08.556732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:18:08.556732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:08.556732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:18:08.556732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:18:08.556732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:08.557263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:18:08.557263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:08.557263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:18:12.969963 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:18:12.970615 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:18:12.970668 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:18:12.971538 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:18:13.008843 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:18:13.009114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:13.009855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:18:13.009855 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:13.009855 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:18:13.009855 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:18:13.009855 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:13.009855 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:18:13.009855 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:13.009855 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:18:13.010576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:13.010576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:18:13.010576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:18:13.010576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:18:13.010576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:18:13.010576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:18:13.010576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:18:13.010576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:13.010576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:18:13.010576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:13.010576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:18:13.010576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:13.010989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:18:13.010989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:13.010989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:18:13.010989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:13.010989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:18:13.010989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:18:13.010989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:13.010989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:18:13.010989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:13.010989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:18:13.010989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:18:13.010989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:13.010989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:18:13.011334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:13.011334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:18:13.011334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:13.011334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:18:13.011334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:18:13.011334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:18:13.011334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:18:13.011334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:18:13.011334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:18:13.011334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:18:13.011334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:18:13.011334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:18:13.011334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:18:13.011334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:18:13.011334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:18:13.011860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:18:13.011860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:18:13.011860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:18:13.011860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:18:13.011860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:18:13.011860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:13.011860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:18:13.011860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:13.011860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:18:13.011860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:13.011860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:18:13.011860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:13.011860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:18:13.011860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:13.011860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:18:13.011860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:18:13.011860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:13.011860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:18:13.011860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:13.012525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:18:13.012525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:18:13.012525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:13.012525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:18:13.012525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:13.012525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:18:13.525125 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:18:13.526573 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:18:13.526573 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:18:13.526573 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:18:13.526573 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:18:13.526834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:13.527301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:18:13.527301 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:13.527301 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:18:13.527301 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:18:13.527301 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:13.527301 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:18:13.527301 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:13.527301 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:18:13.528247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:13.528247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:18:13.528247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:18:13.528247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:18:13.528247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:18:13.528247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:18:13.528247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:18:13.528247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:13.528247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:18:13.528247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:13.528247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:18:13.528247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:13.528801 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:18:13.528801 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:13.528801 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:18:13.528801 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:13.528801 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:18:13.528801 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:18:13.528801 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:13.528801 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:18:13.528801 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:13.528801 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:18:13.528801 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:18:13.528801 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:13.528801 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:18:13.529158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:13.529158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:18:13.529158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:13.529158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:18:13.529158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:18:13.529158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:18:13.529158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:18:13.529158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:18:13.529158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:18:13.529158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:18:13.529158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:18:13.529158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:18:13.529158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:18:13.529158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:18:13.529158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:18:13.529590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:18:13.529590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:18:13.529590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:18:13.529590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:18:13.529590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:18:13.529590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:13.529590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:18:13.529590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:13.529590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:18:13.529590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:13.529590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:18:13.529590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:13.529590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:18:13.529590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:13.529590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:18:13.529590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:18:13.529590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:13.529590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:18:13.529590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:13.530055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:18:13.530055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:18:13.530055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:13.530055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:18:13.530055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:13.530055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:18:13.535286 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:18:13.535822 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:18:13.535946 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:18:13.536085 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:18:13.536789 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:18:13.537092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:13.537895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:18:13.537895 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:13.537895 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:18:13.537895 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:18:13.537895 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:13.537895 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:18:13.537895 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:13.537895 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:18:13.538685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:13.538685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:18:13.538685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:18:13.538685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:18:13.538685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:13.538685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:18:13.538685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:13.538685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:18:13.538685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:13.538685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:18:13.538685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:13.538685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:18:13.538685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:13.538685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:18:13.538685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:18:13.540684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:13.540684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:18:13.540684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:13.540684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:18:13.540684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:18:13.540684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:13.540684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:18:13.540684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:13.540684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:18:13.540684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:13.540684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:18:13.540684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:18:13.540684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:18:13.540684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:18:13.541181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:18:13.541181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:18:13.541181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:18:13.541181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:18:13.541181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:18:13.541181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:13.541181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:18:13.541181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:13.541181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:18:13.541181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:13.541181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:18:13.541181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:13.541181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:18:13.541584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:13.541584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:18:13.541584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:18:13.541584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:13.541584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:18:13.541584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:13.541584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:18:13.541584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:18:13.541584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:13.541584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:18:13.541584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:13.541584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:18:13.542005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:13.542005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:18:13.542005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:18:13.542005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:18:13.542005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:18:13.542005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:18:13.542005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:18:13.542005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:18:13.542005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:18:13.542005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:18:13.542005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:18:13.542005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:18:13.542005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:18:13.542005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:18:13.542005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:18:13.542005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:18:13.542557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:18:13.542557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:18:13.542557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:18:13.542557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:18:13.542557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:18:13.542557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:13.542557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:18:13.542557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:13.542557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:18:13.542557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:13.542557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:18:13.542557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:13.542557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:18:13.542557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:13.542557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:18:13.542557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:18:13.542557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:13.542557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:18:13.543139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:13.543139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:18:13.543139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:18:13.543139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:13.543139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:18:13.543139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:13.543139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:18:13.543471 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:18:13.543870 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:18:13.544043 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:18:13.544280 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:18:13.544837 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:18:13.545197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:13.546028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:18:13.546028 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:13.546028 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:18:13.546028 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:18:13.546028 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:13.546028 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:18:13.546028 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:13.546028 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:18:13.547113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:13.547113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:18:13.547113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:18:13.547113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:18:13.547113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:13.547113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:18:13.547113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:13.547113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:18:13.547113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:13.547113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:18:13.547113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:13.547113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:18:13.547113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:13.547113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:18:13.547113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:18:13.547534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:13.547534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:18:13.547534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:13.547534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:18:13.547534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:18:13.547534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:13.547534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:18:13.547534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:13.547534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:18:13.548459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:13.548459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:18:13.548459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:18:13.548459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:18:13.548459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:18:13.548459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:18:13.548459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:18:13.548459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:18:13.548459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:18:13.548459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:13.548459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:18:13.548965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:13.548965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:18:13.548965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:13.548965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:18:13.548965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:13.548965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:18:13.548965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:13.548965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:18:13.548965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:18:13.548965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:13.548965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:18:13.548965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:13.548965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:18:13.548965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:18:13.549345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:13.549345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:18:13.549345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:13.549345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:18:13.550172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:13.550172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:18:13.550172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:18:13.550172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:18:13.550172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:18:13.550172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:18:13.550172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:18:13.550172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:18:13.550172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:18:13.550172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:18:13.550172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:18:13.550172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:18:13.550172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:18:13.550172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:18:13.550172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:18:13.550172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:18:13.550172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:18:13.550785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:18:13.550785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:18:13.550785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:18:13.550785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:13.550785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:18:13.550785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:13.550785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:18:13.550785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:13.550785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:18:13.550785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:13.550785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:18:13.550785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:13.550785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:18:13.550785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:18:13.550785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:13.550785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:18:13.550785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:13.550785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:18:13.550785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:18:13.551353 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:13.551353 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:18:13.551353 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:13.551353 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:18:13.551949 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:18:13.552321 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:18:13.552491 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:18:13.552721 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:18:13.553220 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:18:13.553826 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:18:13.554044 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:13.554914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:18:13.554914 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:13.554914 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:18:13.554914 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:18:13.554914 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:13.554914 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:18:13.554914 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:13.554914 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:18:13.555935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:13.555935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:18:13.555935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:18:13.555935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:18:13.555935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:18:13.555935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:18:13.555935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:18:13.555935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:13.555935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:18:13.555935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:13.555935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:18:13.555935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:13.556340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:18:13.556340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:13.556340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:18:13.556340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:13.556340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:18:13.556340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:18:13.556340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:13.556340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:18:13.556340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:13.556340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:18:13.556340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:18:13.556340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:13.556340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:18:13.556700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:13.556700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:18:13.557000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:13.557000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:18:13.557000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:18:13.557000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:18:13.557000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:18:13.557000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:18:13.557000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:18:13.557000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:18:13.557000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:18:13.557000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:18:13.557000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:18:13.557000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:18:13.557000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:18:13.557000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:18:13.557000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:18:13.557000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:18:13.557440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:18:13.557440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:18:13.557440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:13.557440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:18:13.557440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:13.557440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:18:13.557440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:13.557440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:18:13.557440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:13.557440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:18:13.557440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:13.557440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:18:13.557440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:18:13.557440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:13.557440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:18:13.557440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:13.557440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:18:13.557440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:18:13.557440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:13.558000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:18:13.558000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:13.558000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:18:17.970659 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:18:17.971722 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:18:17.971722 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:18:17.972385 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:18:18.004584 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:18:18.005170 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:18.006477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:18:18.006477 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:18.006477 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:18:18.006477 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:18:18.006477 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:18.006477 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:18:18.006477 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:18.006477 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:18:18.007966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:18.007966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:18:18.007966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:18:18.007966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:18:18.007966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:18:18.007966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:18:18.007966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:18:18.007966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:18.007966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:18:18.007966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:18.007966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:18:18.007966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:18.008407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:18:18.008407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:18.008407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:18:18.008407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:18.008407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:18:18.008407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:18:18.008407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:18.008407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:18:18.008407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:18.008407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:18:18.008407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:18:18.008407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:18.008407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:18:18.008838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:18.008838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:18:18.008838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:18.008838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:18:18.008838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:18:18.008838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:18:18.008838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:18:18.008838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:18:18.008838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:18:18.008838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:18:18.008838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:18:18.008838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:18:18.008838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:18:18.008838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:18:18.008838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:18:18.009319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:18:18.009319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:18:18.009319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:18:18.009319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:18:18.009319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:18:18.009319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:18.009319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:18:18.009319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:18.009319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:18:18.009319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:18.009319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:18:18.009319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:18.009319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:18:18.009319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:18.009319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:18:18.009319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:18:18.009319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:18.009319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:18:18.009319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:18.010012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:18:18.010012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:18:18.010012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:18.010012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:18:18.010012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:18.010012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:18:18.525459 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:18:18.526083 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:18:18.526281 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:18:18.527557 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:18:18.528028 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:18:18.528365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:18.529618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:18:18.529618 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:18.529618 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:18:18.529618 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:18:18.529618 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:18.529618 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:18:18.529618 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:18.529618 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:18:18.531266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:18.531266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:18:18.531266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:18:18.531266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:18:18.531266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:18:18.531266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:18:18.531266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:18:18.531266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:18.531266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:18:18.531266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:18.531266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:18:18.531266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:18.531647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:18:18.531647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:18.531647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:18:18.531647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:18.531647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:18:18.531647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:18:18.531647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:18.531647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:18:18.531647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:18.531647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:18:18.531647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:18:18.531647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:18.531647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:18:18.532011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:18.532011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:18:18.533575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:18.533575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:18:18.533575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:18:18.533575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:18:18.533575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:18:18.533575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:18:18.533575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:18:18.533575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:18:18.533575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:18:18.533575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:18:18.533575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:18:18.533575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:18:18.533575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:18:18.533575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:18:18.533575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:18:18.533575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:18:18.534022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:18:18.534022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:18:18.534022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:18.534022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:18:18.534022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:18.534022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:18:18.534022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:18.534022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:18:18.534022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:18.534022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:18:18.534022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:18.534022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:18:18.534022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:18:18.534022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:18.534022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:18:18.534022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:18.534022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:18:18.534022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:18:18.534022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:18.534660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:18:18.534660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:18.534660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:18:18.535920 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:18:18.536535 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:18:18.536981 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:18:18.537360 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:18:18.537878 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:18:18.538262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:18.538762 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:18:18.538762 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:18.538762 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:18:18.538762 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:18:18.538762 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:18.538762 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:18:18.538762 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:18.538762 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:18:18.539668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:18.539668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:18:18.539668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:18:18.539668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:18:18.539668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:18.539668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:18:18.539668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:18.539668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:18:18.539668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:18.539668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:18:18.539668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:18.539668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:18:18.539668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:18.539668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:18:18.539668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:18:18.540118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:18.540118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:18:18.540118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:18.540118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:18:18.540118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:18:18.540118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:18.540118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:18:18.540118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:18.540118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:18:18.540524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:18.540524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:18:18.540524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:18:18.540524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:18:18.540524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:18:18.540524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:18:18.540524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:18:18.540524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:18:18.540524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:18:18.540524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:18:18.540524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:18.540524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:18:18.540955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:18.540955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:18:18.540955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:18.540955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:18:18.540955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:18.540955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:18:18.540955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:18.540955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:18:18.540955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:18:18.540955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:18.540955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:18:18.540955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:18.540955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:18:18.540955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:18:18.541338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:18.541338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:18:18.541338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:18.541338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:18:18.541668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:18.541668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:18:18.541668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:18:18.541668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:18:18.541668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:18:18.541668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:18:18.541668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:18:18.541668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:18:18.541668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:18:18.541668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:18:18.541668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:18:18.541668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:18:18.541668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:18:18.541668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:18:18.541668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:18:18.541668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:18:18.542137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:18:18.542137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:18:18.542137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:18:18.542137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:18:18.542137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:18:18.542137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:18.542137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:18:18.542137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:18.542137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:18:18.542137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:18.542137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:18:18.542137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:18.542137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:18:18.542137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:18.542137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:18:18.542137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:18:18.542137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:18.542137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:18:18.542604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:18.542604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:18:18.542604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:18:18.542604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:18.542604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:18:18.542604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:18.542604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:18:18.543940 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:18:18.544272 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:18:18.544390 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:18:18.544525 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:18:18.545038 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:18:18.545423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:18.546246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:18:18.546246 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:18.546246 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:18:18.546246 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:18:18.546246 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:18.546246 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:18:18.546246 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:18.546246 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:18:18.547011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:18.547011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:18:18.547011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:18:18.547011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:18:18.547011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:18.547011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:18:18.547011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:18.547011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:18:18.547011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:18.547011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:18:18.547011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:18.547011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:18:18.547011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:18.547011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:18:18.547011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:18:18.547340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:18.547340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:18:18.547340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:18.547340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:18:18.547340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:18:18.547340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:18.547340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:18:18.547340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:18.547340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:18:18.547893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:18.547893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:18:18.547893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:18:18.547893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:18:18.547893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:18:18.547893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:18:18.547893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:18:18.547893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:18:18.547893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:18:18.547893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:18.547893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:18:18.548185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:18.548185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:18:18.548185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:18.548185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:18:18.548185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:18.548185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:18:18.548185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:18.548185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:18:18.548185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:18:18.548185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:18.548185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:18:18.548185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:18.548185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:18:18.548185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:18:18.548439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:18.548439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:18:18.548439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:18.548439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:18:18.549069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:18.549069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:18:18.549069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:18:18.549069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:18:18.549069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:18:18.549069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:18:18.549069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:18:18.549069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:18:18.549069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:18:18.549069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:18:18.549069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:18:18.549069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:18:18.549069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:18:18.549069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:18:18.549069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:18:18.549069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:18:18.549069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:18:18.549464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:18:18.549464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:18:18.549464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:18:18.549464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:18.549464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:18:18.549464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:18.549464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:18:18.549464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:18.549464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:18:18.549464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:18.549464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:18:18.549464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:18.549464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:18:18.549464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:18:18.549464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:18.549464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:18:18.549464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:18.549464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:18:18.549464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:18:18.550016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:18.550016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:18:18.550016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:18.550016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:18:18.552693 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:18:18.553050 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:18:18.553164 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:18:18.553324 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:18:18.553691 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:18:18.554011 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:18:18.554203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:18.554723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:18:18.554723 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:18.554723 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:18:18.554723 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:18:18.554723 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:18.554723 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:18:18.554723 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:18.554723 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:18:18.555648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:18.555648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:18:18.555648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:18:18.555648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:18:18.555648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:18:18.555648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:18:18.555648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:18:18.555648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:18.555648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:18:18.555648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:18.555648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:18:18.555648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:18.556209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:18:18.556209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:18.556209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:18:18.556209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:18.556209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:18:18.556209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:18:18.556209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:18.556209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:18:18.556209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:18.556209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:18:18.556209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:18:18.556209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:18.556209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:18:18.556723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:18.556723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:18:18.556723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:18.556723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:18:18.556723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:18:18.556723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:18:18.556723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:18:18.556723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:18:18.556723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:18:18.556723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:18:18.556723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:18:18.556723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:18:18.556723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:18:18.556723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:18:18.556723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:18:18.557336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:18:18.557336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:18:18.557336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:18:18.557336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:18:18.557336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:18:18.557336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:18.557336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:18:18.557336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:18.557336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:18:18.557336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:18.557336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:18:18.557336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:18.557336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:18:18.557336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:18.557336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:18:18.557336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:18:18.557336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:18.557336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:18:18.557336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:18.558086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:18:18.558086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:18:18.558086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:18.558086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:18:18.558086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:18.558086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:18:22.971432 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:18:22.972208 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:18:22.972208 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:18:22.973445 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:18:23.013377 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:18:23.014245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:23.014678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:18:23.014678 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:23.014678 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:18:23.014678 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:18:23.014678 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:23.014678 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:18:23.014678 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:23.014678 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:18:23.015814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:23.015814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:18:23.015814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:18:23.015814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:18:23.015814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:18:23.015814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:18:23.015814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:18:23.015814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:23.015814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:18:23.015814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:23.015814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:18:23.015814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:23.016250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:18:23.016250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:23.016250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:18:23.016250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:23.016250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:18:23.016250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:18:23.016250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:23.016250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:18:23.016250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:23.016250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:18:23.016250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:18:23.016250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:23.016250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:18:23.016790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:23.016790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:18:23.017190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:23.017190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:18:23.017190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:18:23.017190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:18:23.017190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:18:23.017190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:18:23.017190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:18:23.017190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:18:23.017190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:18:23.017190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:18:23.017190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:18:23.017190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:18:23.017190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:18:23.017190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:18:23.017190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:18:23.017190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:18:23.017734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:18:23.017734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:18:23.017734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:23.017734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:18:23.017734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:23.017734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:18:23.017734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:23.017734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:18:23.017734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:23.017734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:18:23.017734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:23.017734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:18:23.017734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:18:23.017734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:23.017734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:18:23.017734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:23.017734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:18:23.017734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:18:23.017734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:23.018288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:18:23.018288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:23.018288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:18:23.526640 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:18:23.527115 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:18:23.527366 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:18:23.530085 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:18:23.531035 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:18:23.531035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:23.531632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:18:23.531632 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:23.531632 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:18:23.531632 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:18:23.531632 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:23.531632 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:18:23.531632 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:23.531632 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:18:23.533177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:23.533177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:18:23.533177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:18:23.533177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:18:23.533177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:18:23.533177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:18:23.533177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:18:23.533177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:23.533177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:18:23.533177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:23.533177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:18:23.533177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:23.533582 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:18:23.533582 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:23.533582 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:18:23.533582 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:23.533582 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:18:23.533582 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:18:23.533582 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:23.533582 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:18:23.533582 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:23.533582 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:18:23.533582 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:18:23.533582 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:23.533582 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:18:23.533980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:23.533980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:18:23.535041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:23.535041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:18:23.535041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:18:23.535041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:18:23.535041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:18:23.535041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:18:23.535041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:18:23.535041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:18:23.535041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:18:23.535041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:18:23.535041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:18:23.535041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:18:23.535041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:18:23.535041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:18:23.535041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:18:23.535041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:18:23.535556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:18:23.535556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:18:23.535556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:23.535556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:18:23.535556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:23.535556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:18:23.535556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:23.535556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:18:23.535556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:23.535556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:18:23.535556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:23.535556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:18:23.535556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:18:23.535556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:23.535556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:18:23.535556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:23.535556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:18:23.535556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:18:23.535556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:23.536079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:18:23.536079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:23.536079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:18:23.536519 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:18:23.537010 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:18:23.537279 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:18:23.537463 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:18:23.538049 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:18:23.538265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:23.538905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:18:23.538905 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:23.538905 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:18:23.538905 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:18:23.538905 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:23.538905 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:18:23.538905 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:23.538905 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:18:23.539993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:23.539993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:18:23.539993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:18:23.539993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:18:23.539993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:23.539993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:18:23.539993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:23.539993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:18:23.539993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:23.539993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:18:23.539993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:23.539993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:18:23.539993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:23.539993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:18:23.539993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:18:23.540447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:23.540447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:18:23.540447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:23.540447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:18:23.540447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:18:23.540447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:23.540447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:18:23.540447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:23.540447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:18:23.541349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:23.541349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:18:23.541349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:18:23.541349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:18:23.541349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:18:23.541349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:18:23.541349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:18:23.541349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:18:23.541349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:18:23.541349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:18:23.541349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:23.541349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:18:23.541998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:23.541998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:18:23.541998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:23.541998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:18:23.541998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:23.541998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:18:23.541998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:23.541998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:18:23.541998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:18:23.541998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:23.541998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:18:23.541998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:23.541998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:18:23.541998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:18:23.542320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:23.542320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:18:23.542320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:23.542320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:18:23.543267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:23.543267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:18:23.543267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:18:23.543267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:18:23.543267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:18:23.543267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:18:23.543267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:18:23.543267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:18:23.543267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:18:23.543267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:18:23.543267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:18:23.543267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:18:23.543267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:18:23.543267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:18:23.543267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:18:23.543267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:18:23.543728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:18:23.543728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:18:23.543728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:18:23.543728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:18:23.543728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:18:23.543728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:23.543728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:18:23.543728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:23.543728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:18:23.543728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:23.543728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:18:23.543728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:23.543728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:18:23.543728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:23.543728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:18:23.543728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:18:23.543728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:23.543728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:18:23.544203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:23.544203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:18:23.544203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:18:23.544203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:23.544203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:18:23.544203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:23.544203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:18:23.544902 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:18:23.544902 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:18:23.544982 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:18:23.545913 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:18:23.545913 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:18:23.546092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:23.546585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:18:23.546585 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:23.546585 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:18:23.546585 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:18:23.546585 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:23.546585 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:18:23.546585 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:23.546585 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:18:23.548129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:23.548129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:18:23.548129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:18:23.548129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:18:23.548129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:23.548129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:18:23.548129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:23.548129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:18:23.548129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:23.548129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:18:23.548129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:23.548129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:18:23.548129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:23.548129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:18:23.548129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:18:23.548676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:23.548676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:18:23.548676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:23.548676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:18:23.548676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:18:23.548676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:23.548676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:18:23.548676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:23.548676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:18:23.548676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:23.548676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:18:23.548676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:18:23.548676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:18:23.548676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:18:23.549675 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:18:23.549675 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:18:23.549675 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:18:23.549675 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:18:23.549675 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:23.549675 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:18:23.549675 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:23.549675 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:18:23.549675 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:23.549675 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:18:23.549675 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:23.549675 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:18:23.549675 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:23.550675 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:18:23.550675 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:18:23.550675 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:23.550675 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:18:23.550675 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:23.550675 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:18:23.550675 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:18:23.550675 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:23.550675 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:18:23.550675 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:23.550675 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:18:23.550675 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:23.551192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:18:23.551192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:18:23.551192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:18:23.551192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:18:23.551192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:18:23.551192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:18:23.551192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:18:23.551192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:18:23.551192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:18:23.551192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:18:23.551192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:18:23.551192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:18:23.551192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:18:23.551192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:18:23.551192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:18:23.551192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:18:23.551192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:18:23.551192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:18:23.551192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:18:23.551705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:23.551705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:18:23.551705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:23.551705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:18:23.551705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:23.551705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:18:23.551705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:23.551705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:18:23.551705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:23.551705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:18:23.551705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:18:23.551705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:23.551705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:18:23.551705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:23.551705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:18:23.551705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:18:23.551705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:23.551705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:18:23.551705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:23.551705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:18:23.555550 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:18:23.555550 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:18:23.555550 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:18:23.555550 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:18:23.555550 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:18:23.555550 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:18:23.555915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:23.556447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:18:23.556447 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:23.556447 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:18:23.556447 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:18:23.556447 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:23.556447 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:18:23.556447 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:23.556447 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:18:23.557284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:23.557284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:18:23.557284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:18:23.557284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:18:23.557284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:18:23.557284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:18:23.557284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:18:23.557284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:23.557284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:18:23.557284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:23.557284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:18:23.557284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:23.557812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:18:23.557812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:23.557812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:18:23.557812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:23.557812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:18:23.557812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:18:23.557812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:23.557812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:18:23.557812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:23.557812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:18:23.557812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:18:23.557812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:23.557812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:18:23.558341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:23.558341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:18:23.558972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:23.558972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:18:23.558972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:18:23.558972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:18:23.558972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:18:23.558972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:18:23.558972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:18:23.558972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:18:23.558972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:18:23.558972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:18:23.558972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:18:23.558972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:18:23.558972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:18:23.558972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:18:23.558972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:18:23.558972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:18:23.559571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:18:23.559571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:18:23.559571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:23.559571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:18:23.559571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:23.559571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:18:23.559571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:23.559571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:18:23.559571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:23.559571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:18:23.559571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:23.559571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:18:23.559571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:18:23.559571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:23.559571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:18:23.559571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:23.559571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:18:23.559571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:18:23.559571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:23.560113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:18:23.560113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:23.560113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:18:27.972060 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:18:27.972567 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:18:27.972567 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:18:27.974675 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:18:28.015632 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:18:28.016279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:28.019776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:18:28.019776 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:28.019776 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:18:28.019776 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:18:28.019776 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:28.019776 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:18:28.019776 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:28.019776 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:18:28.019776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.045 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:28.019776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:18:28.019776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:18:28.019776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:18:28.019776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:18:28.019776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:18:28.019776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:18:28.020255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:28.020255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:18:28.020255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:28.020255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:18:28.020255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:28.020255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:18:28.020255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:28.020255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:18:28.020255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:28.020255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:18:28.020255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:18:28.020255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:28.020255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:18:28.020255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:28.020680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:18:28.020680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:18:28.020680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:28.020680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:18:28.020680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:28.020680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:18:28.020680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:28.020680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:18:28.020680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:18:28.020680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:18:28.020680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:18:28.020680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:18:28.020680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:18:28.020680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:18:28.021067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:18:28.021067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:18:28.021067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:18:28.021067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:18:28.021067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:18:28.021067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:18:28.021067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:18:28.021067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:18:28.021067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:18:28.021067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:18:28.021067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:28.021067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:18:28.021067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:28.021067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:18:28.021067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:28.021067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:18:28.021067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:28.021067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:18:28.021067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:28.021067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:18:28.021067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:18:28.021664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:28.021664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:18:28.021664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:28.021664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:18:28.021664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:18:28.021664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:28.021664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:18:28.021664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:28.021664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:18:28.527069 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:18:28.528011 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:18:28.528142 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:18:28.529123 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:18:28.529371 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:18:28.529539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:28.531842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:18:28.531842 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:28.531842 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:18:28.531842 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:18:28.531842 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:28.531842 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:18:28.531842 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:28.531842 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:18:28.531842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:28.531842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:18:28.531842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:18:28.531842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:18:28.531842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:18:28.531842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:18:28.531842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:18:28.532341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:28.532341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:18:28.532341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:28.532341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:18:28.532341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:28.532341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:18:28.532341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:28.532341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:18:28.532341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:28.532341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:18:28.532341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:18:28.532341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:28.532341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:18:28.532341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:28.532614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:18:28.532614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:18:28.532614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:28.532614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:18:28.532614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:28.532614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:18:28.532614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:28.532614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:18:28.532614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:18:28.532614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:18:28.532614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:18:28.532614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:18:28.532614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:18:28.532614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:18:28.532899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:18:28.532899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:18:28.532899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:18:28.532899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:18:28.532899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:18:28.532899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:18:28.532899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:18:28.532899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:18:28.532899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:18:28.532899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:18:28.532899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:28.532899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:18:28.532899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:28.532899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:18:28.532899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:28.532899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:18:28.532899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:28.532899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:18:28.532899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:28.532899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:18:28.533285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:18:28.533285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:28.533285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:18:28.533285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:28.533285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:18:28.533285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:18:28.533285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:28.533285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:18:28.533285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:28.533285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:18:28.536964 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:18:28.537364 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:18:28.537493 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:18:28.537623 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:18:28.538126 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:18:28.538349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:28.538988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:18:28.538988 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:28.538988 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:18:28.538988 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:18:28.538988 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:28.538988 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:18:28.538988 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:28.538988 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:18:28.539820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:28.539820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:18:28.539820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:18:28.539820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:18:28.539820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:28.539820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:18:28.539820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:28.539820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:18:28.539820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:28.539820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:18:28.539820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:28.539820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:18:28.539820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:28.539820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:18:28.539820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:18:28.540251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:28.540251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:18:28.540251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:28.540251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:18:28.540251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:18:28.540251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:28.540251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:18:28.540251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:28.540251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:18:28.540602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:28.540602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:18:28.540602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:18:28.540602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:18:28.540602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:18:28.540602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:18:28.540602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:18:28.540602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:18:28.540602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:18:28.540602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:18:28.540602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:28.540602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:18:28.540945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:28.540945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:18:28.540945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:28.540945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:18:28.540945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:28.540945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:18:28.540945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:28.540945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:18:28.540945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:18:28.540945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:28.540945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:18:28.540945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:28.540945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:18:28.540945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:18:28.541357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:28.541357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:18:28.541357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:28.541357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:18:28.541737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:28.541737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:18:28.541737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:18:28.541737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:18:28.541737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:18:28.541737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:18:28.541737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:18:28.541737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:18:28.541737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:18:28.541737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:18:28.541737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:18:28.541737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:18:28.541737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:18:28.541737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:18:28.541737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:18:28.541737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:18:28.542253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:18:28.542253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:18:28.542253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:18:28.542253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:18:28.542253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:18:28.542253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:28.542253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:18:28.542253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:28.542253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:18:28.542253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:28.542253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:18:28.542253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:28.542253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:18:28.542253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:28.542253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:18:28.542253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:18:28.542253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:28.542253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:18:28.542799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:28.542799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:18:28.542799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:18:28.542799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:28.542799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:18:28.542799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:28.542799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:18:28.545158 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:18:28.545577 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:18:28.545753 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:18:28.545961 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:18:28.546289 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:18:28.546476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:28.547014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:18:28.547014 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:28.547014 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:18:28.547014 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:18:28.547014 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:28.547014 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:18:28.547014 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:28.547014 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:18:28.548109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:28.548109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:18:28.548109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:18:28.548109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:18:28.548109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:28.548109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:18:28.548109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:28.548109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:18:28.548109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:28.548109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:18:28.548109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:28.548109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:18:28.548109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:28.548109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:18:28.548109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:18:28.549070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:28.549070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:18:28.549070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:28.549070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:18:28.549070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:18:28.549070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:28.549070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:18:28.549070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:28.549070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:18:28.551532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:28.551532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:18:28.551532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:18:28.551532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:18:28.551532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:18:28.551532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:18:28.551532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:18:28.551532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:18:28.551532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:18:28.551532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:28.551532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:18:28.552164 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:28.552164 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:18:28.552164 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:28.552164 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:18:28.552164 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:28.552164 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:18:28.552164 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:28.552164 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:18:28.552164 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:18:28.552164 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:28.552164 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:18:28.552164 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:28.552164 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:18:28.552164 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:18:28.552537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:28.552537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:18:28.552537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:28.552537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:18:28.554430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:28.554430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:18:28.554430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:18:28.554430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:18:28.554430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:18:28.554430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:18:28.554430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:18:28.554430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:18:28.554430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:18:28.554430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:18:28.554430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:18:28.554430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:18:28.554430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:18:28.554430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:18:28.554430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:18:28.554430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:18:28.554430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:18:28.554997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:18:28.554997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:18:28.554997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:18:28.554997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:28.554997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:18:28.554997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:28.554997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:18:28.554997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:28.554997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:18:28.554997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:28.554997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:18:28.554997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:28.554997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:18:28.554997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:18:28.554997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:28.554997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:18:28.554997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:28.554997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:18:28.554997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:18:28.555552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:28.555552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:18:28.555552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:28.555552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:18:28.555552 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:18:28.555552 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:18:28.555552 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:18:28.555552 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:18:28.555930 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:18:28.556162 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:18:28.556318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:28.556984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:18:28.556984 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:28.556984 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:18:28.556984 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:18:28.556984 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:28.556984 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:18:28.556984 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:28.556984 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:18:28.558085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:28.558085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:18:28.558085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:18:28.558085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:18:28.558085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:18:28.558085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:18:28.558085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:18:28.558085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:28.558085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:18:28.558085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:28.558085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:18:28.558085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:28.558557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:18:28.558557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:28.558557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:18:28.558557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:28.558557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:18:28.558557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:18:28.558557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:28.558557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:18:28.558557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:28.558557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:18:28.558557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:18:28.558557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:28.558557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:18:28.559309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:28.559309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:18:28.559309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:28.559309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:18:28.559309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:18:28.559309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:18:28.559309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:18:28.559309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:18:28.559309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:18:28.559309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:18:28.559309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:18:28.559309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:18:28.559309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:18:28.559309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:18:28.559309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:18:28.560335 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:18:28.560335 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:18:28.560335 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:18:28.560335 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:18:28.560335 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:18:28.560335 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:28.560335 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:18:28.560335 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:28.560335 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:18:28.560335 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:28.560335 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:18:28.560335 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:28.560335 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:18:28.560335 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:28.560335 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:18:28.560335 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:18:28.560335 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:28.560335 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:18:28.560335 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:28.560998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:18:28.560998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:18:28.560998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:28.560998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:18:28.560998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:28.560998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:18:32.972849 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:18:32.973529 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:18:32.973529 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:18:32.974388 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:18:33.015979 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:18:33.016245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:33.017033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:18:33.017033 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:33.017033 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:18:33.017033 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:18:33.017033 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:33.017033 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:18:33.017033 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:33.017033 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:18:33.017811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.044 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:33.017811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:18:33.017811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:18:33.017811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:18:33.017811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:18:33.017811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:18:33.017811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:18:33.017811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:33.017811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:18:33.017811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:33.017811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:18:33.017811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:33.018184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:18:33.018184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:33.018184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:18:33.018184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:33.018184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:18:33.018184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:18:33.018184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:33.018184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:18:33.018184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:33.018184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:18:33.018184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:18:33.018184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:33.018184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:18:33.018542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:33.018542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:18:33.018607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:33.018607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:18:33.018607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:18:33.018607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:18:33.018607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:18:33.018607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:18:33.018607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:18:33.018607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:18:33.018607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:18:33.018607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:18:33.018607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:18:33.018607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:18:33.018607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:18:33.018607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:18:33.018607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:18:33.018607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:18:33.018990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:18:33.018990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:18:33.018990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:33.018990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:18:33.018990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:33.018990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:18:33.018990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:33.018990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:18:33.018990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:33.018990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:18:33.018990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:33.018990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:18:33.018990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:18:33.018990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:33.018990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:18:33.018990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:33.018990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:18:33.018990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:18:33.018990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:33.019492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:18:33.019492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:33.019492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:18:33.528174 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:18:33.528758 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:18:33.529040 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:18:33.530318 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:18:33.530708 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:18:33.531070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:33.532897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:18:33.532897 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:33.532897 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:18:33.532897 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:18:33.532897 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:33.532897 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:18:33.532897 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:33.532897 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:18:33.533387 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:33.533387 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:18:33.533387 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:18:33.533387 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:18:33.533387 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:18:33.533387 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:18:33.533387 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:18:33.533387 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:33.533387 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:18:33.533387 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:33.533387 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:18:33.533387 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:33.533889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:18:33.533889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:33.533889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:18:33.533889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:33.533889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:18:33.533889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:18:33.533889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:33.533889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:18:33.533889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:33.533889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:18:33.533889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:18:33.533889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:33.533889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:18:33.534249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:33.534249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:18:33.534315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:33.534315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:18:33.534315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:18:33.534315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:18:33.534315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:18:33.534315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:18:33.534315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:18:33.534315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:18:33.534315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:18:33.534315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:18:33.534315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:18:33.534315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:18:33.534315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:18:33.534315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:18:33.534315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:18:33.534315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:18:33.534866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:18:33.534866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:18:33.534866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:33.534866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:18:33.534866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:33.534866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:18:33.534866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:33.534866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:18:33.534866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:33.534866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:18:33.534866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:33.534866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:18:33.534866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:18:33.534866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:33.534866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:18:33.534866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:33.534866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:18:33.534866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:18:33.534866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:33.535369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:18:33.535369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:33.535369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:18:33.537380 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:18:33.537694 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:18:33.537844 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:18:33.537983 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:18:33.539041 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:18:33.539041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:33.539259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:18:33.539259 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:33.539259 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:18:33.539259 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:18:33.539259 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:33.539259 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:18:33.539259 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:33.539259 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:18:33.540193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:33.540193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:18:33.540193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:18:33.540193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:18:33.540193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:33.540193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:18:33.540193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:33.540193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:18:33.540193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:33.540193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:18:33.540193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:33.540193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:18:33.540193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:33.540193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:18:33.540193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:18:33.540604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:33.540604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:18:33.540604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:33.540604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:18:33.540604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:18:33.540604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:33.540604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:18:33.540604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:33.540604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:18:33.541271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:33.541271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:18:33.541271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:18:33.541271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:18:33.541271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:18:33.541271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:18:33.541271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:18:33.541271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:18:33.541271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:18:33.541271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:18:33.541271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:33.541271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:18:33.541702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:33.541702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:18:33.541702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:33.541702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:18:33.541702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:33.541702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:18:33.541702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:33.541702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:18:33.541702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:18:33.541702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:33.541702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:18:33.541702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:33.541702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:18:33.541702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:18:33.542177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:33.542177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:18:33.542177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:33.542177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:18:33.542599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:33.542599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:18:33.542599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:18:33.542599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:18:33.542599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:18:33.542599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:18:33.542599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:18:33.542599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:18:33.542599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:18:33.542599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:18:33.542599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:18:33.542599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:18:33.542599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:18:33.542599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:18:33.542599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:18:33.542599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:18:33.543134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:18:33.543134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:18:33.543134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:18:33.543134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:18:33.543134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:18:33.543134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:33.543134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:18:33.543134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:33.543134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:18:33.543134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:33.543134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:18:33.543134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:33.543134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:18:33.543134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:33.543134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:18:33.543134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:18:33.543134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:33.543134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:18:33.543763 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:33.543763 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:18:33.543763 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:18:33.543763 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:33.543763 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:18:33.543763 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:33.543763 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:18:33.545915 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:18:33.545915 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:18:33.547505 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:18:33.547505 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:18:33.547505 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:18:33.547505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:33.548203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:18:33.548203 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:33.548203 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:18:33.548203 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:18:33.548203 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:33.548203 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:18:33.548203 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:33.548203 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:18:33.548764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:33.548764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:18:33.548764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:18:33.548764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:18:33.548764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:33.548764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:18:33.548764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:33.548764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:18:33.548764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:33.548764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:18:33.548764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:33.548764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:18:33.548764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:33.548764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:18:33.548764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:18:33.549317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:33.549317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:18:33.549317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:33.549317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:18:33.549317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:18:33.549317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:33.549317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:18:33.549317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:33.549317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:18:33.551681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:33.551681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:18:33.551681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:18:33.551681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:18:33.551681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:18:33.551681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:18:33.551681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:18:33.551681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:18:33.551681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:18:33.551681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:33.551681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:18:33.552030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:33.552030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:18:33.552030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:33.552030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:18:33.552030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:33.552030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:18:33.552030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:33.552030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:18:33.552030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:18:33.552030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:33.552030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:18:33.552030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:33.552030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:18:33.552030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:18:33.552310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:33.552310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:18:33.552310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:33.552310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:18:33.552618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:33.552618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:18:33.552618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:18:33.552618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:18:33.552618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:18:33.552618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:18:33.552618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:18:33.552618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:18:33.552618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:18:33.552618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:18:33.552618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:18:33.552618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:18:33.552618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:18:33.552618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:18:33.552618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:18:33.552618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:18:33.552618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:18:33.553118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:18:33.553118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:18:33.553118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:18:33.553118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:33.553118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:18:33.553118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:33.553118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:18:33.553118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:33.553118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:18:33.553118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:33.553118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:18:33.553118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:33.553118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:18:33.553118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:18:33.553118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:33.553118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:18:33.553118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:33.553118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:18:33.553118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:18:33.553473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:33.553473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:18:33.553473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:33.553473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:18:33.555368 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:18:33.555733 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:18:33.556020 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:18:33.556261 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:18:33.556599 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:18:33.557567 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:18:33.557567 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:33.557710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:18:33.557710 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:33.557710 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:18:33.557710 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:18:33.557710 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:33.557710 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:18:33.557710 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:33.557710 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:18:33.559433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:33.559433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:18:33.559433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:18:33.559433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:18:33.559433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:18:33.559433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:18:33.559433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:18:33.559433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:33.559433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:18:33.559433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:33.559433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:18:33.559433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:33.559830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:18:33.559830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:33.559830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:18:33.559830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:33.559830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:18:33.559830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:18:33.559830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:33.559830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:18:33.559830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:33.559830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:18:33.559830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:18:33.559830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:33.559830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:18:33.560179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:33.560179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:18:33.560179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:33.560179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:18:33.560179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:18:33.560179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:18:33.560179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:18:33.560179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:18:33.560179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:18:33.560179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:18:33.560179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:18:33.560179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:18:33.560179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:18:33.560179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:18:33.560179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:18:33.560504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:18:33.560504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:18:33.560504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:18:33.560504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:18:33.560504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:18:33.560504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:33.560504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:18:33.560504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:33.560504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:18:33.560504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:33.560504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:18:33.560504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:33.560504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:18:33.560504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:33.560504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:18:33.560504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:18:33.560504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:33.560504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:18:33.560504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:33.560895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:18:33.560895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:18:33.560895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:33.560895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:18:33.560895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:33.560895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:18:37.975288 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:18:37.975288 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:18:37.975288 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:18:37.975288 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:18:38.016709 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:18:38.016709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:38.016709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:18:38.016709 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:38.016709 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:18:38.016709 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:18:38.016709 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:38.016709 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:18:38.016709 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:38.016709 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:18:38.016709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:38.016709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:18:38.016709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:18:38.018715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:18:38.018715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:18:38.018715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:18:38.018715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:18:38.018715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:38.018715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:18:38.018715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:38.018715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:18:38.018715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:38.018715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:18:38.018715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:38.018715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:18:38.018715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:38.018715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:18:38.020230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:18:38.020230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:38.020230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:18:38.020230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:38.020230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:18:38.020230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:18:38.020230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:38.020230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:18:38.020230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:38.020230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:18:38.020230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:38.020230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:18:38.021255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:18:38.021255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:18:38.021255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:18:38.021255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:18:38.021255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:18:38.021255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:18:38.021255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:18:38.021255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:18:38.021255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:18:38.021255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:18:38.021255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:18:38.021255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:18:38.021255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:18:38.021255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:18:38.021255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:18:38.021255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:18:38.021255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:38.021255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:18:38.021255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:38.022688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:18:38.022688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:38.022688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:18:38.022688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:38.022688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:18:38.022688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:38.022688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:18:38.022688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:18:38.022688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:38.022688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:18:38.022688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:38.022688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:18:38.022688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:18:38.022688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:38.022688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:18:38.022688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:38.022688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:18:38.528738 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:18:38.529312 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:18:38.529431 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:18:38.530092 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:18:38.530256 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:18:38.530397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:38.534787 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:18:38.534787 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:38.534787 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:18:38.534787 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:18:38.534787 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:38.534787 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:18:38.534787 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:38.534787 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:18:38.534787 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:38.534787 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:18:38.534787 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:18:38.534787 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:18:38.534787 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:18:38.534787 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:18:38.534787 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:18:38.535394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:38.535394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:18:38.535394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:38.535394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:18:38.535394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:38.535394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:18:38.535394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:38.535394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:18:38.535394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:38.535394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:18:38.535394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:18:38.535394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:38.535394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:18:38.535394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:38.535794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:18:38.535794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:18:38.535794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:38.535794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:18:38.535794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:38.535794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:18:38.535794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:38.535794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:18:38.535794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:18:38.535794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:18:38.535794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:18:38.535794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:18:38.535794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:18:38.535794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:18:38.536374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:18:38.536374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:18:38.536374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:18:38.536374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:18:38.536374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:18:38.536374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:18:38.536374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:18:38.536374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:18:38.536374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:18:38.536374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:18:38.536374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:38.536374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:18:38.536374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:38.536374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:18:38.536374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:38.536374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:18:38.536374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:38.536374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:18:38.536374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:38.536374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:18:38.536850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:18:38.536850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:38.536850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:18:38.536850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:38.536850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:18:38.536850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:18:38.536850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:38.536850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:18:38.536850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:38.536850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:18:38.537825 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:18:38.538123 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:18:38.538233 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:18:38.538354 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:18:38.540387 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:18:38.540598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:38.541121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:18:38.541121 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:38.541121 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:18:38.541121 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:18:38.541121 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:38.541121 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:18:38.541121 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:38.541121 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:18:38.541862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:38.541862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:18:38.541862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:18:38.541862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:18:38.541862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:38.541862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:18:38.541862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:38.541862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:18:38.541862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:38.541862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:18:38.541862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:38.541862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:18:38.541862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:38.541862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:18:38.541862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:18:38.542252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:38.542252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:18:38.542252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:38.542252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:18:38.542252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:18:38.542252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:38.542252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:18:38.542252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:38.542252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:18:38.542721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:38.542721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:18:38.542721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:18:38.542721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:18:38.542721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:18:38.542721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:18:38.542721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:18:38.542721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:18:38.542721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:18:38.542721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:18:38.542721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:38.542721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:18:38.543074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:38.543074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:18:38.543074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:38.543074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:18:38.543074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:38.543074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:18:38.543074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:38.543074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:18:38.543074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:18:38.543074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:38.543074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:18:38.543074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:38.543074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:18:38.543074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:18:38.543888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:38.543888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:18:38.543888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:38.543888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:18:38.544321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:38.544321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:18:38.544321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:18:38.544321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:18:38.544321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:18:38.544321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:18:38.544321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:18:38.544321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:18:38.544321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:18:38.544321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:18:38.544321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:18:38.544321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:18:38.544321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:18:38.544321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:18:38.544321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:18:38.544321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:18:38.544681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:18:38.544681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:18:38.544681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:18:38.544681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:18:38.544681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:18:38.544681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:38.544681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:18:38.544681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:38.544681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:18:38.544681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:38.544681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:18:38.544681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:38.544681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:18:38.544681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:38.544681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:18:38.544681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:18:38.544681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:38.544681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:18:38.545093 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:38.545093 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:18:38.545093 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:18:38.545093 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:38.545093 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:18:38.545093 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:38.545093 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:18:38.545991 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:18:38.546261 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:18:38.546373 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:18:38.546503 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:18:38.546819 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:18:38.547054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:38.547494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:18:38.547494 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:38.547494 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:18:38.547494 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:18:38.547494 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:38.547494 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:18:38.547494 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:38.547494 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:18:38.548284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:38.548284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:18:38.548284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:18:38.548284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:18:38.548284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:38.548284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:18:38.548284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:38.548284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:18:38.548284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:38.548284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:18:38.548284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:38.548284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:18:38.548284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:38.548284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:18:38.548284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:18:38.548578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:38.548578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:18:38.548578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:38.548578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:18:38.548578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:18:38.548578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:38.548578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:18:38.548578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:38.548578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:18:38.549465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:38.549465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:18:38.549465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:18:38.549465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:18:38.549465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:18:38.549465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:18:38.549465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:18:38.549465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:18:38.549465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:18:38.549465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:38.549465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:18:38.549720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:38.549720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:18:38.549720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:38.549720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:18:38.549720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:38.549720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:18:38.549720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:38.549720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:18:38.549720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:18:38.549720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:38.549720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:18:38.549720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:38.549720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:18:38.549720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:18:38.550032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:38.550032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:18:38.550032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:38.550032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:18:38.550547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:38.550547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:18:38.550547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:18:38.550547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:18:38.550547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:18:38.550547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:18:38.550547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:18:38.550547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:18:38.550547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:18:38.550547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:18:38.550547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:18:38.550547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:18:38.550547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:18:38.550547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:18:38.550547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:18:38.550547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:18:38.550547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:18:38.550994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:18:38.550994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:18:38.550994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:18:38.550994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:38.550994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:18:38.550994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:38.550994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:18:38.550994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:38.550994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:18:38.550994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:38.550994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:18:38.550994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:38.550994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:18:38.550994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:18:38.550994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:38.550994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:18:38.550994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:38.550994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:18:38.550994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:18:38.551390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:38.551390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:18:38.551390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:38.551390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:18:38.555845 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:18:38.556206 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:18:38.556321 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:18:38.556520 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:18:38.556869 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:18:38.557102 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:18:38.557247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:38.557928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:18:38.557928 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:38.557928 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:18:38.557928 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:18:38.557928 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:38.557928 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:18:38.557928 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:38.557928 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:18:38.558803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:38.558803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:18:38.558803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:18:38.558803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:18:38.558803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:18:38.558803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:18:38.558803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:18:38.558803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:38.558803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:18:38.558803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:38.558803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:18:38.558803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:38.559069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:18:38.559069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:38.559069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:18:38.559069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:38.559069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:18:38.559069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:18:38.559069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:38.559069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:18:38.559069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:38.559069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:18:38.559069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:18:38.559069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:38.559069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:18:38.559331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:38.559331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:18:38.559822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:38.559822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:18:38.559822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:18:38.559822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:18:38.559822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:18:38.559822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:18:38.559822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:18:38.559822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:18:38.559822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:18:38.559822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:18:38.559822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:18:38.559822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:18:38.559822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:18:38.559822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:18:38.559822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:18:38.559822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:18:38.560158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:18:38.560158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:18:38.560158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:38.560158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:18:38.560158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:38.560158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:18:38.560158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:38.560158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:18:38.560158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:38.560158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:18:38.560158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:38.560158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:18:38.560158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:18:38.560158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:38.560158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:18:38.560158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:38.560158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:18:38.560158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:18:38.560158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:38.560516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:18:38.560516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:38.560516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:18:42.974074 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:18:42.974904 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:18:42.975150 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:18:42.976592 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:18:43.016918 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:18:43.017001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:43.017451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:18:43.017451 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:43.017451 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:18:43.017451 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:18:43.017451 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:43.017451 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:18:43.017451 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:43.017451 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:18:43.018202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:43.018202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:18:43.018202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:18:43.018202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:18:43.018202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:18:43.018202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:18:43.018202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:18:43.018202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:43.018202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:18:43.018202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:43.018202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:18:43.018202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:43.041144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:18:43.041144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:43.041144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:18:43.041144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:43.041144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:18:43.041144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:18:43.041144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:43.041144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:18:43.041144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:43.041144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:18:43.041144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:18:43.041144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:43.041144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:18:43.042068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:43.042068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:18:43.042068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:43.042068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:18:43.042068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:18:43.042068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:18:43.042068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:18:43.042068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:18:43.042068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:18:43.042068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:18:43.042068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:18:43.042068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:18:43.042068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:18:43.042068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:18:43.042068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:18:43.042781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:18:43.042781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:18:43.042781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:18:43.042781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:18:43.042781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:18:43.042781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:43.042781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:18:43.042781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:43.042781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:18:43.042781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:43.042781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:18:43.042781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:43.042781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:18:43.042781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:43.042781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:18:43.042781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:18:43.042781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:43.042781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:18:43.042781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:43.043642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:18:43.043642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:18:43.043642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:43.043642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:18:43.043642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:43.043642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:18:43.529500 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:18:43.530167 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:18:43.530398 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:18:43.531607 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:18:43.531917 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:18:43.532284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:43.533292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:18:43.533292 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:43.533292 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:18:43.533292 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:18:43.533292 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:43.533292 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:18:43.533292 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:43.533292 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:18:43.534838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:43.534838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:18:43.534838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:18:43.534838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:18:43.534838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:18:43.534838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:18:43.534838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:18:43.534838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:43.534838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:18:43.534838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:43.534838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:18:43.534838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:43.535482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:18:43.535482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:43.535482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:18:43.535482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:43.535482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:18:43.535482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:18:43.535482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:43.535482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:18:43.535482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:43.535482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:18:43.535482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:18:43.535482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:43.535482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:18:43.536073 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:43.536073 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:18:43.536182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:43.536182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:18:43.536182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:18:43.536182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:18:43.536182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:18:43.536182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:18:43.536182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:18:43.536182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:18:43.536182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:18:43.536182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:18:43.536182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:18:43.536182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:18:43.536182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:18:43.536182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:18:43.536182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:18:43.536182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:18:43.536936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:18:43.536936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:18:43.536936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:43.536936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:18:43.536936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:43.536936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:18:43.536936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:43.536936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:18:43.536936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:43.536936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:18:43.536936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:43.536936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:18:43.536936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:18:43.536936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:43.536936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:18:43.536936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:43.536936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:18:43.536936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:18:43.536936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:43.538156 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:18:43.538156 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:43.538156 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:18:43.538394 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:18:43.538776 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:18:43.539004 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:18:43.539111 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:18:43.539498 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:18:43.539729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:43.540257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:18:43.540257 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:43.540257 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:18:43.540257 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:18:43.540257 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:43.540257 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:18:43.540257 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:43.540257 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:18:43.541118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:43.541118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:18:43.541118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:18:43.541118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:18:43.541118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:43.541118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:18:43.541118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:43.541118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:18:43.541118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:43.541118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:18:43.541118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:43.541118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:18:43.541118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:43.541118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:18:43.541118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:18:43.571045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:43.571045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:18:43.571045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:43.571045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:18:43.571045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:18:43.571045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:43.571045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:18:43.571045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:43.571045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:18:43.571045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:43.571045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:18:43.571045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:18:43.571045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:18:43.571045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:18:43.571841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:18:43.571841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:18:43.571841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:18:43.571841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:18:43.571841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:18:43.571841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:43.571841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:18:43.571841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:43.571841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:18:43.571841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:43.571841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:18:43.571841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:43.571841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:18:43.572520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:43.572520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:18:43.572520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:18:43.572520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:43.572520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:18:43.572520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:43.572520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:18:43.572520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:18:43.572520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:43.572520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:18:43.572520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:43.572520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:18:43.573314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:43.573314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:18:43.573314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:18:43.573314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:18:43.573314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:18:43.573314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:18:43.573314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:18:43.573314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:18:43.573314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:18:43.573314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:18:43.573314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:18:43.573314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:18:43.573314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:18:43.573314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:18:43.573314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:18:43.573314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:18:43.574207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:18:43.574207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:18:43.574207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:18:43.574207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:18:43.574207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:18:43.574207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:43.574207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:18:43.574207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:43.574207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:18:43.574207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:43.574207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:18:43.574207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:43.574207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:18:43.574207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:43.574207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:18:43.574207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:18:43.574207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:43.574207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:18:43.575039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:43.575039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:18:43.575039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:18:43.575039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:43.575039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:18:43.575039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:43.575039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:18:43.575039 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:18:43.575039 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:18:43.575039 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:18:43.575389 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:18:43.575389 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:18:43.575389 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:43.575389 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:18:43.575389 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:43.575389 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:18:43.575389 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:18:43.575389 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:43.575389 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:18:43.575389 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:43.575389 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:18:43.575389 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:43.575814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:18:43.575814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:18:43.575814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:18:43.575814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:43.575814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:18:43.575814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:43.575814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:18:43.575814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:43.575814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:18:43.575814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:43.575814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:18:43.575814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:43.575814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:18:43.575814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:18:43.575814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:43.575814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:18:43.575814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:43.576369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:18:43.576369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:18:43.576369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:43.576369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:18:43.576369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:43.576369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:18:43.576369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:43.576369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:18:43.576369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:18:43.576369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:18:43.576369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:18:43.576369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:18:43.576369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:18:43.577059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:18:43.577059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:18:43.577059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:43.577059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:18:43.577059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:43.577059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:18:43.577059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:43.577059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:18:43.577059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:43.577059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:18:43.577059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:43.577059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:18:43.577059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:18:43.577509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:43.577509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:18:43.577509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:43.577509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:18:43.577509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:18:43.577509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:43.577509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:18:43.577509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:43.577509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:18:43.577509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:43.577509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:18:43.577509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:18:43.577857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:18:43.577857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:18:43.577857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:18:43.577857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:18:43.577857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:18:43.577857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:18:43.577857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:18:43.577857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:18:43.577857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:18:43.577857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:18:43.577857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:18:43.577857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:18:43.577857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:18:43.577857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:18:43.577857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:18:43.577857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:18:43.577857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:18:43.577857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:43.577857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:18:43.578362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:43.578362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:18:43.578362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:43.578362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:18:43.578362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:43.578362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:18:43.578362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:43.578362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:18:43.578362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:18:43.578362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:43.578362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:18:43.578362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:43.578362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:18:43.578362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:18:43.578362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:43.578362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:18:43.578362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:43.578362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:18:43.578953 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:18:43.578953 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:18:43.578953 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:18:43.578953 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:18:43.578953 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:18:43.578953 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:18:43.579141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:43.579141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:18:43.579141 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:43.579141 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:18:43.579141 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:18:43.579141 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:43.579141 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:18:43.579141 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:43.579141 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:18:43.579141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:43.579141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:18:43.579141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:18:43.579141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:18:43.579141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:18:43.579141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:18:43.579554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:18:43.579554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:43.579554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:18:43.579554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:43.579554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:18:43.579554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:43.579554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:18:43.579554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:43.579554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:18:43.579554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:43.579554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:18:43.579554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:18:43.579554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:43.579554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:18:43.580060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:43.580060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:18:43.580060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:18:43.580060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:43.580060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:18:43.580060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:43.580060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:18:43.580060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:43.580060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:18:43.580060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:18:43.580060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:18:43.580060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:18:43.580060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:18:43.580475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:18:43.580475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:18:43.580475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:18:43.580475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:18:43.580475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:18:43.580475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:18:43.580475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:18:43.580475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:18:43.580475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:18:43.580475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:18:43.580475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:18:43.580475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:18:43.580475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:43.580475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:18:43.580475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:43.580475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:18:43.580475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:43.580475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:18:43.580475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:43.580475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:18:43.581218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:43.581218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:18:43.581218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:18:43.581218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:43.581218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:18:43.581218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:43.581218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:18:43.581218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:18:43.581218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:43.581218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:18:43.581218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:43.581218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:18:47.974730 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:18:47.975381 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:18:47.975381 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:18:47.976095 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:18:48.018125 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:18:48.018581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:48.019591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:18:48.019591 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:48.019591 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:18:48.019591 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:18:48.019591 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:48.019591 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:18:48.019591 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:48.019591 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:18:48.020751 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.044 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:48.020751 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:18:48.020751 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:18:48.020751 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:18:48.020751 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:18:48.020751 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:18:48.020751 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:18:48.020751 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:48.020751 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:18:48.020751 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:48.020751 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:18:48.020751 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:48.021223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:18:48.021223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:48.021223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:18:48.021223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:48.021223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:18:48.021223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:18:48.021223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:48.021223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:18:48.021223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:48.021223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:18:48.021223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:18:48.021223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:48.021223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:18:48.021648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:48.021648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:18:48.022310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:48.022310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:18:48.022310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:18:48.022310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:18:48.022310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:18:48.022310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:18:48.022310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:18:48.022310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:18:48.022310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:18:48.022310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:18:48.022310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:18:48.022310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:18:48.022310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:18:48.022310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:18:48.022310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:18:48.022310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:18:48.022825 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:18:48.022825 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:18:48.022825 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:48.022825 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:18:48.022825 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:48.022825 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:18:48.022825 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:48.022825 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:18:48.022825 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:48.022825 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:18:48.022825 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:48.022825 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:18:48.022825 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:18:48.022825 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:48.022825 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:18:48.022825 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:48.022825 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:18:48.022825 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:18:48.022825 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:48.023411 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:18:48.023411 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:48.023411 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:18:48.530133 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:18:48.530726 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:18:48.530886 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:18:48.532025 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:18:48.532600 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:18:48.532600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:48.534093 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:18:48.534093 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:48.534093 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:18:48.534093 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:18:48.534093 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:48.534093 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:18:48.534093 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:48.534093 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:18:48.535786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:48.535786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:18:48.535786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:18:48.535786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:18:48.535786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:18:48.535786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:18:48.535786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:18:48.535786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:48.535786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:18:48.535786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:48.535786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:18:48.535786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:48.536083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:18:48.536083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:48.536083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:18:48.536083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:48.536083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:18:48.536083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:18:48.536083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:48.536083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:18:48.536083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:48.536083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:18:48.536083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:18:48.536083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:48.536083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:18:48.536329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:48.536329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:18:48.537539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:48.537539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:18:48.537539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:18:48.537539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:18:48.537539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:18:48.537539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:18:48.537539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:18:48.537539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:18:48.537539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:18:48.537539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:18:48.537539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:18:48.537539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:18:48.537539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:18:48.537539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:18:48.537539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:18:48.537539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:18:48.537902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:18:48.537902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:18:48.537902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:48.537902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:18:48.537902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:48.537902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:18:48.537902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:48.537902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:18:48.537902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:48.537902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:18:48.537902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:48.537902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:18:48.537902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:18:48.537902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:48.537902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:18:48.537902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:48.537902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:18:48.537902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:18:48.537902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:48.538260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:18:48.538260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:48.538260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:18:48.539195 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:18:48.539366 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:18:48.539475 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:18:48.539612 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:18:48.540359 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:18:48.543857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:48.543857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:18:48.543857 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:48.543857 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:18:48.543857 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:18:48.543857 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:48.543857 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:18:48.543857 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:48.543857 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:18:48.543857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:48.543857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:18:48.543857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:18:48.543857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:18:48.543857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:48.543857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:18:48.543857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:48.543857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:18:48.544230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:48.544230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:18:48.544230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:48.544230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:18:48.544230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:48.544230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:18:48.544230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:18:48.544230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:48.544230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:18:48.544230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:48.544230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:18:48.544230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:18:48.544230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:48.544230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:18:48.544230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:48.544230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:18:48.544554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:48.544554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:18:48.544554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:18:48.544554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:18:48.544554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:18:48.544554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:18:48.544554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:18:48.544554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:18:48.544554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:18:48.544554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:18:48.544554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:48.544554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:18:48.544811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:48.544811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:18:48.544811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:48.544811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:18:48.544811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:48.544811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:18:48.544811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:48.544811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:18:48.544811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:18:48.544811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:48.544811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:18:48.544811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:48.544811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:18:48.544811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:18:48.545074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:48.545074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:18:48.545074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:48.545074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:18:48.545074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:48.545074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:18:48.545074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:18:48.545074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:18:48.545074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:18:48.545074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:18:48.545074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:18:48.545074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:18:48.545074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:18:48.545074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:18:48.545339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:18:48.545339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:18:48.545339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:18:48.545339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:18:48.545339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:18:48.545339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:18:48.545339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:18:48.545339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:18:48.545339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:18:48.545339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:18:48.545339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:18:48.545339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:48.545339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:18:48.545339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:48.545339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:18:48.545339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:48.545339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:18:48.545339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:48.545339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:18:48.545687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:48.545687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:18:48.545687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:18:48.545687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:48.545687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:18:48.545687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:48.545687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:18:48.545687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:18:48.545687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:48.545687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:18:48.545687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:48.545687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:18:48.546855 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:18:48.547286 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:18:48.547393 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:18:48.547527 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:18:48.548207 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:18:48.548567 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:48.549619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:18:48.549619 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:48.549619 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:18:48.549619 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:18:48.549619 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:48.549619 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:18:48.549619 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:48.549619 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:18:48.550437 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:48.550437 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:18:48.550437 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:18:48.550437 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:18:48.550437 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:48.550437 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:18:48.550437 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:48.550437 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:18:48.550437 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:48.550437 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:18:48.550437 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:48.550437 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:18:48.550437 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:48.550437 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:18:48.550437 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:18:48.555635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:48.555635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:18:48.555635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:48.555635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:18:48.555635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:18:48.555635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:48.555635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:18:48.555635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:48.555635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:18:48.555635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:48.555635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:18:48.555635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:18:48.555635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:18:48.555635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:18:48.556268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:18:48.556268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:18:48.556268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:18:48.556268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:18:48.556268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:48.556268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:18:48.556268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:48.556268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:18:48.556268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:48.556268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:18:48.556268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:48.556268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:18:48.556268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:48.556536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:18:48.556536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:18:48.556536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:48.556536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:18:48.556536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:48.556536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:18:48.556536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:18:48.556536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:48.556536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:18:48.556536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:48.556536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:18:48.556536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:48.556824 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:18:48.556824 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:18:48.556824 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:18:48.556824 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:18:48.556824 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:18:48.556824 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:18:48.556824 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:18:48.556824 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:18:48.556824 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:18:48.556824 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:18:48.556824 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:18:48.556824 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:18:48.556824 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:18:48.556824 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:18:48.556824 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:18:48.556824 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:18:48.556824 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:18:48.556824 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:18:48.556824 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:18:48.557201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:48.557201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:18:48.557201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:48.557201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:18:48.557201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:48.557201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:18:48.557201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:48.557201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:18:48.557201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:48.557201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:18:48.557201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:18:48.557201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:48.557201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:18:48.557201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:48.557201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:18:48.557201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:18:48.557201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:48.557201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:18:48.557201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:48.557201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:18:48.557862 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:18:48.557862 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:18:48.557862 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:18:48.557862 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:18:48.558624 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:18:48.558905 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:18:48.559103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:48.559689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:18:48.559689 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:48.559689 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:18:48.559689 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:18:48.559689 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:48.559689 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:18:48.559689 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:48.559689 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:18:48.560638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:48.560638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:18:48.560638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:18:48.560638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:18:48.560638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:18:48.560638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:18:48.560638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:18:48.560638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:48.560638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:18:48.560638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:48.560638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:18:48.560638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:48.560968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:18:48.560968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:48.560968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:18:48.560968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:48.560968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:18:48.560968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:18:48.560968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:48.560968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:18:48.560968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:48.560968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:18:48.560968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:18:48.560968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:48.560968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:18:48.561324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:48.561324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:18:48.561631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:48.561631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:18:48.561631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:18:48.561631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:18:48.561631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:18:48.561631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:18:48.561631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:18:48.561631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:18:48.561631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:18:48.561631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:18:48.561631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:18:48.561631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:18:48.561631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:18:48.561631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:18:48.561631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:18:48.561631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:18:48.562061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:18:48.562061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:18:48.562061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:48.562061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:18:48.562061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:48.562061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:18:48.562061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:48.562061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:18:48.562061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:48.562061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:18:48.562061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:48.562061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:18:48.562061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:18:48.562061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:48.562061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:18:48.562061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:48.562061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:18:48.562061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:18:48.562061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:48.562892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:18:48.562892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:48.562892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:18:52.975312 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:18:52.975961 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:18:52.975961 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:18:52.976805 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:18:52.982275 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG dbcounter [-] [106314] Writing DB stats neutron:SELECT=240 {{(pid=106314) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 09 09:18:53.019625 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:18:53.020457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:53.021414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:18:53.021414 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:53.021414 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:18:53.021414 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:18:53.021414 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:53.021414 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:18:53.021414 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:53.021414 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:18:53.022968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.045 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:53.022968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:18:53.022968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:18:53.022968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:18:53.022968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:18:53.022968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:18:53.022968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:18:53.022968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:53.022968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:18:53.022968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:53.022968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:18:53.022968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:53.023324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:18:53.023324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:53.023324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:18:53.023324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:53.023324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:18:53.023324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:18:53.023324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:53.023324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:18:53.023324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:53.023324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:18:53.023324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:18:53.023324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:53.023324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:18:53.023619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:53.023619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:18:53.024762 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:53.024762 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:18:53.024762 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:18:53.024762 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:18:53.024762 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:18:53.024762 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:18:53.024762 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:18:53.024762 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:18:53.024762 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:18:53.024762 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:18:53.024762 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:18:53.024762 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:18:53.024762 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:18:53.024762 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:18:53.024762 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:18:53.024762 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:18:53.025198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:18:53.025198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:18:53.025198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:53.025198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:18:53.025198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:53.025198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:18:53.025198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:53.025198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:18:53.025198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:53.025198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:18:53.025198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:53.025198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:18:53.025198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:18:53.025198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:53.025198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:18:53.025198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:53.025198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:18:53.025198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:18:53.025198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:53.025652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:18:53.025652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:53.025652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:18:53.530806 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:18:53.531408 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:18:53.531719 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:18:53.533078 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:18:53.533280 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:18:53.533410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:53.534071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:18:53.534071 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:53.534071 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:18:53.534071 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:18:53.534071 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:53.534071 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:18:53.534071 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:53.534071 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:18:53.536982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:53.536982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:18:53.536982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:18:53.536982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:18:53.536982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:18:53.536982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:18:53.536982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:18:53.536982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:53.536982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:18:53.536982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:53.536982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:18:53.536982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:53.541519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:18:53.541519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:53.541519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:18:53.541519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:53.541519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:18:53.541519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:18:53.541519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:53.541519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:18:53.541519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:53.541519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:18:53.541519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:18:53.541519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:53.541519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:18:53.541903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:53.541903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:18:53.541903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:53.541903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:18:53.541903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:18:53.541903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:18:53.541903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:18:53.541903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:18:53.541903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:18:53.541903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:18:53.541903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:18:53.541903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:18:53.541903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:18:53.541903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:18:53.541903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:18:53.542309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:18:53.542309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:18:53.542309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:18:53.542309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:18:53.542309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:18:53.542309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:53.542309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:18:53.542309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:53.542309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:18:53.542309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:53.542309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:18:53.542309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:53.542309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:18:53.542309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:53.542309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:18:53.542309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:18:53.542309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:53.542309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:18:53.542309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:53.542846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:18:53.542846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:18:53.542846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:53.542846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:18:53.542846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:53.542846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:18:53.542846 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:18:53.542846 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:18:53.542846 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:18:53.542846 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:18:53.543137 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:18:53.543137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:53.543137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:18:53.543137 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:53.543137 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:18:53.543137 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:18:53.543137 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:53.543137 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:18:53.543137 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:53.543137 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:18:53.543137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:53.543137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:18:53.543137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:18:53.543137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:18:53.543537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:53.543537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:18:53.543537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:53.543537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:18:53.543537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:53.543537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:18:53.543537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:53.543537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:18:53.543537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:53.543537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:18:53.543537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:18:53.543537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:53.543537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:18:53.543537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:53.543537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:18:53.543537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:18:53.543537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:53.543537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:18:53.544067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:53.544067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:18:53.544067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:53.544067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:18:53.544067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:18:53.544067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:18:53.544067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:18:53.544067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:18:53.544067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:18:53.544067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:18:53.544067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:18:53.544067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:18:53.544408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:53.544408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:18:53.544408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:53.544408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:18:53.544408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:53.544408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:18:53.544408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:53.544408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:18:53.544408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:53.544408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:18:53.544408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:18:53.544408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:53.544408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:18:53.544408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:53.544762 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:18:53.544762 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:18:53.544762 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:53.544762 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:18:53.544762 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:53.544762 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:18:53.544762 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:53.544762 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:18:53.544762 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:18:53.544762 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:18:53.544762 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:18:53.544762 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:18:53.544762 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:18:53.544762 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:18:53.545151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:18:53.545151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:18:53.545151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:18:53.545151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:18:53.545151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:18:53.545151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:18:53.545151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:18:53.545151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:18:53.545151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:18:53.545151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:18:53.545151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:18:53.545151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:18:53.545151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:18:53.545151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:53.545151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:18:53.545151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:53.545151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:18:53.545151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:53.545151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:18:53.545631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:53.545631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:18:53.545631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:53.545631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:18:53.545631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:18:53.545631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:53.545631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:18:53.545631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:53.545631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:18:53.545631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:18:53.545631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:53.545631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:18:53.545631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:53.545631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:18:53.547702 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:18:53.548064 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:18:53.548245 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:18:53.548351 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:18:53.548758 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:18:53.548992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:53.549472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:18:53.549472 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:53.549472 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:18:53.549472 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:18:53.549472 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:53.549472 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:18:53.549472 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:53.549472 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:18:53.550406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:53.550406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:18:53.550406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:18:53.550406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:18:53.550406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:53.550406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:18:53.550406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:53.550406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:18:53.550406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:53.550406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:18:53.550406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:53.550406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:18:53.550406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:53.550406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:18:53.550406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:18:53.550899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:53.550899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:18:53.550899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:53.550899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:18:53.550899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:18:53.550899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:53.550899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:18:53.550899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:53.550899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:18:53.551356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:53.551356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:18:53.551356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:18:53.551356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:18:53.551356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:18:53.551356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:18:53.551356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:18:53.551356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:18:53.551356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:18:53.551356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:53.551356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:18:53.551724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:53.551724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:18:53.551724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:53.551724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:18:53.551724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:53.551724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:18:53.551724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:53.551724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:18:53.551724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:18:53.551724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:53.551724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:18:53.551724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:53.551724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:18:53.551724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:18:53.552341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:53.552341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:18:53.552341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:53.552341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:18:53.552651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:53.552651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:18:53.552651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:18:53.552651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:18:53.552651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:18:53.552651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:18:53.552651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:18:53.552651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:18:53.552651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:18:53.552651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:18:53.552651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:18:53.552651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:18:53.552651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:18:53.552651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:18:53.552651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:18:53.552651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:18:53.552651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:18:53.553233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:18:53.553233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:18:53.553233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:18:53.553233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:53.553233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:18:53.553233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:53.553233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:18:53.553233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:53.553233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:18:53.553233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:53.553233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:18:53.553233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:53.553233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:18:53.553233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:18:53.553233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:53.553233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:18:53.553233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:53.553233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:18:53.553233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:18:53.553895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:53.553895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:18:53.553895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:53.553895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:18:53.557645 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:18:53.558152 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:18:53.558327 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:18:53.558484 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:18:53.559063 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:18:53.559310 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:18:53.559469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:53.560119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:18:53.560119 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:53.560119 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:18:53.560119 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:18:53.560119 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:53.560119 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:18:53.560119 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:53.560119 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:18:53.561291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:53.561291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:18:53.561291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:18:53.561291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:18:53.561291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:18:53.561291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:18:53.561291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:18:53.561291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:53.561291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:18:53.561291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:53.561291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:18:53.561291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:53.562032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:18:53.562032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:53.562032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:18:53.562032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:53.562032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:18:53.562032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:18:53.562032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:53.562032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:18:53.562032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:53.562032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:18:53.562032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:18:53.562032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:53.562032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:18:53.562504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:53.562504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:18:53.562882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:53.562882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:18:53.562882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:18:53.562882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:18:53.562882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:18:53.562882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:18:53.562882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:18:53.562882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:18:53.562882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:18:53.562882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:18:53.562882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:18:53.562882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:18:53.562882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:18:53.562882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:18:53.562882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:18:53.562882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:18:53.563484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:18:53.563484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:18:53.563484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:53.563484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:18:53.563484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:53.563484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:18:53.563484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:53.563484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:18:53.563484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:53.563484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:18:53.563484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:53.563484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:18:53.563484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:18:53.563484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:53.563484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:18:53.563484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:53.563484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:18:53.563484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:18:53.563484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:53.564035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:18:53.564035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:53.564035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:18:57.975907 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:18:57.976643 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:18:57.976643 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:18:57.977645 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:18:58.014705 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:18:58.014839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:58.015516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:18:58.015516 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:58.015516 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:18:58.015516 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:18:58.015516 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:58.015516 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:18:58.015516 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:58.015516 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:18:58.016282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:58.016282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:18:58.016282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:18:58.016282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:18:58.016282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:18:58.016282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:18:58.016282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:18:58.016282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:58.016282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:18:58.016282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:58.016282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:18:58.016282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:58.016722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:18:58.016722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:58.016722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:18:58.016722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:58.016722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:18:58.016722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:18:58.016722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:58.016722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:18:58.016722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:58.016722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:18:58.016722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:18:58.016722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:58.016722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:18:58.017212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:58.017212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:18:58.017212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:58.017212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:18:58.017212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:18:58.017212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:18:58.017212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:18:58.017212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:18:58.017212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:18:58.017212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:18:58.017212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:18:58.017212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:18:58.017212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:18:58.017212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:18:58.017212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:18:58.017774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:18:58.017774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:18:58.017774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:18:58.017774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:18:58.017774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:18:58.017774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:58.017774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:18:58.017774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:58.017774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:18:58.017774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:58.017774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:18:58.017774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:58.017774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:18:58.017774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:58.017774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:18:58.017774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:18:58.017774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:58.017774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:18:58.017774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:58.018382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:18:58.018382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:18:58.018382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:58.018382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:18:58.018382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:58.018382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:18:58.531597 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:18:58.532206 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:18:58.532476 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:18:58.533537 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:18:58.533762 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:18:58.533952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:58.534810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:18:58.534810 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:58.534810 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:18:58.534810 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:18:58.534810 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:58.534810 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:18:58.534810 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:58.534810 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:18:58.537078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:58.537078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:18:58.537078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:18:58.537078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:18:58.537078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:18:58.537078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:18:58.537078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:18:58.537078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:58.537078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:18:58.537078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:58.537078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:18:58.537078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:58.538041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:18:58.538041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:58.538041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:18:58.538041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:58.538041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:18:58.538041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:18:58.538041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:58.538041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:18:58.538041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:58.538041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:18:58.538041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:18:58.538041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:58.538041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:18:58.538957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:58.538957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:18:58.540581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:58.540581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:18:58.540581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:18:58.540581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:18:58.540581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:18:58.540581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:18:58.540581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:18:58.540581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:18:58.540581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:18:58.540581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:18:58.540581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:18:58.540581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:18:58.540581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:18:58.540581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:18:58.540581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:18:58.540581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:18:58.541727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:18:58.541727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:18:58.541727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:58.541727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:18:58.541727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:58.541727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:18:58.541727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:58.541727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:18:58.541727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:58.541727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:18:58.541727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:58.541727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:18:58.541727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:18:58.541727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:58.541727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:18:58.541727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:58.541727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:18:58.541727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:18:58.541727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:58.542400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:18:58.542400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:58.542400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:18:58.542400 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:18:58.542400 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:18:58.542400 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:18:58.542400 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:18:58.542400 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:18:58.542678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:58.543191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:18:58.543191 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:58.543191 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:18:58.543191 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:18:58.543191 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:58.543191 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:18:58.543191 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:58.543191 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:18:58.543996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:58.543996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:18:58.543996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:18:58.543996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:18:58.543996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:58.543996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:18:58.543996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:58.543996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:18:58.543996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:58.543996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:18:58.543996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:58.543996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:18:58.543996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:58.543996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:18:58.543996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:18:58.544720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:58.544720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:18:58.544720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:58.544720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:18:58.544720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:18:58.544720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:58.544720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:18:58.544720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:58.544720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:18:58.545010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:58.545010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:18:58.545010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:18:58.545010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:18:58.545010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:18:58.545010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:18:58.545010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:18:58.545010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:18:58.545010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:18:58.545010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:18:58.545010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:58.545010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:18:58.545356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:58.545356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:18:58.545356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:58.545356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:18:58.545356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:58.545356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:18:58.545356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:58.545356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:18:58.545356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:18:58.545356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:58.545356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:18:58.545356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:58.545356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:18:58.545356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:18:58.545783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:58.545783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:18:58.545783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:58.545783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:18:58.545920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:58.545920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:18:58.545920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:18:58.545920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:18:58.545920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:18:58.545920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:18:58.545920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:18:58.545920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:18:58.545920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:18:58.545920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:18:58.545920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:18:58.545920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:18:58.545920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:18:58.545920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:18:58.545920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:18:58.545920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:18:58.546340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:18:58.546340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:18:58.546340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:18:58.546340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:18:58.546340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:18:58.546340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:58.546340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:18:58.546340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:58.546340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:18:58.546340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:58.546340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:18:58.546340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:58.546340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:18:58.546340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:58.546340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:18:58.546340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:18:58.546340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:58.546340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:18:58.546687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:58.546687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:18:58.546687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:18:58.546687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:58.546687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:18:58.546687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:58.546687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:18:58.548082 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:18:58.548379 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:18:58.548503 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:18:58.548708 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:18:58.549238 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:18:58.549419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:58.549996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:18:58.549996 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:58.549996 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:18:58.549996 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:18:58.549996 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:58.549996 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:18:58.549996 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:58.549996 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:18:58.550985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:58.550985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:18:58.550985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:18:58.550985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:18:58.550985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:58.550985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:18:58.550985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:58.550985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:18:58.550985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:58.550985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:18:58.550985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:58.550985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:18:58.550985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:58.550985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:18:58.550985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:18:58.551456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:58.551456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:18:58.551456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:58.551456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:18:58.551456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:18:58.551456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:58.551456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:18:58.551456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:58.551456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:18:58.551949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:58.551949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:18:58.551949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:18:58.551949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:18:58.551949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:18:58.551949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:18:58.551949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:18:58.551949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:18:58.551949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:18:58.551949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:58.551949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:18:58.552280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:58.552280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:18:58.552280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:58.552280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:18:58.552280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:58.552280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:18:58.552280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:58.552280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:18:58.552280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:18:58.552280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:58.552280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:18:58.552280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:58.552280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:18:58.552280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:18:58.552694 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:58.552694 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:18:58.552694 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:58.552694 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:18:58.553229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:58.553229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:18:58.553229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:18:58.553229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:18:58.553229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:18:58.553229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:18:58.553229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:18:58.553229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:18:58.553229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:18:58.553229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:18:58.553229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:18:58.553229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:18:58.553229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:18:58.553229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:18:58.553229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:18:58.553229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:18:58.553229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:18:58.553737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:18:58.553737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:18:58.553737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:18:58.553737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:58.553737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:18:58.553737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:58.553737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:18:58.553737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:58.553737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:18:58.553737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:58.553737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:18:58.553737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:58.553737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:18:58.553737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:18:58.553737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:58.553737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:18:58.553737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:58.553737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:18:58.553737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:18:58.554274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:58.554274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:18:58.554274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:58.554274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:18:58.558175 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:18:58.558468 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:18:58.558580 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:18:58.558761 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:18:58.560380 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:18:58.560654 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:18:58.560843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:58.561585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:18:58.561585 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:58.561585 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:18:58.561585 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:18:58.561585 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:58.561585 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:18:58.561585 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:58.561585 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:18:58.562620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:58.562620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:18:58.562620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:18:58.562620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:18:58.562620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:18:58.562620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:18:58.562620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:18:58.562620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:58.562620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:18:58.562620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:58.562620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:18:58.562620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:58.563135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:18:58.563135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:58.563135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:18:58.563135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:58.563135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:18:58.563135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:18:58.563135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:58.563135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:18:58.563135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:58.563135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:18:58.563135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:18:58.563135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:58.563135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:18:58.563509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:58.563509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:18:58.563825 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:58.563825 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:18:58.563825 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:18:58.563825 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:18:58.563825 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:18:58.563825 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:18:58.563825 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:18:58.563825 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:18:58.563825 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:18:58.563825 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:18:58.563825 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:18:58.563825 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:18:58.563825 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:18:58.563825 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:18:58.563825 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:18:58.563825 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:18:58.564981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:18:58.564981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:18:58.564981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:58.564981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:18:58.564981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:18:58.564981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:18:58.564981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:18:58.564981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:18:58.564981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:18:58.564981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:18:58.564981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:18:58.564981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:18:58.564981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:18:58.564981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:18:58.564981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:18:58.564981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:18:58.564981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:18:58.564981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:18:58.564981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:18:58.565722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:18:58.565722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:18:58.565722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:19:02.976510 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:19:02.977309 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:19:02.977309 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:19:02.978319 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:19:03.022542 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:19:03.022687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:03.026801 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:19:03.026801 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:03.026801 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:19:03.026801 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:19:03.026801 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:03.026801 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:19:03.026801 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:03.026801 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:19:03.026801 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.046 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:03.026801 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:19:03.026801 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:19:03.026801 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:19:03.026801 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:19:03.026801 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:19:03.026801 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:19:03.027403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:03.027403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:19:03.027403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:03.027403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:19:03.027403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:03.027403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:19:03.027403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:03.027403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:19:03.027403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:03.027403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:19:03.027403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:19:03.027403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:03.027403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:19:03.027403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:03.027831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:19:03.027831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:19:03.027831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:03.027831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:19:03.027831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:03.027831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:19:03.027831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:03.027831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:19:03.027831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:19:03.027831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:19:03.027831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:19:03.027831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:19:03.027831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:19:03.027831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:19:03.028260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:19:03.028260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:19:03.028260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:19:03.028260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:19:03.028260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:19:03.028260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:19:03.028260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:19:03.028260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:19:03.028260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:19:03.028260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:19:03.028260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:03.028260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:19:03.028260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:03.028260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:19:03.028260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:03.028260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:19:03.028260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:03.028260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:19:03.028260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:03.028260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:19:03.028260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:19:03.028906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:03.028906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:19:03.028906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:03.028906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:19:03.028906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:19:03.028906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:03.028906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:19:03.028906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:03.028906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:19:03.532194 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:19:03.532705 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:19:03.532827 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:19:03.534840 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:19:03.534840 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:19:03.534840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:03.534840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:19:03.534840 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:03.534840 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:19:03.534840 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:19:03.534840 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:03.534840 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:19:03.534840 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:03.534840 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:19:03.535214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:03.535214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:19:03.535214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:19:03.535214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:19:03.535214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:19:03.535214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:19:03.535214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:19:03.535214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:03.535214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:19:03.535214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:03.535214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:19:03.535214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:03.535536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:19:03.535536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:03.535536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:19:03.535536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:03.535536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:19:03.535536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:19:03.535536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:03.535536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:19:03.535536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:03.535536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:19:03.535536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:19:03.535536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:03.535536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:19:03.535903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:03.535903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:19:03.536046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:03.536046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:19:03.536046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:19:03.536046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:19:03.536046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:19:03.536046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:19:03.536046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:19:03.536046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:19:03.536046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:19:03.536046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:19:03.536046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:19:03.536046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:19:03.536046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:19:03.536046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:19:03.536046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:19:03.536046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:19:03.536459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:19:03.536459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:19:03.536459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:03.536459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:19:03.536459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:03.536459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:19:03.536459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:03.536459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:19:03.536459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:03.536459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:19:03.536459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:03.536459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:19:03.536459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:19:03.536459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:03.536459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:19:03.536459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:03.536459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:19:03.536459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:19:03.536459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:03.537068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:19:03.537068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:03.537068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:19:03.541576 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:19:03.541843 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:19:03.541934 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:19:03.542034 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:19:03.542391 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:19:03.542538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:03.543083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:19:03.543083 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:03.543083 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:19:03.543083 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:19:03.543083 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:03.543083 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:19:03.543083 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:03.543083 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:19:03.543681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:03.543681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:19:03.543681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:19:03.543681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:19:03.543681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:03.543681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:19:03.543681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:03.543681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:19:03.543681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:03.543681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:19:03.543681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:03.543681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:19:03.543681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:03.543681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:19:03.543681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:19:03.544097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:03.544097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:19:03.544097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:03.544097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:19:03.544097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:19:03.544097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:03.544097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:19:03.544097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:03.544097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:19:03.544274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:03.544274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:19:03.544274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:19:03.544274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:19:03.544274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:19:03.544274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:19:03.544274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:19:03.544274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:19:03.544274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:19:03.544274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:19:03.544274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:03.544274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:19:03.544552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:03.544552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:19:03.544552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:03.544552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:19:03.544552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:03.544552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:19:03.544552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:03.544552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:19:03.544552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:19:03.544552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:03.544552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:19:03.544552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:03.544552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:19:03.544552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:19:03.544845 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:03.544845 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:19:03.544845 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:03.544845 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:19:03.545137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:03.545137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:19:03.545137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:19:03.545137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:19:03.545137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:19:03.545137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:19:03.545137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:19:03.545137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:19:03.545137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:19:03.545137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:19:03.545137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:19:03.545137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:19:03.545137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:19:03.545137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:19:03.545137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:19:03.545137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:19:03.545562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:19:03.545562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:19:03.545562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:19:03.545562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:19:03.545562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:19:03.545562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:03.545562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:19:03.545562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:03.545562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:19:03.545562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:03.545562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:19:03.545562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:03.545562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:19:03.545562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:03.545562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:19:03.545562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:19:03.545562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:03.545562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:19:03.546056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:03.546056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:19:03.546056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:19:03.546056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:03.546056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:19:03.546056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:03.546056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:19:03.548557 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:19:03.549910 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:19:03.549910 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:19:03.549910 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:19:03.550199 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:19:03.550445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:03.551095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:19:03.551095 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:03.551095 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:19:03.551095 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:19:03.551095 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:03.551095 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:19:03.551095 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:03.551095 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:19:03.552483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:03.552483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:19:03.552483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:19:03.552483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:19:03.552483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:03.552483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:19:03.552483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:03.552483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:19:03.552483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:03.552483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:19:03.552483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:03.552483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:19:03.552483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:03.552483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:19:03.552483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:19:03.553105 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:03.553105 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:19:03.553105 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:03.553105 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:19:03.553105 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:19:03.553105 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:03.553105 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:19:03.553105 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:03.553105 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:19:03.553105 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:03.553105 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:19:03.553105 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:19:03.553105 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:19:03.553105 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:19:03.553830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:19:03.553830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:19:03.553830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:19:03.553830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:19:03.553830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:03.553830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:19:03.553830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:03.553830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:19:03.553830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:03.553830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:19:03.553830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:03.553830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:19:03.553830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:03.554306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:19:03.554306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:19:03.554306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:03.554306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:19:03.554306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:03.554306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:19:03.554306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:19:03.554306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:03.554306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:19:03.554306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:03.554306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:19:03.554306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:03.554721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:19:03.554721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:19:03.554721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:19:03.554721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:19:03.554721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:19:03.554721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:19:03.554721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:19:03.554721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:19:03.554721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:19:03.554721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:19:03.554721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:19:03.554721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:19:03.554721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:19:03.554721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:19:03.554721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:19:03.554721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:19:03.554721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:19:03.554721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:19:03.554721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:19:03.555252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:03.555252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:19:03.555252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:03.555252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:19:03.555252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:03.555252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:19:03.555252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:03.555252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:19:03.555252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:03.555252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:19:03.555252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:19:03.555252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:03.555252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:19:03.555252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:03.555252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:19:03.555252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:19:03.555252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:03.555252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:19:03.555252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:03.555252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:19:03.558677 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:19:03.559011 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:19:03.559093 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:19:03.559259 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:19:03.559823 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:19:03.560601 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:19:03.561008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:03.561681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:19:03.561681 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:03.561681 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:19:03.561681 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:19:03.561681 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:03.561681 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:19:03.561681 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:03.561681 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:19:03.562789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:03.562789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:19:03.562789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:19:03.562789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:19:03.562789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:19:03.562789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:19:03.562789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:19:03.562789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:03.562789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:19:03.562789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:03.562789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:19:03.562789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:03.563133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:19:03.563133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:03.563133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:19:03.563133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:03.563133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:19:03.563133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:19:03.563133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:03.563133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:19:03.563133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:03.563133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:19:03.563133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:19:03.563133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:03.563133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:19:03.563473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:03.563473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:19:03.563473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:03.563473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:19:03.563473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:19:03.563473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:19:03.563473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:19:03.563473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:19:03.563473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:19:03.563473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:19:03.563473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:19:03.563473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:19:03.563473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:19:03.563473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:19:03.563473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:19:03.563900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:19:03.563900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:19:03.563900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:19:03.563900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:19:03.563900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:19:03.563900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:03.563900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:19:03.563900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:03.563900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:19:03.563900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:03.563900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:19:03.563900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:03.563900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:19:03.563900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:03.563900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:19:03.563900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:19:03.563900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:03.563900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:19:03.563900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:03.564637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:19:03.564637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:19:03.564637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:03.564637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:19:03.564637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:03.564637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:19:07.977163 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:19:07.977959 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:19:07.977959 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:19:07.979014 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:19:08.020596 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:19:08.020834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:08.021561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:19:08.021561 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:08.021561 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:19:08.021561 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:19:08.021561 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:08.021561 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:19:08.021561 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:08.021561 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:19:08.022596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.044 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:08.022596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:19:08.022596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:19:08.022596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:19:08.022596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:19:08.022596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:19:08.022596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:19:08.022596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:08.022596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:19:08.022596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:08.022596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:19:08.022596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:08.023413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:19:08.023413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:08.023413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:19:08.023413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:08.023413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:19:08.023413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:19:08.023413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:08.023413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:19:08.023413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:08.023413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:19:08.023413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:19:08.023413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:08.023413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:19:08.023977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:08.023977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:19:08.023977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:08.023977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:19:08.023977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:19:08.023977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:19:08.023977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:19:08.023977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:19:08.023977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:19:08.023977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:19:08.023977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:19:08.023977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:19:08.023977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:19:08.023977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:19:08.023977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:19:08.024551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:19:08.024551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:19:08.024551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:19:08.024551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:19:08.024551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:19:08.024551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:08.024551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:19:08.024551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:08.024551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:19:08.024551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:08.024551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:19:08.024551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:08.024551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:19:08.024551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:08.024551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:19:08.024551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:19:08.024551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:08.024551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:19:08.024551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:08.025305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:19:08.025305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:19:08.025305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:08.025305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:19:08.025305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:08.025305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:19:08.573301 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:19:08.573301 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:19:08.573301 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:19:08.573301 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:19:08.573301 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:19:08.573301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:08.573647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:19:08.573647 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:08.573647 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:19:08.573647 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:19:08.573647 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:08.573647 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:19:08.573647 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:08.573647 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:19:08.573647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:08.573647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:19:08.573647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:19:08.573647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:19:08.573647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:19:08.573647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:19:08.573647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:19:08.574143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:08.574143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:19:08.574143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:08.574143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:19:08.574143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:08.574143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:19:08.574143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:08.574143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:19:08.574143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:08.574143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:19:08.574143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:19:08.574143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:08.574143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:19:08.574143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:08.574544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:19:08.574544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:19:08.574544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:08.574544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:19:08.574544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:08.574544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:19:08.574544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:08.574544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:19:08.574544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:19:08.574544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:19:08.574544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:19:08.574544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:19:08.574544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:19:08.574544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:19:08.574989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:19:08.574989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:19:08.574989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:19:08.574989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:19:08.574989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:19:08.574989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:19:08.574989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:19:08.574989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:19:08.574989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:19:08.574989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:19:08.574989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:08.574989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:19:08.574989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:08.574989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:19:08.574989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:08.574989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:19:08.574989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:08.574989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:19:08.574989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:08.574989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:19:08.575623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:19:08.575623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:08.575623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:19:08.575623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:08.575623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:19:08.575623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:19:08.575623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:08.575623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:19:08.575623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:08.575623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:19:08.575623 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:19:08.575623 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:19:08.575962 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:19:08.575962 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:19:08.575962 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:19:08.575962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:08.575962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:19:08.575962 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:08.575962 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:19:08.575962 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:19:08.575962 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:08.575962 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:19:08.575962 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:08.575962 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:19:08.576467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:08.576467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:19:08.576467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:19:08.576467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:19:08.576467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:08.576467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:19:08.576467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:08.576467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:19:08.576467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:08.576467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:19:08.576467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:08.576467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:19:08.576467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:08.576467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:19:08.576467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:19:08.577177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:08.577177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:19:08.577177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:08.577177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:19:08.577177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:19:08.577177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:08.577177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:19:08.577177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:08.577177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:19:08.577177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:08.577177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:19:08.577177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:19:08.577177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:19:08.577177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:19:08.577855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:19:08.577855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:19:08.577855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:19:08.577855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:19:08.577855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:19:08.577855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:08.577855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:19:08.577855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:08.577855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:19:08.577855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:08.577855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:19:08.577855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:08.577855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:19:08.578195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:08.578195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:19:08.578195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:19:08.578195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:08.578195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:19:08.578195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:08.578195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:19:08.578195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:19:08.578195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:08.578195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:19:08.578195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:08.578195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:19:08.578528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:08.578528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:19:08.578528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:19:08.578528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:19:08.578528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:19:08.578528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:19:08.578528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:19:08.578528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:19:08.578528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:19:08.578528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:19:08.578528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:19:08.578528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:19:08.578528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:19:08.578528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:19:08.578528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:19:08.578528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:19:08.578993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:19:08.578993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:19:08.578993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:19:08.578993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:19:08.578993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:19:08.578993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:08.578993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:19:08.578993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:08.578993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:19:08.578993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:08.578993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:19:08.578993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:08.578993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:19:08.578993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:08.578993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:19:08.578993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:19:08.578993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:08.578993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:19:08.579319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:08.579319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:19:08.579319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:19:08.579319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:08.579319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:19:08.579319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:08.579319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:19:08.579319 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:19:08.579319 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:19:08.579319 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:19:08.579555 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:19:08.579555 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:19:08.579555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:08.579555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:19:08.579555 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:08.579555 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:19:08.579555 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:19:08.579555 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:08.579555 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:19:08.579555 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:08.579555 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:19:08.579555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:08.579798 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:19:08.579798 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:19:08.579798 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:19:08.579798 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:08.579798 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:19:08.579798 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:08.579798 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:19:08.579798 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:08.579798 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:19:08.579798 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:08.579798 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:19:08.579798 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:08.579798 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:19:08.579798 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:19:08.579798 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:08.579798 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:19:08.579798 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:08.580135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:19:08.580135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:19:08.580135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:08.580135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:19:08.580135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:08.580135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:19:08.580135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:08.580135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:19:08.580135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:19:08.580135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:19:08.580135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:19:08.580135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:19:08.580135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:19:08.580511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:19:08.580511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:19:08.580511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:08.580511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:19:08.580511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:08.580511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:19:08.580511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:08.580511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:19:08.580511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:08.580511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:19:08.580511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:08.580511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:19:08.580511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:19:08.580769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:08.580769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:19:08.580769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:08.580769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:19:08.580769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:19:08.580769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:08.580769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:19:08.580769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:08.580769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:19:08.580769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:08.580769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:19:08.580769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:19:08.581000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:19:08.581000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:19:08.581000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:19:08.581000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:19:08.581000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:19:08.581000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:19:08.581000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:19:08.581000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:19:08.581000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:19:08.581000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:19:08.581000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:19:08.581000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:19:08.581000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:19:08.581000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:19:08.581000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:19:08.581000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:19:08.581000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:19:08.581000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:08.581000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:19:08.581590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:08.581590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:19:08.581590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:08.581590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:19:08.581590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:08.581590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:19:08.581590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:08.581590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:19:08.581590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:19:08.581590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:08.581590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:19:08.581590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:08.581590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:19:08.581590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:19:08.581590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:08.581590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:19:08.581590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:08.581590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:19:08.581935 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:19:08.581935 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:19:08.581935 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:19:08.581935 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:19:08.581935 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:19:08.581935 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:19:08.582073 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:08.582073 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:19:08.582073 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:08.582073 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:19:08.582073 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:19:08.582073 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:08.582073 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:19:08.582073 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:08.582073 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:19:08.582073 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:08.582073 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:19:08.582073 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:19:08.582073 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:19:08.582073 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:19:08.582073 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:19:08.582372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:19:08.582372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:08.582372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:19:08.582372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:08.582372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:19:08.582372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:08.582372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:19:08.582372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:08.582372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:19:08.582372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:08.582372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:19:08.582372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:19:08.582372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:08.582372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:19:08.582725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:08.582725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:19:08.582725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:19:08.582725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:08.582725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:19:08.582725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:08.582725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:19:08.582725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:08.582725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:19:08.582725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:19:08.582725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:19:08.582725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:19:08.582725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:19:08.582992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:19:08.582992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:19:08.582992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:19:08.582992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:19:08.582992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:19:08.582992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:19:08.582992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:19:08.582992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:19:08.582992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:19:08.582992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:19:08.582992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:19:08.582992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:19:08.582992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:08.582992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:19:08.582992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:08.582992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:19:08.582992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:08.582992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:19:08.582992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:08.582992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:19:08.583346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:08.583346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:19:08.583346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:19:08.583346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:08.583346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:19:08.583346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:08.583346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:19:08.583346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:19:08.583346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:08.583346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:19:08.583346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:08.583346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:19:12.978000 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:19:12.978964 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:19:12.978964 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:19:12.979441 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:19:13.015713 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:19:13.015999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:13.016534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:19:13.016534 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:13.016534 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:19:13.016534 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:19:13.016534 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:13.016534 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:19:13.016534 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:13.016534 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:19:13.017780 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:13.017780 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:19:13.017780 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:19:13.017780 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:19:13.017780 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:19:13.017780 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:19:13.017780 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:19:13.017780 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:13.017780 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:19:13.017780 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:13.017780 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:19:13.017780 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:13.018101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:19:13.018101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:13.018101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:19:13.018101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:13.018101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:19:13.018101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:19:13.018101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:13.018101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:19:13.018101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:13.018101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:19:13.018101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:19:13.018101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:13.018101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:19:13.018472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:13.018472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:19:13.018472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:13.018472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:19:13.018472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:19:13.018472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:19:13.018472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:19:13.018472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:19:13.018472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:19:13.018472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:19:13.018472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:19:13.018472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:19:13.018472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:19:13.018472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:19:13.018472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:19:13.018916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:19:13.018916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:19:13.018916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:19:13.018916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:19:13.018916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:19:13.018916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:13.018916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:19:13.018916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:13.018916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:19:13.018916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:13.018916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:19:13.018916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:13.018916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:19:13.018916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:13.018916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:19:13.018916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:19:13.018916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:13.018916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:19:13.018916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:13.019344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:19:13.019344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:19:13.019344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:13.019344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:19:13.019344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:13.019344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:19:13.533613 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:19:13.534156 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:19:13.534156 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:19:13.534522 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:19:13.534682 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:19:13.534852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:13.535427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:19:13.535427 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:13.535427 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:19:13.535427 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:19:13.535427 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:13.535427 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:19:13.535427 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:13.535427 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:19:13.537219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:13.537219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:19:13.537219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:19:13.537219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:19:13.537219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:19:13.537219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:19:13.537219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:19:13.537219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:13.537219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:19:13.537219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:13.537219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:19:13.537219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:13.537604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:19:13.537604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:13.537604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:19:13.537604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:13.537604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:19:13.537604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:19:13.537604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:13.537604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:19:13.537604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:13.537604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:19:13.537604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:19:13.537604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:13.537604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:19:13.537992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:13.537992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:19:13.539235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:13.539235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:19:13.539235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:19:13.539235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:19:13.539235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:19:13.539235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:19:13.539235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:19:13.539235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:19:13.539235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:19:13.539235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:19:13.539235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:19:13.539235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:19:13.539235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:19:13.539235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:19:13.539235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:19:13.539235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:19:13.539632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:19:13.539632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:19:13.539632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:13.539632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:19:13.539632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:13.539632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:19:13.539632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:13.539632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:19:13.539632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:13.539632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:19:13.539632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:13.539632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:19:13.539632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:19:13.539632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:13.539632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:19:13.539632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:13.539632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:19:13.539632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:19:13.539632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:13.540124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:19:13.540124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:13.540124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:19:13.542315 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:19:13.542863 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:19:13.543070 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:19:13.543360 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:19:13.543714 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:19:13.544012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:13.544523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:19:13.544523 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:13.544523 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:19:13.544523 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:19:13.544523 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:13.544523 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:19:13.544523 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:13.544523 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:19:13.546499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:13.546499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:19:13.546499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:19:13.546499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:19:13.546499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:13.546499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:19:13.546499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:13.546499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:19:13.546499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:13.546499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:19:13.546499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:13.546499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:19:13.546499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:13.546499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:19:13.546499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:19:13.548556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:13.548556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:19:13.548556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:13.548556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:19:13.548556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:19:13.548556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:13.548556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:19:13.548556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:13.548556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:19:13.549135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:13.549135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:19:13.549135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:19:13.549135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:19:13.549135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:19:13.549135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:19:13.549135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:19:13.549135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:19:13.549135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:19:13.549135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:19:13.549135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:13.549135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:19:13.549654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:13.549654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:19:13.549654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:13.549654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:19:13.549654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:13.549654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:19:13.549654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:13.549654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:19:13.549654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:19:13.549654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:13.549654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:19:13.549654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:13.549654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:19:13.549654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:19:13.553043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:13.553043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:19:13.553043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:13.553043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:19:13.553043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:13.553043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:19:13.553043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:19:13.553043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:19:13.553043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:19:13.553043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:19:13.553043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:19:13.553043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:19:13.553043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:19:13.553043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:19:13.554147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:19:13.554147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:19:13.554147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:19:13.554147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:19:13.554147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:19:13.554147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:19:13.554147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:19:13.554147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:19:13.554147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:19:13.554147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:19:13.554147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:19:13.554147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:13.554147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:19:13.554147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:13.554147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:19:13.554147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:13.554147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:19:13.554147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:13.554147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:19:13.554962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:13.554962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:19:13.554962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:19:13.554962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:13.554962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:19:13.554962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:13.554962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:19:13.554962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:19:13.554962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:13.554962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:19:13.554962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:13.554962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:19:13.554962 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:19:13.554962 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:19:13.556858 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:19:13.556858 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:19:13.556858 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:19:13.556858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:13.556858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:19:13.556858 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:13.556858 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:19:13.556858 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:19:13.556858 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:13.556858 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:19:13.556858 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:13.556858 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:19:13.557649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:13.557649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:19:13.557649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:19:13.557649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:19:13.557649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:13.557649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:19:13.557649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:13.557649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:19:13.557649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:13.557649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:19:13.557649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:13.557649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:19:13.557649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:13.557649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:19:13.557649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:19:13.558173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:13.558173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:19:13.558173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:13.558173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:19:13.558173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:19:13.558173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:13.558173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:19:13.558173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:13.558173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:19:13.558173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:13.558173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:19:13.558173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:19:13.558173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:19:13.558173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:19:13.558582 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:19:13.558582 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:19:13.558582 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:19:13.558582 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:19:13.558582 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:13.558582 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:19:13.558582 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:13.558582 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:19:13.558582 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:13.558582 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:19:13.558582 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:13.558582 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:19:13.558582 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:13.558922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:19:13.558922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:19:13.558922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:13.558922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:19:13.558922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:13.558922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:19:13.558922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:19:13.558922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:13.558922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:19:13.558922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:13.558922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:19:13.558922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:13.559239 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:19:13.559239 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:19:13.559239 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:19:13.559239 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:19:13.559239 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:19:13.559239 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:19:13.559239 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:19:13.559239 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:19:13.559239 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:19:13.559239 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:19:13.559239 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:19:13.559239 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:19:13.559239 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:19:13.559239 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:19:13.559239 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:19:13.559239 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:19:13.559239 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:19:13.559239 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:19:13.559239 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:19:13.559926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:13.559926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:19:13.559926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:13.559926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:19:13.559926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:13.559926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:19:13.559926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:13.559926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:19:13.559926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:13.559926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:19:13.559926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:19:13.559926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:13.559926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:19:13.559926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:13.559926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:19:13.559926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:19:13.559926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:13.559926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:19:13.559926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:13.559926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:19:13.560788 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:19:13.560788 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:19:13.560788 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:19:13.560788 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:19:13.560788 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:19:13.560971 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:19:13.562080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:13.562080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:19:13.562080 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:13.562080 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:19:13.562080 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:19:13.562080 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:13.562080 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:19:13.562080 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:13.562080 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:19:13.562357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:13.562357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:19:13.562357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:19:13.562357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:19:13.562357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:19:13.562357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:19:13.562357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:19:13.562357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:13.562357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:19:13.562357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:13.562357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:19:13.562357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:13.563005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:19:13.563005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:13.563005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:19:13.563005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:13.563005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:19:13.563005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:19:13.563005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:13.563005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:19:13.563005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:13.563005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:19:13.563005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:19:13.563005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:13.563005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:19:13.563370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:13.563370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:19:13.563534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:13.563534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:19:13.563534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:19:13.563534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:19:13.563534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:19:13.563534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:19:13.563534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:19:13.563534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:19:13.563534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:19:13.563534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:19:13.563534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:19:13.563534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:19:13.563534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:19:13.563534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:19:13.563534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:19:13.563534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:19:13.563992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:19:13.563992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:19:13.563992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:13.563992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:19:13.563992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:13.563992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:19:13.563992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:13.563992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:19:13.563992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:13.563992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:19:13.563992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:13.563992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:19:13.563992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:19:13.563992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:13.563992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:19:13.563992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:13.563992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:19:13.563992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:19:13.563992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:13.565283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:19:13.565283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:13.565283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:19:17.978554 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:19:17.979189 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:19:17.979189 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:19:17.979590 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:19:18.017038 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:19:18.017245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:18.017964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:19:18.017964 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:18.017964 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:19:18.017964 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:19:18.017964 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:18.017964 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:19:18.017964 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:18.017964 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:19:18.018949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:18.018949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:19:18.018949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:19:18.018949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:19:18.018949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:19:18.018949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:19:18.018949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:19:18.018949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:18.018949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:19:18.018949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:18.018949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:19:18.018949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:18.019205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:19:18.019205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:18.019205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:19:18.019205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:18.019205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:19:18.019205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:19:18.019205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:18.019205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:19:18.019205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:18.019205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:19:18.019205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:19:18.019205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:18.019205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:19:18.019466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:18.019466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:19:18.020080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:18.020080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:19:18.020080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:19:18.020080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:19:18.020080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:19:18.020080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:19:18.020080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:19:18.020080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:19:18.020080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:19:18.020080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:19:18.020080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:19:18.020080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:19:18.020080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:19:18.020080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:19:18.020080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:19:18.020080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:19:18.020399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:19:18.020399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:19:18.020399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:18.020399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:19:18.020399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:18.020399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:19:18.020399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:18.020399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:19:18.020399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:18.020399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:19:18.020399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:18.020399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:19:18.020399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:19:18.020399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:18.020399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:19:18.020399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:18.020399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:19:18.020399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:19:18.020399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:18.020776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:19:18.020776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:18.020776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:19:18.533995 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:19:18.534502 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:19:18.534809 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:19:18.535716 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:19:18.535920 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:19:18.536035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:18.536878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:19:18.536878 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:18.536878 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:19:18.536878 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:19:18.536878 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:18.536878 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:19:18.536878 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:18.536878 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:19:18.539693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:18.539693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:19:18.539693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:19:18.539693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:19:18.539693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:19:18.539693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:19:18.539693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:19:18.539693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:18.539693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:19:18.539693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:18.539693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:19:18.539693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:18.540129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:19:18.540129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:18.540129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:19:18.540129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:18.540129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:19:18.540129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:19:18.540129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:18.540129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:19:18.540129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:18.540129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:19:18.540129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:19:18.540129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:18.540129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:19:18.540538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:18.540538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:19:18.542866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:18.542866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:19:18.542866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:19:18.542866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:19:18.542866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:19:18.542866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:19:18.542866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:19:18.542866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:19:18.542866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:19:18.542866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:19:18.542866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:19:18.542866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:19:18.542866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:19:18.542866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:19:18.542866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:19:18.542866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:19:18.543375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:19:18.543375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:19:18.543375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:18.543375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:19:18.543375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:18.543375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:19:18.543375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:18.543375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:19:18.543375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:18.543375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:19:18.543375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:18.543375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:19:18.543375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:19:18.543375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:18.543375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:19:18.543375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:18.543375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:19:18.543375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:19:18.543375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:18.543963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:19:18.543963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:18.543963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:19:18.543963 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:19:18.544544 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:19:18.544757 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:19:18.545017 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:19:18.545663 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:19:18.546029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:18.547132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:19:18.547132 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:18.547132 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:19:18.547132 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:19:18.547132 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:18.547132 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:19:18.547132 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:18.547132 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:19:18.548454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:18.548454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:19:18.548454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:19:18.548454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:19:18.548454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:18.548454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:19:18.548454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:18.548454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:19:18.548454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:18.548454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:19:18.548454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:18.548454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:19:18.548454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:18.548454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:19:18.548454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:19:18.549209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:18.549209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:19:18.549209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:18.549209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:19:18.549209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:19:18.549209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:18.549209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:19:18.549209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:18.549209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:19:18.549738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:18.549738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:19:18.549738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:19:18.549738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:19:18.549738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:19:18.549738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:19:18.549738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:19:18.549738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:19:18.549738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:19:18.549738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:19:18.549738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:18.549738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:19:18.550164 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:18.550164 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:19:18.550164 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:18.550164 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:19:18.550164 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:18.550164 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:19:18.550164 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:18.550164 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:19:18.550164 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:19:18.550164 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:18.550164 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:19:18.550164 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:18.550164 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:19:18.550164 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:19:18.550573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:18.550573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:19:18.550573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:18.550573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:19:18.551346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:18.551346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:19:18.551346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:19:18.551346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:19:18.551346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:19:18.551346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:19:18.551346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:19:18.551346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:19:18.551346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:19:18.551346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:19:18.551346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:19:18.551346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:19:18.551346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:19:18.551346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:19:18.551346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:19:18.551346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:19:18.551804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:19:18.551804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:19:18.551804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:19:18.551804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:19:18.551804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:19:18.551804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:18.551804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:19:18.551804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:18.551804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:19:18.551804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:18.551804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:19:18.551804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:18.551804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:19:18.551804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:18.551804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:19:18.551804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:19:18.551804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:18.551804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:19:18.552283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:18.552283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:19:18.552283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:19:18.552283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:18.552283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:19:18.552283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:18.552283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:19:18.552967 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:19:18.553259 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:19:18.553397 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:19:18.553592 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:19:18.554438 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:19:18.554794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:18.555562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:19:18.555562 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:18.555562 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:19:18.555562 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:19:18.555562 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:18.555562 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:19:18.555562 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:18.555562 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:19:18.556761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:18.556761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:19:18.556761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:19:18.556761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:19:18.556761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:18.556761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:19:18.556761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:18.556761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:19:18.556761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:18.556761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:19:18.556761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:18.556761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:19:18.556761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:18.556761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:19:18.556761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:19:18.557161 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:18.557161 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:19:18.557161 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:18.557161 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:19:18.557161 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:19:18.557161 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:18.557161 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:19:18.557161 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:18.557161 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:19:18.558091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:18.558091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:19:18.558091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:19:18.558091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:19:18.558091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:19:18.558091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:19:18.558091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:19:18.558091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:19:18.558091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:19:18.558091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:18.558091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:19:18.558402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:18.558402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:19:18.558402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:18.558402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:19:18.558402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:18.558402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:19:18.558402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:18.558402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:19:18.558402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:19:18.558402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:18.558402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:19:18.558402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:18.558402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:19:18.558402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:19:18.558786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:18.558786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:19:18.558786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:18.558786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:19:18.559853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:18.559853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:19:18.559853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:19:18.559853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:19:18.559853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:19:18.559853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:19:18.559853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:19:18.559853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:19:18.559853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:19:18.559853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:19:18.559853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:19:18.559853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:19:18.559853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:19:18.559853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:19:18.559853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:19:18.559853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:19:18.559853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:19:18.560339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:19:18.560339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:19:18.560339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:19:18.560339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:18.560339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:19:18.560339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:18.560339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:19:18.560339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:18.560339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:19:18.560339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:18.560339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:19:18.560339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:18.560339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:19:18.560339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:19:18.560339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:18.560339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:19:18.560339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:18.560339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:19:18.560339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:19:18.560896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:18.560896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:19:18.560896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:18.560896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:19:18.560896 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:19:18.560896 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:19:18.560896 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:19:18.561169 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:19:18.561409 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:19:18.561644 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:19:18.561813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:18.562421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:19:18.562421 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:18.562421 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:19:18.562421 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:19:18.562421 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:18.562421 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:19:18.562421 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:18.562421 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:19:18.563281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:18.563281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:19:18.563281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:19:18.563281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:19:18.563281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:19:18.563281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:19:18.563281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:19:18.563281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:18.563281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:19:18.563281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:18.563281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:19:18.563281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:18.563618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:19:18.563618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:18.563618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:19:18.563618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:18.563618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:19:18.563618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:19:18.563618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:18.563618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:19:18.563618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:18.563618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:19:18.563618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:19:18.563618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:18.563618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:19:18.563968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:18.563968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:19:18.564030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:18.564030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:19:18.564030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:19:18.564030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:19:18.564030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:19:18.564030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:19:18.564030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:19:18.564030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:19:18.564030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:19:18.564030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:19:18.564030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:19:18.564030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:19:18.564030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:19:18.564030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:19:18.564030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:19:18.564030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:19:18.564433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:19:18.564433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:19:18.564433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:18.564433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:19:18.564433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:18.564433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:19:18.564433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:18.564433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:19:18.564433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:18.564433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:19:18.564433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:18.564433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:19:18.564433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:19:18.564433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:18.564433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:19:18.564433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:18.564433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:19:18.564433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:19:18.564433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:18.564943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:19:18.564943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:18.564943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:19:22.979343 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:19:22.982665 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:19:22.982665 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:19:22.982665 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:19:23.022244 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:19:23.022583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:23.024124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:19:23.024124 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:23.024124 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:19:23.024124 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:19:23.024124 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:23.024124 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:19:23.024124 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:23.024124 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:19:23.025214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.043 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:23.025214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:19:23.025214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:19:23.025214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:19:23.025214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:19:23.025214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:19:23.025214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:19:23.025214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:23.025214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:19:23.025214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:23.025214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:19:23.025214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:23.025581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:19:23.025581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:23.025581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:19:23.025581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:23.025581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:19:23.025581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:19:23.025581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:23.025581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:19:23.025581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:23.025581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:19:23.025581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:19:23.025581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:23.025581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:19:23.025949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:23.025949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:19:23.027525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:23.027525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:19:23.027525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:19:23.027525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:19:23.027525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:19:23.027525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:19:23.027525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:19:23.027525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:19:23.027525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:19:23.027525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:19:23.027525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:19:23.027525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:19:23.027525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:19:23.027525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:19:23.027525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:19:23.027525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:19:23.028004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:19:23.028004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:19:23.028004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:23.028004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:19:23.028004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:23.028004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:19:23.028004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:23.028004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:19:23.028004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:23.028004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:19:23.028004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:23.028004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:19:23.028004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:19:23.028004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:23.028004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:19:23.028004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:23.028004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:19:23.028004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:19:23.028004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:23.028478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:19:23.028478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:23.028478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:19:23.534836 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:19:23.535465 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:19:23.535700 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:19:23.536781 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:19:23.537085 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:19:23.537373 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:23.538271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:19:23.538271 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:23.538271 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:19:23.538271 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:19:23.538271 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:23.538271 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:19:23.538271 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:23.538271 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:19:23.542091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:23.542091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:19:23.542091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:19:23.542091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:19:23.542091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:19:23.542091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:19:23.542091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:19:23.542091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:23.542091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:19:23.542091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:23.542091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:19:23.542091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:23.543687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:19:23.543687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:23.543687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:19:23.543687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:23.543687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:19:23.543687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:19:23.543687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:23.543687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:19:23.543687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:23.543687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:19:23.543687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:19:23.543687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:23.543687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:19:23.544160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:23.544160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:19:23.545211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:23.545211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:19:23.545211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:19:23.545211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:19:23.545211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:19:23.545211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:19:23.545211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:19:23.545211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:19:23.545211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:19:23.545211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:19:23.545211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:19:23.545211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:19:23.545211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:19:23.545211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:19:23.545211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:19:23.545211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:19:23.546482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:19:23.546482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:19:23.546482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:23.546482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:19:23.546482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:23.546482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:19:23.546482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:23.546482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:19:23.546482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:23.546482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:19:23.546482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:23.546482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:19:23.546482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:19:23.546482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:23.546482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:19:23.546482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:23.546482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:19:23.546482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:19:23.546482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:23.548177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:19:23.548177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:23.548177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:19:23.548177 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:19:23.548177 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:19:23.548177 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:19:23.548177 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:19:23.548177 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:19:23.548546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:23.550779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:19:23.550779 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:23.550779 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:19:23.550779 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:19:23.550779 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:23.550779 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:19:23.550779 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:23.550779 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:19:23.550779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:23.550779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:19:23.550779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:19:23.550779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:19:23.550779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:23.550779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:19:23.550779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:23.550779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:19:23.550779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:23.550779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:19:23.551495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:23.551495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:19:23.551495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:23.551495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:19:23.551495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:19:23.551495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:23.551495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:19:23.551495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:23.551495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:19:23.551495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:19:23.551495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:23.551495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:19:23.551495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:23.551495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:19:23.551495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:23.552569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:19:23.552569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:19:23.552569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:19:23.552569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:19:23.552569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:19:23.552569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:19:23.552569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:19:23.552569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:19:23.552569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:19:23.552569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:23.552569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:19:23.552569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:23.552569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:19:23.553115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:23.553115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:19:23.553115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:23.553115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:19:23.553115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:23.553115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:19:23.553115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:19:23.553115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:23.553115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:19:23.553115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:23.553115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:19:23.553115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:19:23.553115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:23.553115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:19:23.553602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:23.553602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:19:23.553602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:23.553602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:19:23.553602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:19:23.553602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:19:23.553602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:19:23.553602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:19:23.553602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:19:23.553602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:19:23.553602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:19:23.553602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:19:23.553602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:19:23.553602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:19:23.553602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:19:23.553890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:19:23.553890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:19:23.553890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:19:23.553890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:19:23.553890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:19:23.553890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:19:23.553890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:19:23.553890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:19:23.553890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:23.553890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:19:23.553890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:23.553890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:19:23.553890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:23.553890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:19:23.553890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:23.553890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:19:23.553890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:23.553890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:19:23.553890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:19:23.556679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:23.556679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:19:23.556679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:23.556679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:19:23.556679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:19:23.556679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:23.556679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:19:23.556679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:23.556679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:19:23.556679 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:19:23.556679 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:19:23.556679 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:19:23.557021 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:19:23.557021 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:19:23.557021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:23.557021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:19:23.557021 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:23.557021 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:19:23.557021 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:19:23.557021 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:23.557021 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:19:23.557021 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:23.557021 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:19:23.557021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:23.557362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:19:23.557362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:19:23.557362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:19:23.557362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:23.557362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:19:23.557362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:23.557362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:19:23.557362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:23.557362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:19:23.557362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:23.557362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:19:23.557362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:23.557362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:19:23.557362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:19:23.557362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:23.557362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:19:23.557362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:23.557814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:19:23.557814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:19:23.557814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:23.557814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:19:23.557814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:23.557814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:19:23.557814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:23.557814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:19:23.557814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:19:23.557814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:19:23.557814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:19:23.557814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:19:23.557814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:19:23.558333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:19:23.558333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:19:23.558333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:23.558333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:19:23.558333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:23.558333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:19:23.558333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:23.558333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:19:23.558333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:23.558333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:19:23.558333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:23.558333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:19:23.558333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:19:23.558571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:23.558571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:19:23.558571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:23.558571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:19:23.558571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:19:23.558571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:23.558571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:19:23.558571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:23.558571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:19:23.558861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:23.558861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:19:23.558861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:19:23.558861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:19:23.558861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:19:23.558861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:19:23.558861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:19:23.558861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:19:23.558861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:19:23.558861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:19:23.558861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:19:23.558861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:19:23.558861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:19:23.558861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:19:23.558861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:19:23.558861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:19:23.558861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:19:23.559271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:19:23.559271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:19:23.559271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:19:23.559271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:23.559271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:19:23.559271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:23.559271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:19:23.559271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:23.559271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:19:23.559271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:23.559271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:19:23.559271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:23.559271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:19:23.559271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:19:23.559271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:23.559271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:19:23.559271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:23.559271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:19:23.559271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:19:23.559741 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:23.559741 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:19:23.559741 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:23.559741 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:19:23.561122 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:19:23.561721 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:19:23.561865 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:19:23.562112 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:19:23.562704 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:19:23.565659 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:19:23.565659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:23.565659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:19:23.565659 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:23.565659 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:19:23.565659 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:19:23.565659 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:23.565659 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:19:23.565659 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:23.565659 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:19:23.565659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:23.565659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:19:23.565659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:19:23.565659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:19:23.566148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:19:23.566148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:19:23.566148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:19:23.566148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:23.566148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:19:23.566148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:23.566148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:19:23.566148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:23.566148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:19:23.566148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:23.566148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:19:23.566148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:23.566148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:19:23.566148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:19:23.566511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:23.566511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:19:23.566511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:23.566511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:19:23.566511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:19:23.566511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:23.566511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:19:23.566511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:23.566511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:19:23.566511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:23.566511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:19:23.566511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:19:23.566847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:19:23.566847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:19:23.566847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:19:23.566847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:19:23.566847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:19:23.566847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:19:23.566847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:19:23.566847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:19:23.566847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:19:23.566847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:19:23.566847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:19:23.566847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:19:23.566847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:19:23.566847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:19:23.566847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:19:23.566847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:23.566847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:19:23.566847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:23.566847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:19:23.566847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:23.567356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:19:23.567356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:23.567356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:19:23.567356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:23.567356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:19:23.567356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:19:23.567356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:23.567356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:19:23.567356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:23.567356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:19:23.567356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:19:23.567356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:23.567356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:19:23.567356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:23.567356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:19:27.980197 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:19:27.981005 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:19:27.981005 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:19:27.981681 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:19:28.016248 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:19:28.016798 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:28.017986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:19:28.017986 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:28.017986 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:19:28.017986 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:19:28.017986 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:28.017986 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:19:28.017986 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:28.017986 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:19:28.019320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:28.019320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:19:28.019320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:19:28.019320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:19:28.019320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:19:28.019320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:19:28.019320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:19:28.019320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:28.019320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:19:28.019320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:28.019320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:19:28.019320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:28.019606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:19:28.019606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:28.019606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:19:28.019606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:28.019606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:19:28.019606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:19:28.019606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:28.019606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:19:28.019606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:28.019606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:19:28.019606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:19:28.019606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:28.019606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:19:28.019864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:28.019864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:19:28.020823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:28.020823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:19:28.020823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:19:28.020823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:19:28.020823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:19:28.020823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:19:28.020823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:19:28.020823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:19:28.020823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:19:28.020823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:19:28.020823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:19:28.020823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:19:28.020823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:19:28.020823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:19:28.020823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:19:28.020823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:19:28.021621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:19:28.021621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:19:28.021621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:28.021621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:19:28.021621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:28.021621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:19:28.021621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:28.021621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:19:28.021621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:28.021621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:19:28.021621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:28.021621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:19:28.021621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:19:28.021621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:28.021621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:19:28.021621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:28.021621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:19:28.021621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:19:28.021621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:28.021993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:19:28.021993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:28.021993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:19:28.535718 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:19:28.536408 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:19:28.536623 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:19:28.537682 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:19:28.537937 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:19:28.538065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:28.538731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:19:28.538731 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:28.538731 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:19:28.538731 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:19:28.538731 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:28.538731 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:19:28.538731 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:28.538731 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:19:28.539829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:28.539829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:19:28.539829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:19:28.539829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:19:28.539829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:19:28.539829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:19:28.539829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:19:28.539829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:28.539829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:19:28.539829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:28.539829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:19:28.539829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:28.540653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:19:28.540653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:28.540653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:19:28.540653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:28.540653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:19:28.540653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:19:28.540653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:28.540653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:19:28.540653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:28.540653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:19:28.540653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:19:28.540653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:28.540653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:19:28.541252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:28.541252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:19:28.541252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:28.541252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:19:28.541252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:19:28.541252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:19:28.541252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:19:28.541252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:19:28.541252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:19:28.541252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:19:28.541252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:19:28.541252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:19:28.541252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:19:28.541252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:19:28.541252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:19:28.542320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:19:28.542320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:19:28.542320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:19:28.542320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:19:28.542320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:19:28.542320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:28.542320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:19:28.542320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:28.542320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:19:28.542320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:28.542320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:19:28.542320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:28.542320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:19:28.542320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:28.542320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:19:28.542320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:19:28.542320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:28.542320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:19:28.542320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:28.544216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:19:28.544216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:19:28.544216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:28.544216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:19:28.544216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:28.544216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:19:28.546716 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:19:28.547213 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:19:28.547429 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:19:28.547429 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:19:28.548004 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:19:28.548224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:28.548930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:19:28.548930 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:28.548930 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:19:28.548930 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:19:28.548930 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:28.548930 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:19:28.548930 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:28.548930 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:19:28.549961 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:28.549961 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:19:28.549961 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:19:28.549961 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:19:28.549961 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:28.549961 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:19:28.549961 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:28.549961 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:19:28.549961 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:28.549961 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:19:28.549961 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:28.549961 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:19:28.549961 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:28.549961 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:19:28.549961 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:19:28.550589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:28.550589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:19:28.550589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:28.550589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:19:28.550589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:19:28.550589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:28.550589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:19:28.550589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:28.550589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:19:28.550935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:28.550935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:19:28.550935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:19:28.550935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:19:28.550935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:19:28.550935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:19:28.550935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:19:28.550935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:19:28.550935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:19:28.550935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:19:28.550935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:28.550935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:19:28.551359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:28.551359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:19:28.551359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:28.551359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:19:28.551359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:28.551359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:19:28.551359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:28.551359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:19:28.551359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:19:28.551359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:28.551359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:19:28.551359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:28.551359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:19:28.551359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:19:28.552167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:28.552167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:19:28.552167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:28.552167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:19:28.552403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:28.552403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:19:28.552403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:19:28.552403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:19:28.552403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:19:28.552403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:19:28.552403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:19:28.552403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:19:28.552403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:19:28.552403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:19:28.552403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:19:28.552403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:19:28.552403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:19:28.552403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:19:28.552403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:19:28.552403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:19:28.553013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:19:28.553013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:19:28.553013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:19:28.553013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:19:28.553013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:19:28.553013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:28.553013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:19:28.553013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:28.553013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:19:28.553013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:28.553013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:19:28.553013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:28.553013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:19:28.553013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:28.553013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:19:28.553013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:19:28.553013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:28.553013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:19:28.553710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:28.553710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:19:28.553710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:19:28.553710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:28.553710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:19:28.553710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:28.553710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:19:28.554088 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:19:28.554376 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:19:28.554530 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:19:28.554665 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:19:28.555170 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:19:28.555445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:28.556103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:19:28.556103 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:28.556103 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:19:28.556103 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:19:28.556103 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:28.556103 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:19:28.556103 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:28.556103 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:19:28.556919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:28.556919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:19:28.556919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:19:28.556919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:19:28.556919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:28.556919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:19:28.556919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:28.556919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:19:28.556919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:28.556919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:19:28.556919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:28.556919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:19:28.556919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:28.556919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:19:28.556919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:19:28.557564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:28.557564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:19:28.557564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:28.557564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:19:28.557564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:19:28.557564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:28.557564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:19:28.557564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:28.557564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:19:28.557950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:28.557950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:19:28.557950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:19:28.557950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:19:28.557950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:19:28.557950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:19:28.557950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:19:28.557950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:19:28.557950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:19:28.557950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:28.557950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:19:28.558386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:28.558386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:19:28.558386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:28.558386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:19:28.558386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:28.558386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:19:28.558386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:28.558386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:19:28.558386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:19:28.558386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:28.558386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:19:28.558386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:28.558386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:19:28.558386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:19:28.559041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:28.559041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:19:28.559041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:28.559041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:19:28.559041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:28.559041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:19:28.559041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:19:28.559041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:19:28.559041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:19:28.559041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:19:28.559041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:19:28.559041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:19:28.559041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:19:28.559041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:19:28.560057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:19:28.560057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:19:28.560057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:19:28.560057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:19:28.560057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:19:28.560057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:19:28.560057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:19:28.560057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:19:28.560057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:19:28.560057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:19:28.560057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:28.560057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:19:28.560057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:28.560057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:19:28.560057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:28.560057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:19:28.560057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:28.560057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:19:28.560057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:28.560671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:19:28.560671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:19:28.560671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:28.560671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:19:28.560671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:28.560671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:19:28.560671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:19:28.560671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:28.560671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:19:28.560671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:28.560671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:19:28.561863 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:19:28.562145 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:19:28.562234 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:19:28.562375 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:19:28.563244 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:19:28.563549 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:19:28.563718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:28.564326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:19:28.564326 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:28.564326 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:19:28.564326 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:19:28.564326 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:28.564326 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:19:28.564326 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:28.564326 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:19:28.565036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:28.565036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:19:28.565036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:19:28.565036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:19:28.565036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:19:28.565036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:19:28.565036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:19:28.565036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:28.565036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:19:28.565036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:28.565036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:19:28.565036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:28.565462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:19:28.565462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:28.565462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:19:28.565462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:28.565462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:19:28.565462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:19:28.565462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:28.565462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:19:28.565462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:28.565462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:19:28.565462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:19:28.565462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:28.565462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:19:28.565877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:28.565877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:19:28.565877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:28.565877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:19:28.565877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:19:28.565877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:19:28.565877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:19:28.565877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:19:28.565877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:19:28.565877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:19:28.565877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:19:28.565877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:19:28.565877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:19:28.565877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:19:28.565877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:19:28.566255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:19:28.566255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:19:28.566255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:19:28.566255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:19:28.566255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:19:28.566255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:28.566255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:19:28.566255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:28.566255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:19:28.566255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:28.566255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:19:28.566255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:28.566255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:19:28.566255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:28.566255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:19:28.566255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:19:28.566255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:28.566255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:19:28.566255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:28.566707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:19:28.566707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:19:28.566707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:28.566707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:19:28.566707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:28.566707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:19:32.980974 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:19:32.981619 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:19:32.981619 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:19:32.982556 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:19:33.021615 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:19:33.022100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:33.023493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:19:33.023493 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:33.023493 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:19:33.023493 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:19:33.023493 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:33.023493 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:19:33.023493 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:33.023493 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:19:33.024649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:33.024649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:19:33.024649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:19:33.024649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:19:33.024649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:19:33.024649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:19:33.024649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:19:33.024649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:33.024649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:19:33.024649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:33.024649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:19:33.024649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:33.025132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:19:33.025132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:33.025132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:19:33.025132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:33.025132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:19:33.025132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:19:33.025132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:33.025132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:19:33.025132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:33.025132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:19:33.025132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:19:33.025132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:33.025132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:19:33.025591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:33.025591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:19:33.025883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:33.025883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:19:33.025883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:19:33.025883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:19:33.025883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:19:33.025883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:19:33.025883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:19:33.025883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:19:33.025883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:19:33.025883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:19:33.025883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:19:33.025883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:19:33.025883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:19:33.025883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:19:33.025883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:19:33.025883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:19:33.027200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:19:33.027200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:19:33.027200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:33.027200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:19:33.027200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:33.027200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:19:33.027200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:33.027200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:19:33.027200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:33.027200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:19:33.027200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:33.027200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:19:33.027200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:19:33.027200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:33.027200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:19:33.027200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:33.027200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:19:33.027200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:19:33.027200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:33.027800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:19:33.027800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:33.027800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:19:33.536614 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:19:33.537005 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:19:33.537072 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:19:33.538080 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:19:33.538383 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:19:33.538599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:33.539359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:19:33.539359 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:33.539359 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:19:33.539359 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:19:33.539359 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:33.539359 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:19:33.539359 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:33.539359 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:19:33.541138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:33.541138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:19:33.541138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:19:33.541138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:19:33.541138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:19:33.541138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:19:33.541138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:19:33.541138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:33.541138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:19:33.541138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:33.541138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:19:33.541138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:33.541434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:19:33.541434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:33.541434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:19:33.541434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:33.541434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:19:33.541434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:19:33.541434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:33.541434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:19:33.541434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:33.541434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:19:33.541434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:19:33.541434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:33.541434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:19:33.541680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:33.541680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:19:33.543713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:33.543713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:19:33.543713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:19:33.543713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:19:33.543713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:19:33.543713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:19:33.543713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:19:33.543713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:19:33.543713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:19:33.543713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:19:33.543713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:19:33.543713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:19:33.543713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:19:33.543713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:19:33.543713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:19:33.543713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:19:33.544113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:19:33.544113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:19:33.544113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:33.544113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:19:33.544113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:33.544113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:19:33.544113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:33.544113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:19:33.544113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:33.544113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:19:33.544113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:33.544113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:19:33.544113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:19:33.544113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:33.544113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:19:33.544113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:33.544113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:19:33.544113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:19:33.544113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:33.544524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:19:33.544524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:33.544524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:19:33.547206 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:19:33.547564 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:19:33.547697 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:19:33.547900 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:19:33.548493 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:19:33.548721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:33.549510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:19:33.549510 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:33.549510 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:19:33.549510 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:19:33.549510 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:33.549510 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:19:33.549510 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:33.549510 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:19:33.550361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:33.550361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:19:33.550361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:19:33.550361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:19:33.550361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:33.550361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:19:33.550361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:33.550361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:19:33.550361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:33.550361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:19:33.550361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:33.550361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:19:33.550361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:33.550361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:19:33.550361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:19:33.550803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:33.550803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:19:33.550803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:33.550803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:19:33.550803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:19:33.550803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:33.550803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:19:33.550803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:33.550803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:19:33.551951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:33.551951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:19:33.551951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:19:33.551951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:19:33.551951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:19:33.551951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:19:33.551951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:19:33.551951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:19:33.551951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:19:33.551951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:19:33.551951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:33.551951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:19:33.552299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:33.552299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:19:33.552299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:33.552299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:19:33.552299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:33.552299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:19:33.552299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:33.552299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:19:33.552299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:19:33.552299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:33.552299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:19:33.552299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:33.552299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:19:33.552299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:19:33.552684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:33.552684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:19:33.552684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:33.552684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:19:33.553549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:33.553549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:19:33.553549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:19:33.553549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:19:33.553549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:19:33.553549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:19:33.553549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:19:33.553549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:19:33.553549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:19:33.553549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:19:33.553549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:19:33.553549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:19:33.553549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:19:33.553549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:19:33.553549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:19:33.553549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:19:33.553990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:19:33.553990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:19:33.553990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:19:33.553990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:19:33.553990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:19:33.553990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:33.553990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:19:33.553990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:33.553990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:19:33.553990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:33.553990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:19:33.553990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:33.553990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:19:33.553990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:33.553990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:19:33.553990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:19:33.553990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:33.553990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:19:33.554526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:33.554526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:19:33.554526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:19:33.554526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:33.554526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:19:33.554526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:33.554526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:19:33.554526 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:19:33.554972 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:19:33.555165 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:19:33.555307 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:19:33.555719 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:19:33.556053 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:33.556738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:19:33.556738 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:33.556738 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:19:33.556738 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:19:33.556738 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:33.556738 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:19:33.556738 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:33.556738 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:19:33.557631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:33.557631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:19:33.557631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:19:33.557631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:19:33.557631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:33.557631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:19:33.557631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:33.557631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:19:33.557631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:33.557631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:19:33.557631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:33.557631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:19:33.557631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:33.557631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:19:33.557631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:19:33.558067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:33.558067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:19:33.558067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:33.558067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:19:33.558067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:19:33.558067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:33.558067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:19:33.558067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:33.558067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:19:33.558736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:33.558736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:19:33.558736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:19:33.558736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:19:33.558736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:19:33.558736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:19:33.558736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:19:33.558736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:19:33.558736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:19:33.558736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:33.558736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:19:33.559124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:33.559124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:19:33.559124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:33.559124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:19:33.559124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:33.559124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:19:33.559124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:33.559124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:19:33.559124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:19:33.559124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:33.559124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:19:33.559124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:33.559124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:19:33.559124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:19:33.559497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:33.559497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:19:33.559497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:33.559497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:19:33.560609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:33.560609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:19:33.560609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:19:33.560609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:19:33.560609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:19:33.560609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:19:33.560609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:19:33.560609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:19:33.560609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:19:33.560609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:19:33.560609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:19:33.560609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:19:33.560609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:19:33.560609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:19:33.560609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:19:33.560609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:19:33.560609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:19:33.561135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:19:33.561135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:19:33.561135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:19:33.561135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:33.561135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:19:33.561135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:33.561135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:19:33.561135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:33.561135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:19:33.561135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:33.561135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:19:33.561135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:33.561135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:19:33.561135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:19:33.561135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:33.561135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:19:33.561135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:33.561135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:19:33.561135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:19:33.561662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:33.561662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:19:33.561662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:33.561662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:19:33.562492 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:19:33.562887 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:19:33.563018 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:19:33.563215 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:19:33.563562 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:19:33.563870 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:19:33.564069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:33.564702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:19:33.564702 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:33.564702 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:19:33.564702 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:19:33.564702 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:33.564702 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:19:33.564702 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:33.564702 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:19:33.565655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:33.565655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:19:33.565655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:19:33.565655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:19:33.565655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:19:33.565655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:19:33.565655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:19:33.565655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:33.565655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:19:33.565655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:33.565655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:19:33.565655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:33.566042 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:19:33.566042 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:33.566042 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:19:33.566042 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:33.566042 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:19:33.566042 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:19:33.566042 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:33.566042 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:19:33.566042 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:33.566042 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:19:33.566042 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:19:33.566042 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:33.566042 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:19:33.566532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:33.566532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:19:33.566995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:33.566995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:19:33.566995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:19:33.566995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:19:33.566995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:19:33.566995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:19:33.566995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:19:33.566995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:19:33.566995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:19:33.566995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:19:33.566995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:19:33.566995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:19:33.566995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:19:33.566995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:19:33.566995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:19:33.566995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:19:33.567448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:19:33.567448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:19:33.567448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:33.567448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:19:33.567448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:33.567448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:19:33.567448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:33.567448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:19:33.567448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:33.567448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:19:33.567448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:33.567448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:19:33.567448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:19:33.567448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:33.567448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:19:33.567448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:33.567448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:19:33.567448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:19:33.567448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:33.567994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:19:33.567994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:33.567994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:19:37.981455 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:19:37.981876 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:19:37.982066 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:19:37.984277 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:19:38.020532 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:19:38.021068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:38.022073 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:19:38.022073 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:38.022073 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:19:38.022073 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:19:38.022073 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:38.022073 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:19:38.022073 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:38.022073 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:19:38.023037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:38.023037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:19:38.023037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:19:38.023037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:19:38.023037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:19:38.023037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:19:38.023037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:19:38.023037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:38.023037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:19:38.023037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:38.023037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:19:38.023037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:38.023515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:19:38.023515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:38.023515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:19:38.023515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:38.023515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:19:38.023515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:19:38.023515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:38.023515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:19:38.023515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:38.023515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:19:38.023515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:19:38.023515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:38.023515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:19:38.024045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:38.024045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:19:38.024045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:38.024045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:19:38.024045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:19:38.024045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:19:38.024045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:19:38.024045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:19:38.024045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:19:38.024045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:19:38.024045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:19:38.024045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:19:38.024045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:19:38.024045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:19:38.024045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:19:38.024579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:19:38.024579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:19:38.024579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:19:38.024579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:19:38.024579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:19:38.024579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:38.024579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:19:38.024579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:38.024579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:19:38.024579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:38.024579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:19:38.024579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:38.024579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:19:38.024579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:38.024579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:19:38.024579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:19:38.024579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:38.024579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:19:38.024579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:38.025261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:19:38.025261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:19:38.025261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:38.025261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:19:38.025261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:38.025261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:19:38.537308 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:19:38.537776 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:19:38.537885 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:19:38.538691 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:19:38.539014 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:19:38.539241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:38.539859 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:19:38.539859 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:38.539859 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:19:38.539859 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:19:38.539859 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:38.539859 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:19:38.539859 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:38.539859 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:19:38.540666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:38.540666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:19:38.540666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:19:38.540666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:19:38.540666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:19:38.540666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:19:38.540666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:19:38.540666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:38.540666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:19:38.540666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:38.540666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:19:38.540666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:38.541114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:19:38.541114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:38.541114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:19:38.541114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:38.541114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:19:38.541114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:19:38.541114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:38.541114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:19:38.541114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:38.541114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:19:38.541114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:19:38.541114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:38.541114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:19:38.541451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:38.541451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:19:38.541451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:38.541451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:19:38.541451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:19:38.541451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:19:38.541451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:19:38.541451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:19:38.541451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:19:38.541451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:19:38.541451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:19:38.541451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:19:38.541451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:19:38.541451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:19:38.541451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:19:38.542217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:19:38.542217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:19:38.542217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:19:38.542217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:19:38.542217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:19:38.542217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:38.542217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:19:38.542217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:38.542217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:19:38.542217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:38.542217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:19:38.542217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:38.542217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:19:38.542217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:38.542217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:19:38.542217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:19:38.542217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:38.542217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:19:38.542217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:38.543226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:19:38.543226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:19:38.543226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:38.543226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:19:38.543226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:38.543226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:19:38.547609 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:19:38.547852 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:19:38.547929 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:19:38.548027 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:19:38.548416 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:19:38.548619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:38.549223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:19:38.549223 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:38.549223 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:19:38.549223 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:19:38.549223 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:38.549223 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:19:38.549223 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:38.549223 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:19:38.550023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:38.550023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:19:38.550023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:19:38.550023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:19:38.550023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:38.550023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:19:38.550023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:38.550023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:19:38.550023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:38.550023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:19:38.550023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:38.550023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:19:38.550023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:38.550023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:19:38.550023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:19:38.550809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:38.550809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:19:38.550809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:38.550809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:19:38.550809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:19:38.550809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:38.550809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:19:38.550809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:38.550809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:19:38.551162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:38.551162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:19:38.551162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:19:38.551162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:19:38.551162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:19:38.551162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:19:38.551162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:19:38.551162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:19:38.551162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:19:38.551162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:19:38.551162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:38.551162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:19:38.551648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:38.551648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:19:38.551648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:38.551648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:19:38.551648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:38.551648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:19:38.551648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:38.551648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:19:38.551648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:19:38.551648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:38.551648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:19:38.551648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:38.551648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:19:38.551648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:19:38.552186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:38.552186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:19:38.552186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:38.552186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:19:38.552186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:38.552186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:19:38.552186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:19:38.552186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:19:38.552186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:19:38.552186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:19:38.552186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:19:38.552186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:19:38.552186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:19:38.552186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:19:38.552630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:19:38.552630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:19:38.552630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:19:38.552630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:19:38.552630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:19:38.552630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:19:38.552630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:19:38.552630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:19:38.552630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:19:38.552630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:19:38.552630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:19:38.552630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:38.552630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:19:38.552630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:38.552630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:19:38.552630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:38.552630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:19:38.552630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:38.552630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:19:38.553203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:38.553203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:19:38.553203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:19:38.553203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:38.553203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:19:38.553203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:38.553203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:19:38.553203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:19:38.553203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:38.553203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:19:38.553203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:38.553203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:19:38.555121 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:19:38.555417 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:19:38.555517 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:19:38.555669 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:19:38.556162 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:19:38.556451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:38.557061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:19:38.557061 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:38.557061 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:19:38.557061 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:19:38.557061 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:38.557061 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:19:38.557061 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:38.557061 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:19:38.557804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:38.557804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:19:38.557804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:19:38.557804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:19:38.557804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:38.557804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:19:38.557804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:38.557804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:19:38.557804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:38.557804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:19:38.557804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:38.557804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:19:38.557804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:38.557804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:19:38.557804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:19:38.558316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:38.558316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:19:38.558316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:38.558316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:19:38.558316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:19:38.558316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:38.558316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:19:38.558316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:38.558316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:19:38.559479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:38.559479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:19:38.559479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:19:38.559479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:19:38.559479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:19:38.559479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:19:38.559479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:19:38.559479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:19:38.559479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:19:38.559479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:38.559479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:19:38.559913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:38.559913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:19:38.559913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:38.559913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:19:38.559913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:38.559913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:19:38.559913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:38.559913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:19:38.559913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:19:38.559913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:38.559913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:19:38.559913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:38.559913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:19:38.559913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:19:38.560273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:38.560273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:19:38.560273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:38.560273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:19:38.560623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:38.560623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:19:38.560623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:19:38.560623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:19:38.560623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:19:38.560623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:19:38.560623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:19:38.560623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:19:38.560623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:19:38.560623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:19:38.560623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:19:38.560623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:19:38.560623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:19:38.560623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:19:38.560623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:19:38.560623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:19:38.560623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:19:38.561172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:19:38.561172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:19:38.561172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:19:38.561172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:38.561172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:19:38.561172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:38.561172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:19:38.561172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:38.561172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:19:38.561172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:38.561172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:19:38.561172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:38.561172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:19:38.561172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:19:38.561172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:38.561172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:19:38.561172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:38.561172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:19:38.561172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:19:38.561696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:38.561696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:19:38.561696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:38.561696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:19:38.564309 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:19:38.564666 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:19:38.564844 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:19:38.565108 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:19:38.565734 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:19:38.566183 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:19:38.566496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:38.567262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:19:38.567262 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:38.567262 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:19:38.567262 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:19:38.567262 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:38.567262 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:19:38.567262 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:38.567262 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:19:38.568174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:38.568174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:19:38.568174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:19:38.568174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:19:38.568174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:19:38.568174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:19:38.568174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:19:38.568174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:38.568174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:19:38.568174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:38.568174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:19:38.568174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:38.568665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:19:38.568665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:38.568665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:19:38.568665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:38.568665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:19:38.568665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:19:38.568665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:38.568665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:19:38.568665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:38.568665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:19:38.568665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:19:38.568665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:38.568665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:19:38.569151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:38.569151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:19:38.569715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:38.569715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:19:38.569715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:19:38.569715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:19:38.569715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:19:38.569715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:19:38.569715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:19:38.569715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:19:38.569715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:19:38.569715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:19:38.569715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:19:38.569715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:19:38.569715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:19:38.569715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:19:38.569715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:19:38.569715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:19:38.570197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:19:38.570197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:19:38.570197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:38.570197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:19:38.570197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:38.570197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:19:38.570197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:38.570197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:19:38.570197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:38.570197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:19:38.570197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:38.570197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:19:38.570197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:19:38.570197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:38.570197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:19:38.570197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:38.570197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:19:38.570197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:19:38.570197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:38.570837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:19:38.570837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:38.570837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:19:42.983968 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:19:42.983968 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:19:42.983968 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:19:42.983968 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:19:43.015435 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:19:43.015706 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:43.016493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:19:43.016493 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:43.016493 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:19:43.016493 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:19:43.016493 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:43.016493 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:19:43.016493 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:43.016493 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:19:43.017486 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:43.017486 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:19:43.017486 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:19:43.017486 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:19:43.017486 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:19:43.017486 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:19:43.017486 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:19:43.017486 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:43.017486 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:19:43.017486 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:43.017486 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:19:43.017486 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:43.018072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:19:43.018072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:43.018072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:19:43.018072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:43.018072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:19:43.018072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:19:43.018072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:43.018072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:19:43.018072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:43.018072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:19:43.018072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:19:43.018072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:43.018072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:19:43.018757 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:43.018757 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:19:43.018757 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:43.018757 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:19:43.018757 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:19:43.018757 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:19:43.018757 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:19:43.018757 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:19:43.018757 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:19:43.018757 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:19:43.018757 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:19:43.018757 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:19:43.018757 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:19:43.018757 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:19:43.018757 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:19:43.019196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:19:43.019196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:19:43.019196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:19:43.019196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:19:43.019196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:19:43.019196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:43.019196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:19:43.019196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:43.019196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:19:43.019196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:43.019196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:19:43.019196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:43.019196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:19:43.019196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:43.019196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:19:43.019196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:19:43.019196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:43.019196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:19:43.019196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:43.019730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:19:43.019730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:19:43.019730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:43.019730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:19:43.019730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:43.019730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:19:43.537809 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:19:43.538564 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:19:43.538564 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:19:43.539534 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:19:43.539959 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:19:43.540737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:43.541384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:19:43.541384 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:43.541384 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:19:43.541384 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:19:43.541384 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:43.541384 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:19:43.541384 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:43.541384 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:19:43.542612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:43.542612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:19:43.542612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:19:43.542612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:19:43.542612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:19:43.542612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:19:43.542612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:19:43.542612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:43.542612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:19:43.542612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:43.542612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:19:43.542612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:43.543012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:19:43.543012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:43.543012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:19:43.543012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:43.543012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:19:43.543012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:19:43.543012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:43.543012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:19:43.543012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:43.543012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:19:43.543012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:19:43.543012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:43.543012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:19:43.543357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:43.543357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:19:43.543658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:43.543658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:19:43.543658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:19:43.543658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:19:43.543658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:19:43.543658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:19:43.543658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:19:43.543658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:19:43.543658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:19:43.543658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:19:43.543658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:19:43.543658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:19:43.543658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:19:43.543658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:19:43.543658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:19:43.543658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:19:43.544072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:19:43.544072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:19:43.544072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:43.544072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:19:43.544072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:43.544072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:19:43.544072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:43.544072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:19:43.544072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:43.544072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:19:43.544072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:43.544072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:19:43.544072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:19:43.544072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:43.544072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:19:43.544072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:43.544072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:19:43.544072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:19:43.544072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:43.544570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:19:43.544570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:43.544570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:19:43.547947 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:19:43.548527 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:19:43.548527 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:19:43.548527 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:19:43.548976 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:19:43.549208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:43.549764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:19:43.549764 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:43.549764 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:19:43.549764 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:19:43.549764 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:43.549764 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:19:43.549764 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:43.549764 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:19:43.550962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:43.550962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:19:43.550962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:19:43.550962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:19:43.550962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:43.550962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:19:43.550962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:43.550962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:19:43.550962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:43.550962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:19:43.550962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:43.550962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:19:43.550962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:43.550962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:19:43.550962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:19:43.551574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:43.551574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:19:43.551574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:43.551574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:19:43.551574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:19:43.551574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:43.551574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:19:43.551574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:43.551574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:19:43.553728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:43.553728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:19:43.553728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:19:43.553728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:19:43.553728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:19:43.553728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:19:43.553728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:19:43.553728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:19:43.553728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:19:43.553728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:19:43.553728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:43.553728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:19:43.554131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:43.554131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:19:43.554131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:43.554131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:19:43.554131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:43.554131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:19:43.554131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:43.554131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:19:43.554131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:19:43.554131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:43.554131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:19:43.554131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:43.554131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:19:43.554131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:19:43.554699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:43.554699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:19:43.554699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:43.554699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:19:43.555511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:43.555511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:19:43.555511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:19:43.555511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:19:43.555511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:19:43.555511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:19:43.555511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:19:43.555511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:19:43.555511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:19:43.555511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:19:43.555511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:19:43.555511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:19:43.555511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:19:43.555511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:19:43.555511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:19:43.555511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:19:43.556105 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:19:43.556105 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:19:43.556105 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:19:43.556105 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:19:43.556105 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:19:43.556105 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:43.556105 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:19:43.556105 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:43.556105 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:19:43.556105 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:43.556105 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:19:43.556105 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:43.556105 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:19:43.556105 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:43.556105 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:19:43.556105 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:19:43.556105 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:43.556105 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:19:43.557356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:43.557356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:19:43.557356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:19:43.557356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:43.557356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:19:43.557356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:43.557356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:19:43.557356 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:19:43.557356 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:19:43.558185 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:19:43.558185 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:19:43.558561 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:19:43.559066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:43.559502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:19:43.559502 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:43.559502 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:19:43.559502 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:19:43.559502 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:43.559502 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:19:43.559502 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:43.559502 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:19:43.560306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:43.560306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:19:43.560306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:19:43.560306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:19:43.560306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:43.560306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:19:43.560306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:43.560306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:19:43.560306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:43.560306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:19:43.560306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:43.560306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:19:43.560306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:43.560306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:19:43.560306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:19:43.560892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:43.560892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:19:43.560892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:43.560892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:19:43.560892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:19:43.560892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:43.560892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:19:43.560892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:43.560892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:19:43.561249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:43.561249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:19:43.561249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:19:43.561249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:19:43.561249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:19:43.561249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:19:43.561249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:19:43.561249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:19:43.561249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:19:43.561249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:43.561249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:19:43.561605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:43.561605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:19:43.561605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:43.561605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:19:43.561605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:43.561605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:19:43.561605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:43.561605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:19:43.561605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:19:43.561605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:43.561605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:19:43.561605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:43.561605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:19:43.561605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:19:43.562430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:43.562430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:19:43.562430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:43.562430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:19:43.562430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:43.562430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:19:43.562430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:19:43.562430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:19:43.562430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:19:43.562430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:19:43.562430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:19:43.562430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:19:43.562430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:19:43.562430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:19:43.562822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:19:43.562822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:19:43.562822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:19:43.562822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:19:43.562822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:19:43.562822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:19:43.562822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:19:43.562822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:19:43.562822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:19:43.562822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:19:43.562822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:43.562822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:19:43.562822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:43.562822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:19:43.562822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:43.562822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:19:43.562822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:43.562822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:19:43.562822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:43.563536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:19:43.563536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:19:43.563536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:43.563536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:19:43.563536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:43.563536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:19:43.563536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:19:43.563536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:43.563536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:19:43.563536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:43.563536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:19:43.564941 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:19:43.565152 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:19:43.565261 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:19:43.565415 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:19:43.566497 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:19:43.566689 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:19:43.566828 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:43.567258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:19:43.567258 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:43.567258 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:19:43.567258 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:19:43.567258 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:43.567258 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:19:43.567258 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:43.567258 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:19:43.568341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:43.568341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:19:43.568341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:19:43.568341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:19:43.568341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:19:43.568341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:19:43.568341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:19:43.568341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:43.568341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:19:43.568341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:43.568341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:19:43.568341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:43.568683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:19:43.568683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:43.568683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:19:43.568683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:43.568683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:19:43.568683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:19:43.568683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:43.568683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:19:43.568683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:43.568683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:19:43.568683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:19:43.568683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:43.568683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:19:43.568969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:43.568969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:19:43.570812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:43.570812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:19:43.570812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:19:43.570812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:19:43.570812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:19:43.570812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:19:43.570812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:19:43.570812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:19:43.570812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:19:43.570812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:19:43.570812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:19:43.570812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:19:43.570812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:19:43.570812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:19:43.570812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:19:43.570812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:19:43.571570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:19:43.571570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:19:43.571570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:43.571570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:19:43.571570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:43.571570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:19:43.571570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:43.571570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:19:43.571570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:43.571570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:19:43.571570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:43.571570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:19:43.571570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:19:43.571570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:43.571570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:19:43.571570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:43.571570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:19:43.571570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:19:43.571570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:43.572428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:19:43.572428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:43.572428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:19:47.980683 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_for_inconsistencies' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:19:47.981231 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_for_inconsistencies {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:19:47.981231 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Checking Neutron and OVN revision consistency. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:19:47.983547 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:19:47.993528 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Maintenance task: Synchronizing Neutron and OVN databases started {{(pid=106314) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:440}}
May 09 09:19:47.993717 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Maintenance task: Number of inconsistencies found at create/update: networks=4, routers=1, router_ports=1 {{(pid=106314) _log /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:419}}
May 09 09:19:47.993717 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Maintenance task: Fixing resource f8a38fcb-e8a2-421a-a340-bf7f2f5cc72b (type: networks) at create/update {{(pid=106314) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 09 09:19:48.068536 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-f8a38fcb-e8a2-421a-a340-bf7f2f5cc72b, resource={'id': 'f8a38fcb-e8a2-421a-a340-bf7f2f5cc72b', 'name': 'private', 'tenant_id': 'f4d54dfe001d43fbb344093a1ccabd6a', 'project_id': 'f4d54dfe001d43fbb344093a1ccabd6a', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['96e72a03-97b8-412a-b2b9-1bf048ce1066', 'bbd6957a-953f-4fc1-8761-5d93128564d9'], 'standard_attr_id': 12, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-05-09T09:04:34Z', 'updated_at': '2026-05-09T09:04:37Z', 'revision_number': 3, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 3730}, resource_type=networks, if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:19:48.070351 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-f8a38fcb-e8a2-421a-a340-bf7f2f5cc72b, col_values=(('external_ids', {'neutron:network_name': 'private', 'neutron:mtu': '1372', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:19:48.070611 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-78a32c7c-7637-4b83-b0ab-62aee8bb3038, external_ids_update=None, columns={'tag': 3730}, if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:19:48.070713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:48.071453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:19:48.071453 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:48.071453 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:19:48.071453 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:19:48.071453 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:48.071453 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:19:48.071453 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:48.071453 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:19:48.072463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Maintenance task: Failed to fix resource f8a38fcb-e8a2-421a-a340-bf7f2f5cc72b (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:48.072463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:19:48.072463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 09 09:19:48.072463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 09 09:19:48.072463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 09 09:19:48.072463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 09 09:19:48.072463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2332, in update_network
May 09 09:19:48.072463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:19:48.072463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:48.072463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:19:48.072463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:48.072866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:19:48.072866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:48.072866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:19:48.072866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:48.072866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:19:48.072866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:48.072866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:19:48.072866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:19:48.072866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:48.072866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:19:48.072866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:48.072866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:19:48.072866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:19:48.072866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:48.073268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:19:48.073268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:48.073268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:19:48.073268 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Maintenance task: Fixing resource d18a9fac-2ebf-4d1e-a86c-0f33fba8eeda (type: networks) at create/update {{(pid=106314) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 09 09:19:48.104267 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-d18a9fac-2ebf-4d1e-a86c-0f33fba8eeda, resource={'id': 'd18a9fac-2ebf-4d1e-a86c-0f33fba8eeda', 'name': 'public', 'tenant_id': 'f4d54dfe001d43fbb344093a1ccabd6a', 'project_id': 'f4d54dfe001d43fbb344093a1ccabd6a', 'admin_state_up': True, 'mtu': 1430, 'status': 'ACTIVE', 'subnets': ['088bbd0f-7e81-4a76-8399-24d15c984009', 'bbd6a8cc-80e6-4336-a70b-1c962c561590'], 'standard_attr_id': 18, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': True, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'is_default': True, 'tags': [], 'created_at': '2026-05-09T09:04:40Z', 'updated_at': '2026-05-09T09:04:50Z', 'revision_number': 3, 'provider:network_type': 'flat', 'provider:physical_network': 'public', 'provider:segmentation_id': None}, resource_type=networks, if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:19:48.104632 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-d18a9fac-2ebf-4d1e-a86c-0f33fba8eeda, col_values=(('external_ids', {'neutron:network_name': 'public', 'neutron:mtu': '1430', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'flat', 'neutron:provnet-physical-network': 'public'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false', 'fdb_age_threshold': '300', 'broadcast-arps-to-all-routers': 'true'})), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:19:48.104723 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-1163f525-70cd-4b84-ae9b-7d56378305e9, external_ids_update=None, columns={'tag': []}, if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:19:48.104972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:48.105516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:19:48.105516 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:48.105516 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:19:48.105516 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:19:48.105516 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:48.105516 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:19:48.105516 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:48.105516 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:19:48.106487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Maintenance task: Failed to fix resource d18a9fac-2ebf-4d1e-a86c-0f33fba8eeda (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:48.106487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:19:48.106487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 09 09:19:48.106487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 09 09:19:48.106487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 09 09:19:48.106487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 09 09:19:48.106487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2332, in update_network
May 09 09:19:48.106487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:19:48.106487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:48.106487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:19:48.106487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:48.106795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:19:48.106795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:48.106795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:19:48.106795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:48.106795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:19:48.106795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:48.106795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:19:48.106795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:19:48.106795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:48.106795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:19:48.106795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:48.106795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:19:48.106795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:19:48.106795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:48.107077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:19:48.107077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:48.107077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:19:48.107077 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Maintenance task: Fixing resource 417da935-2882-452f-8ba7-9076d68fcd2b (type: networks) at create/update {{(pid=106314) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 09 09:19:48.134582 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-417da935-2882-452f-8ba7-9076d68fcd2b, resource={'id': '417da935-2882-452f-8ba7-9076d68fcd2b', 'name': 'admin_net', 'tenant_id': '38e536cfc9c94b6cbf529caf55cc7739', 'project_id': '38e536cfc9c94b6cbf529caf55cc7739', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['d3c53d14-f490-4615-a986-3715a315f827'], 'standard_attr_id': 42, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-05-09T09:10:38Z', 'updated_at': '2026-05-09T09:10:39Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 32823}, resource_type=networks, if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:19:48.134895 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-417da935-2882-452f-8ba7-9076d68fcd2b, col_values=(('external_ids', {'neutron:network_name': 'admin_net', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:19:48.135275 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-5738ab2a-b471-447b-9292-7f8c6bc97168, external_ids_update=None, columns={'tag': 32823}, if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:19:48.135404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:48.136007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:19:48.136007 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:48.136007 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:19:48.136007 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:19:48.136007 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:48.136007 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:19:48.136007 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:48.136007 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:19:48.138031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Maintenance task: Failed to fix resource 417da935-2882-452f-8ba7-9076d68fcd2b (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:48.138031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:19:48.138031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 09 09:19:48.138031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 09 09:19:48.138031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 09 09:19:48.138031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 09 09:19:48.138031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2332, in update_network
May 09 09:19:48.138031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:19:48.138031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:48.138031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:19:48.138031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:48.138553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:19:48.138553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:48.138553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:19:48.138553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:48.138553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:19:48.138553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:48.138553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:19:48.138553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:19:48.138553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:48.138553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:19:48.138553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:48.138553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:19:48.138553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:19:48.138553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:48.138948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:19:48.138948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:48.138948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:19:48.138948 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Maintenance task: Fixing resource 433617e8-cdf8-414a-8cc6-63a7b5d0c678 (type: networks) at create/update {{(pid=106314) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 09 09:19:48.161737 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-433617e8-cdf8-414a-8cc6-63a7b5d0c678, resource={'id': '433617e8-cdf8-414a-8cc6-63a7b5d0c678', 'name': 'k8s-clusterapi-cluster-magnum-system-kube-whtb8', 'tenant_id': '3e013d45a44544c3b50eb0e83e4b1efc', 'project_id': '3e013d45a44544c3b50eb0e83e4b1efc', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['9187f065-9264-427a-8df8-672a6981191d'], 'standard_attr_id': 52, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-05-09T09:11:20Z', 'updated_at': '2026-05-09T09:11:20Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 32644}, resource_type=networks, if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:19:48.161957 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-433617e8-cdf8-414a-8cc6-63a7b5d0c678, col_values=(('external_ids', {'neutron:network_name': 'k8s-clusterapi-cluster-magnum-system-kube-whtb8', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:19:48.162347 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-8c3fe586-c182-47ba-9845-d7608113cb71, external_ids_update=None, columns={'tag': 32644}, if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:19:48.162444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:48.163555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:19:48.163555 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:48.163555 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:19:48.163555 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:19:48.163555 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:48.163555 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:19:48.163555 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:48.163555 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:19:48.163939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Maintenance task: Failed to fix resource 433617e8-cdf8-414a-8cc6-63a7b5d0c678 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:48.163939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:19:48.163939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 09 09:19:48.163939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 09 09:19:48.163939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 09 09:19:48.163939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 09 09:19:48.163939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2332, in update_network
May 09 09:19:48.163939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:19:48.163939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:48.163939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:19:48.163939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:48.164257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:19:48.164257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:48.164257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:19:48.164257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:48.164257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:19:48.164257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:48.164257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:19:48.164257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:19:48.164257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:48.164257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:19:48.164257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:48.164257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:19:48.164257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:19:48.164257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:48.164598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:19:48.164598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:48.164598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:19:48.164598 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Maintenance task: Fixing resource cab10384-14c5-492c-8bd0-061f5838310c (type: routers) at create/update {{(pid=106314) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 09 09:19:48.247008 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-cab10384-14c5-492c-8bd0-061f5838310c, resource={'id': 'cab10384-14c5-492c-8bd0-061f5838310c', 'name': 'k8s-clusterapi-cluster-magnum-system-kube-whtb8', 'project_id': '3e013d45a44544c3b50eb0e83e4b1efc', 'admin_state_up': True, 'status': 'ACTIVE', 'enable_snat': True, 'external_gateway_info': {'network_id': 'd18a9fac-2ebf-4d1e-a86c-0f33fba8eeda', 'external_fixed_ips': [{'subnet_id': 'bbd6a8cc-80e6-4336-a70b-1c962c561590', 'ip_address': '172.24.5.116'}, {'subnet_id': '088bbd0f-7e81-4a76-8399-24d15c984009', 'ip_address': '2001:db8::1b7'}], 'enable_snat': True}, 'gw_port_id': 'af118f11-dc01-429a-b20a-e97b77b8d5ec', 'description': 'Created by cluster-api-provider-openstack cluster magnum-system-kube-whtb8', 'availability_zones': [], 'distributed': False, 'ha': True, 'ha_vr_id': 0, 'availability_zone_hints': [], 'enable_default_route_ecmp': False, 'enable_default_route_bfd': False, 'external_gateways': [{'network_id': 'd18a9fac-2ebf-4d1e-a86c-0f33fba8eeda', 'external_fixed_ips': [{'ip_address': '172.24.5.116', 'subnet_id': 'bbd6a8cc-80e6-4336-a70b-1c962c561590'}, {'ip_address': '2001:db8::1b7', 'subnet_id': '088bbd0f-7e81-4a76-8399-24d15c984009'}]}], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2026-05-09T09:11:21Z', 'updated_at': '2026-05-09T09:11:24Z', 'revision_number': 3, 'tenant_id': '3e013d45a44544c3b50eb0e83e4b1efc'}, resource_type=routers, if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:19:48.247231 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterCommand(_result=None, name=neutron-cab10384-14c5-492c-8bd0-061f5838310c, columns={'external_ids': {'neutron:router_name': 'k8s-clusterapi-cluster-magnum-system-kube-whtb8', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': ''}, 'enabled': True}, if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:19:48.247501 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): QoSDelCommand(_result=None, switch=neutron-d18a9fac-2ebf-4d1e-a86c-0f33fba8eeda, conditions=[('direction', '=', 'from-lport'), ('priority', '=', 2002), ('match', '=', 'inport == "af118f11-dc01-429a-b20a-e97b77b8d5ec"')], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:19:48.247687 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): QoSDelCommand(_result=None, switch=neutron-d18a9fac-2ebf-4d1e-a86c-0f33fba8eeda, conditions=[('direction', '=', 'to-lport'), ('priority', '=', 2002), ('match', '=', 'outport == "af118f11-dc01-429a-b20a-e97b77b8d5ec"')], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:19:48.247877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:48.248877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:19:48.248877 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:48.248877 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:19:48.248877 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:19:48.248877 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:48.248877 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:19:48.248877 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:48.248877 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:19:48.249117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_client [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Unable to update router cab10384-14c5-492c-8bd0-061f5838310c. Error: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:48.252829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Maintenance task: Failed to fix resource cab10384-14c5-492c-8bd0-061f5838310c (type: routers): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:48.252829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:19:48.252829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 09 09:19:48.252829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 09 09:19:48.252829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 09 09:19:48.252829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 09 09:19:48.252829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1635, in update_router
May 09 09:19:48.252829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with excutils.save_and_reraise_exception():
May 09 09:19:48.252829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 271, in __exit__
May 09 09:19:48.252829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.force_reraise()
May 09 09:19:48.252829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 233, in force_reraise
May 09 09:19:48.253453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise self.value
May 09 09:19:48.253453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1569, in update_router
May 09 09:19:48.253453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:19:48.253453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:48.253453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:19:48.253453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:48.253453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:19:48.253453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:48.253453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:19:48.253453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:48.253453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:19:48.253453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:48.253453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:19:48.253453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:19:48.254109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:48.254109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:19:48.254109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:48.254109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:19:48.254109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:19:48.254109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:48.254109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:19:48.254109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:48.254109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:19:48.254109 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Maintenance task: Fixing resource af118f11-dc01-429a-b20a-e97b77b8d5ec (type: router_ports) at create/update {{(pid=106314) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 09 09:19:48.370243 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=lrp-af118f11-dc01-429a-b20a-e97b77b8d5ec, resource={'id': 'af118f11-dc01-429a-b20a-e97b77b8d5ec', 'name': '', 'network_id': 'd18a9fac-2ebf-4d1e-a86c-0f33fba8eeda', 'project_id': '', 'mac_address': 'fa:16:3e:51:04:94', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'cab10384-14c5-492c-8bd0-061f5838310c', 'device_owner': 'network:router_gateway', 'standard_attr_id': 57, 'fixed_ips': [{'subnet_id': 'bbd6a8cc-80e6-4336-a70b-1c962c561590', 'ip_address': '172.24.5.116'}, {'subnet_id': '088bbd0f-7e81-4a76-8399-24d15c984009', 'ip_address': '2001:db8::1b7'}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'np0000173033', 'binding:vif_type': 'ovs', 'binding:vif_details': {'port_filter': True, 'connectivity': 'l2', 'ovs_create_tap': False, 'bridge_name': 'br-int', 'datapath_type': 'system', 'bound_drivers': {'0': 'ovn'}}, 'port_security_enabled': False, 'tags': [], 'created_at': '2026-05-09T09:11:21Z', 'updated_at': '2026-05-09T09:11:24Z', 'revision_number': 5, 'tenant_id': ''}, resource_type=router_ports, if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:19:48.370470 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterPortCommand(_result=None, name=lrp-af118f11-dc01-429a-b20a-e97b77b8d5ec, columns={'external_ids': {'neutron:revision_number': '5', 'neutron:subnet_ids': 'bbd6a8cc-80e6-4336-a70b-1c962c561590 088bbd0f-7e81-4a76-8399-24d15c984009', 'neutron:network_name': 'neutron-d18a9fac-2ebf-4d1e-a86c-0f33fba8eeda', 'neutron:is_ext_gw': 'True', 'neutron:router_name': 'neutron-cab10384-14c5-492c-8bd0-061f5838310c'}, 'options': {'reside-on-redirect-chassis': 'true', 'gateway_mtu': '1372'}, 'networks': ['172.24.5.116/24', '2001:db8::1b7/64'], 'ipv6_ra_configs': {}}, if_exists=False) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:19:48.370975 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLRouterPortInLSwitchPortCommand(_result=None, lswitch_port=af118f11-dc01-429a-b20a-e97b77b8d5ec, lrouter_port=lrp-af118f11-dc01-429a-b20a-e97b77b8d5ec, is_gw_port=True, if_exists=True, lsp_address=router) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:19:48.371313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:48.372018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:19:48.372018 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:48.372018 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:19:48.372018 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:19:48.372018 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:48.372018 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:19:48.372018 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:48.372018 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:19:48.373344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Maintenance task: Failed to fix resource af118f11-dc01-429a-b20a-e97b77b8d5ec (type: router_ports): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:48.373344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:19:48.373344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 09 09:19:48.373344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 09 09:19:48.373344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 09 09:19:48.373344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 09 09:19:48.373344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1976, in update_router_port
May 09 09:19:48.373344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:19:48.373344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:48.373344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:19:48.373344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:48.373658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:19:48.373658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:48.373658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:19:48.373658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:48.373658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:19:48.373658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:48.373658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:19:48.373658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:19:48.373658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:48.373658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:19:48.373658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:48.373658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:19:48.373658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:19:48.373658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:48.373949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:19:48.373949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:48.373949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:19:48.373949 np0000173033 neutron-ovn-maintenance-worker[106314]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_for_inconsistencies finished in 0.392 seconds
May 09 09:19:48.373949 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:19:48.373949 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:19:48.374838 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:19:48.409359 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:19:48.409566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:48.410271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:19:48.410271 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:48.410271 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:19:48.410271 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:19:48.410271 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:48.410271 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:19:48.410271 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:48.410271 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:19:48.411484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:48.411484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:19:48.411484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:19:48.411484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:19:48.411484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:19:48.411484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:19:48.411484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:19:48.411484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:48.411484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:19:48.411484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:48.411484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:19:48.411484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:48.411910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:19:48.411910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:48.411910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:19:48.411910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:48.411910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:19:48.411910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:19:48.411910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:48.411910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:19:48.411910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:48.411910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:19:48.411910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:19:48.411910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:48.411910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:19:48.412208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:48.412208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:19:48.412987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:48.412987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:19:48.412987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:19:48.412987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:19:48.412987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:19:48.412987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:19:48.412987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:19:48.412987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:19:48.412987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:19:48.412987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:19:48.412987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:19:48.412987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:19:48.412987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:19:48.412987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:19:48.412987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:19:48.412987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:19:48.413498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:19:48.413498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:19:48.413498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:48.413498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:19:48.413498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:48.413498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:19:48.413498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:48.413498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:19:48.413498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:48.413498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:19:48.413498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:48.413498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:19:48.413498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:19:48.413498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:48.413498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:19:48.413498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:48.413498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:19:48.413498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:19:48.413498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:48.414105 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:19:48.414105 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:48.414105 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:19:48.538841 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:19:48.539160 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:19:48.539432 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:19:48.540713 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:19:48.541218 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:19:48.541607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:48.542667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:19:48.542667 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:48.542667 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:19:48.542667 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:19:48.542667 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:48.542667 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:19:48.542667 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:48.542667 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:19:48.543723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:48.543723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:19:48.543723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:19:48.543723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:19:48.543723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:19:48.543723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:19:48.543723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:19:48.543723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:48.543723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:19:48.543723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:48.543723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:19:48.543723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:48.544125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:19:48.544125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:48.544125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:19:48.544125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:48.544125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:19:48.544125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:19:48.544125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:48.544125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:19:48.544125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:48.544125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:19:48.544125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:19:48.544125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:48.544125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:19:48.544485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:48.544485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:19:48.545030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:48.545030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:19:48.545030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:19:48.545030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:19:48.545030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:19:48.545030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:19:48.545030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:19:48.545030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:19:48.545030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:19:48.545030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:19:48.545030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:19:48.545030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:19:48.545030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:19:48.545030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:19:48.545030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:19:48.545030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:19:48.545502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:19:48.545502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:19:48.545502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:48.545502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:19:48.545502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:48.545502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:19:48.545502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:48.545502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:19:48.545502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:48.545502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:19:48.545502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:48.545502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:19:48.545502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:19:48.545502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:48.545502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:19:48.545502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:48.545502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:19:48.545502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:19:48.545502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:48.546013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:19:48.546013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:48.546013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:19:48.548405 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:19:48.548732 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:19:48.548892 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:19:48.549015 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:19:48.549387 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:19:48.549664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:48.550261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:19:48.550261 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:48.550261 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:19:48.550261 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:19:48.550261 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:48.550261 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:19:48.550261 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:48.550261 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:19:48.551281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:48.551281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:19:48.551281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:19:48.551281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:19:48.551281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:48.551281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:19:48.551281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:48.551281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:19:48.551281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:48.551281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:19:48.551281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:48.551281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:19:48.551281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:48.551281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:19:48.551281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:19:48.551988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:48.551988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:19:48.551988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:48.551988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:19:48.551988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:19:48.551988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:48.551988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:19:48.551988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:48.551988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:19:48.552723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:48.552723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:19:48.552723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:19:48.552723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:19:48.552723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:19:48.552723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:19:48.552723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:19:48.552723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:19:48.552723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:19:48.552723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:19:48.552723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:48.552723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:19:48.553266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:48.553266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:19:48.553266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:48.553266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:19:48.553266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:48.553266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:19:48.553266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:48.553266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:19:48.553266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:19:48.553266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:48.553266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:19:48.553266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:48.553266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:19:48.553266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:19:48.553842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:48.553842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:19:48.553842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:48.553842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:19:48.553842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:48.553842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:19:48.553842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:19:48.553842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:19:48.553842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:19:48.553842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:19:48.553842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:19:48.553842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:19:48.553842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:19:48.553842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:19:48.554409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:19:48.554409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:19:48.554409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:19:48.554409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:19:48.554409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:19:48.554409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:19:48.554409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:19:48.554409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:19:48.554409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:19:48.554409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:19:48.554409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:19:48.554409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:48.554409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:19:48.554409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:48.554409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:19:48.554409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:48.554409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:19:48.554409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:48.554409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:19:48.555499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:48.555499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:19:48.555499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:19:48.555499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:48.555499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:19:48.555499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:48.555499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:19:48.555499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:19:48.555499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:48.555499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:19:48.555499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:48.555499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:19:48.556511 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:19:48.556931 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:19:48.557094 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:19:48.557280 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:19:48.557762 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:19:48.558366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:48.559002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:19:48.559002 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:48.559002 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:19:48.559002 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:19:48.559002 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:48.559002 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:19:48.559002 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:48.559002 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:19:48.559829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:48.559829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:19:48.559829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:19:48.559829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:19:48.559829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:48.559829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:19:48.559829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:48.559829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:19:48.559829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:48.559829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:19:48.559829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:48.559829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:19:48.559829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:48.559829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:19:48.559829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:19:48.560342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:48.560342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:19:48.560342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:48.560342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:19:48.560342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:19:48.560342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:48.560342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:19:48.560342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:48.560342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:19:48.560792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:48.560792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:19:48.560792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:19:48.560792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:19:48.560792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:19:48.560792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:19:48.560792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:19:48.560792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:19:48.560792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:19:48.560792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:48.560792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:19:48.561367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:48.561367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:19:48.561367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:48.561367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:19:48.561367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:48.561367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:19:48.561367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:48.561367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:19:48.561367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:19:48.561367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:48.561367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:19:48.561367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:48.561367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:19:48.561367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:19:48.561824 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:48.561824 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:19:48.561824 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:48.561824 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:19:48.562068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:48.562068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:19:48.562068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:19:48.562068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:19:48.562068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:19:48.562068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:19:48.562068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:19:48.562068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:19:48.562068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:19:48.562068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:19:48.562068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:19:48.562068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:19:48.562068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:19:48.562068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:19:48.562068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:19:48.562068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:19:48.562068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:19:48.562854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:19:48.562854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:19:48.562854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:19:48.562854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:48.562854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:19:48.562854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:48.562854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:19:48.562854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:48.562854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:19:48.562854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:48.562854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:19:48.562854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:48.562854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:19:48.562854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:19:48.562854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:48.562854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:19:48.562854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:48.562854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:19:48.562854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:19:48.563441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:48.563441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:19:48.563441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:48.563441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:19:48.565439 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:19:48.565760 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:19:48.565870 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:19:48.566001 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:19:48.566343 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:19:48.566606 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:19:48.566757 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:48.567144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:19:48.567144 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:48.567144 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:19:48.567144 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:19:48.567144 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:48.567144 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:19:48.567144 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:48.567144 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:19:48.567790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:48.567790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:19:48.567790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:19:48.567790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:19:48.567790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:19:48.567790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:19:48.567790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:19:48.567790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:48.567790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:19:48.567790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:48.567790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:19:48.567790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:48.568119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:19:48.568119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:48.568119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:19:48.568119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:48.568119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:19:48.568119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:19:48.568119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:48.568119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:19:48.568119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:48.568119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:19:48.568119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:19:48.568119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:48.568119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:19:48.568517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:48.568517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:19:48.568517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:48.568517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:19:48.568517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:19:48.568517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:19:48.568517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:19:48.568517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:19:48.568517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:19:48.568517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:19:48.568517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:19:48.568517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:19:48.568517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:19:48.568517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:19:48.568517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:19:48.568908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:19:48.568908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:19:48.568908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:19:48.568908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:19:48.568908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:19:48.568908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:48.568908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:19:48.568908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:48.568908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:19:48.568908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:48.568908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:19:48.568908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:48.568908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:19:48.568908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:48.568908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:19:48.568908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:19:48.568908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:48.568908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:19:48.568908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:48.569301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:19:48.569301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:19:48.569301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:48.569301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:19:48.569301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:48.569301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:19:53.374075 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:19:53.374861 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:19:53.374932 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:19:53.376333 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:19:53.378490 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG dbcounter [-] [106314] Writing DB stats neutron:SELECT=433 {{(pid=106314) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 09 09:19:53.407523 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:19:53.407723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:53.408181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:19:53.408181 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:53.408181 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:19:53.408181 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:19:53.408181 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:53.408181 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:19:53.408181 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:53.408181 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:19:53.409373 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.033 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:53.409373 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:19:53.409373 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:19:53.409373 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:19:53.409373 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:19:53.409373 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:19:53.409373 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:19:53.409373 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:53.409373 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:19:53.409373 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:53.409373 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:19:53.409373 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:53.410039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:19:53.410039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:53.410039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:19:53.410039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:53.410039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:19:53.410039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:19:53.410039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:53.410039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:19:53.410039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:53.410039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:19:53.410039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:19:53.410039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:53.410039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:19:53.410545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:53.410545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:19:53.410545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:53.410545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:19:53.410545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:19:53.410545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:19:53.410545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:19:53.410545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:19:53.410545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:19:53.410545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:19:53.410545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:19:53.410545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:19:53.410545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:19:53.410545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:19:53.410545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:19:53.411273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:19:53.411273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:19:53.411273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:19:53.411273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:19:53.411273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:19:53.411273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:53.411273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:19:53.411273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:53.411273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:19:53.411273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:53.411273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:19:53.411273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:53.411273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:19:53.411273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:53.411273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:19:53.411273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:19:53.411273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:53.411273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:19:53.411273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:53.412272 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:19:53.412272 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:19:53.412272 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:53.412272 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:19:53.412272 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:53.412272 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:19:53.539062 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:19:53.539611 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:19:53.539808 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:19:53.540842 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:19:53.541009 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:19:53.541154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:53.541952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:19:53.541952 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:53.541952 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:19:53.541952 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:19:53.541952 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:53.541952 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:19:53.541952 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:53.541952 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:19:53.543389 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:53.543389 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:19:53.543389 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:19:53.543389 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:19:53.543389 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:19:53.543389 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:19:53.543389 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:19:53.543389 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:53.543389 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:19:53.543389 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:53.543389 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:19:53.543389 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:53.543974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:19:53.543974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:53.543974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:19:53.543974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:53.543974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:19:53.543974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:19:53.543974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:53.543974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:19:53.543974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:53.543974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:19:53.543974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:19:53.543974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:53.543974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:19:53.544468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:53.544468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:19:53.544468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:53.544468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:19:53.544468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:19:53.544468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:19:53.544468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:19:53.544468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:19:53.544468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:19:53.544468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:19:53.544468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:19:53.544468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:19:53.544468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:19:53.544468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:19:53.544468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:19:53.545095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:19:53.545095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:19:53.545095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:19:53.545095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:19:53.545095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:19:53.545095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:53.545095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:19:53.545095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:53.545095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:19:53.545095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:53.545095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:19:53.545095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:53.545095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:19:53.545095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:53.545095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:19:53.545095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:19:53.545095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:53.545095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:19:53.545095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:53.545629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:19:53.545629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:19:53.545629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:53.545629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:19:53.545629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:53.545629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:19:53.548786 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:19:53.549159 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:19:53.549319 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:19:53.549400 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:19:53.549804 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:19:53.550109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:53.550923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:19:53.550923 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:53.550923 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:19:53.550923 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:19:53.550923 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:53.550923 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:19:53.550923 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:53.550923 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:19:53.552189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:53.552189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:19:53.552189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:19:53.552189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:19:53.552189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:53.552189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:19:53.552189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:53.552189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:19:53.552189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:53.552189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:19:53.552189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:53.552189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:19:53.552189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:53.552189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:19:53.552189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:19:53.552779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:53.552779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:19:53.552779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:53.552779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:19:53.552779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:19:53.552779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:53.552779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:19:53.552779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:53.552779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:19:53.553764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:53.553764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:19:53.553764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:19:53.553764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:19:53.553764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:19:53.553764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:19:53.553764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:19:53.553764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:19:53.553764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:19:53.553764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:19:53.553764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:53.553764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:19:53.554139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:53.554139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:19:53.554139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:53.554139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:19:53.554139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:53.554139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:19:53.554139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:53.554139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:19:53.554139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:19:53.554139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:53.554139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:19:53.554139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:53.554139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:19:53.554139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:19:53.554517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:53.554517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:19:53.554517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:53.554517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:19:53.555981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:53.555981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:19:53.555981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:19:53.555981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:19:53.555981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:19:53.555981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:19:53.555981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:19:53.555981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:19:53.555981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:19:53.555981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:19:53.555981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:19:53.555981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:19:53.555981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:19:53.555981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:19:53.555981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:19:53.555981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:19:53.556577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:19:53.556577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:19:53.556577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:19:53.556577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:19:53.556577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:19:53.556577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:53.556577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:19:53.556577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:53.556577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:19:53.556577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:53.556577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:19:53.556577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:53.556577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:19:53.556577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:53.556577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:19:53.556577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:19:53.556577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:53.556577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:19:53.557207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:53.557207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:19:53.557207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:19:53.557207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:53.557207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:19:53.557207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:53.557207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:19:53.557207 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:19:53.557525 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:19:53.557737 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:19:53.557855 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:19:53.558173 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:19:53.558369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:53.558720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:19:53.558720 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:53.558720 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:19:53.558720 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:19:53.558720 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:53.558720 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:19:53.558720 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:53.558720 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:19:53.559315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:53.559315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:19:53.559315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:19:53.559315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:19:53.559315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:53.559315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:19:53.559315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:53.559315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:19:53.559315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:53.559315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:19:53.559315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:53.559315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:19:53.559315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:53.559315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:19:53.559315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:19:53.559915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:53.559915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:19:53.559915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:53.559915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:19:53.559915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:19:53.559915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:53.559915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:19:53.559915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:53.559915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:19:53.560281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:53.560281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:19:53.560281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:19:53.560281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:19:53.560281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:19:53.560281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:19:53.560281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:19:53.560281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:19:53.560281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:19:53.560281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:53.560281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:19:53.560736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:53.560736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:19:53.560736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:53.560736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:19:53.560736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:53.560736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:19:53.560736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:53.560736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:19:53.560736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:19:53.560736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:53.560736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:19:53.560736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:53.560736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:19:53.560736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:19:53.561272 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:53.561272 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:19:53.561272 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:53.561272 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:19:53.561272 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:53.561272 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:19:53.561272 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:19:53.561272 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:19:53.561272 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:19:53.561272 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:19:53.561272 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:19:53.561272 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:19:53.561272 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:19:53.561272 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:19:53.561807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:19:53.561807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:19:53.561807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:19:53.561807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:19:53.561807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:19:53.561807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:19:53.561807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:19:53.561807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:19:53.561807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:19:53.561807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:19:53.561807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:53.561807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:19:53.561807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:53.561807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:19:53.561807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:53.561807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:19:53.561807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:53.561807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:19:53.561807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:53.562497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:19:53.562497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:19:53.562497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:53.562497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:19:53.562497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:53.562497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:19:53.562497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:19:53.562497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:53.562497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:19:53.562497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:53.562497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:19:53.565970 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:19:53.566248 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:19:53.566377 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:19:53.566568 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:19:53.566882 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:19:53.567119 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:19:53.567271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:53.567757 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:19:53.567757 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:53.567757 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:19:53.567757 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:19:53.567757 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:53.567757 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:19:53.567757 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:53.567757 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:19:53.568639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:53.568639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:19:53.568639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:19:53.568639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:19:53.568639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:19:53.568639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:19:53.568639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:19:53.568639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:53.568639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:19:53.568639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:53.568639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:19:53.568639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:53.569025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:19:53.569025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:53.569025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:19:53.569025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:53.569025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:19:53.569025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:19:53.569025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:53.569025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:19:53.569025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:53.569025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:19:53.569025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:19:53.569025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:53.569025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:19:53.569389 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:53.569389 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:19:53.569604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:53.569604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:19:53.569604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:19:53.569604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:19:53.569604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:19:53.569604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:19:53.569604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:19:53.569604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:19:53.569604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:19:53.569604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:19:53.569604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:19:53.569604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:19:53.569604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:19:53.569604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:19:53.569604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:19:53.569604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:19:53.570084 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:19:53.570084 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:19:53.570084 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:53.570084 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:19:53.570084 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:53.570084 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:19:53.570084 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:53.570084 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:19:53.570084 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:53.570084 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:19:53.570084 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:53.570084 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:19:53.570084 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:19:53.570084 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:53.570084 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:19:53.570084 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:53.570084 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:19:53.570084 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:19:53.570084 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:53.570603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:19:53.570603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:53.570603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:19:58.374846 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:19:58.375938 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:19:58.375938 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:19:58.376935 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:19:58.415209 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:19:58.423242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:58.423242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:19:58.423242 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:58.423242 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:19:58.423242 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:19:58.423242 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:58.423242 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:19:58.423242 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:58.423242 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:19:58.423242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:58.423242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:19:58.423242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:19:58.423242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:19:58.423242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:19:58.423242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:19:58.425416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:19:58.425416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:58.425416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:19:58.425416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:58.425416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:19:58.425416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:58.425416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:19:58.425416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:58.425416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:19:58.425416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:58.425416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:19:58.425416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:19:58.425416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:58.425416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:19:58.426171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:58.426171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:19:58.426171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:19:58.426171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:58.426171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:19:58.426171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:58.426171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:19:58.426171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:58.426171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:19:58.426171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:19:58.426171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:19:58.426171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:19:58.426171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:19:58.428130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:19:58.428130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:19:58.428130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:19:58.428130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:19:58.428130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:19:58.428130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:19:58.428130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:19:58.428130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:19:58.428130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:19:58.428130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:19:58.428130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:19:58.428130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:19:58.428130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:58.428130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:19:58.428130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:58.428130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:19:58.428130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:58.428130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:19:58.428130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:58.428130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:19:58.429105 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:58.429105 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:19:58.429105 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:19:58.429105 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:58.429105 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:19:58.429105 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:58.429105 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:19:58.429105 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:19:58.429105 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:58.429105 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:19:58.429105 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:58.429105 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:19:58.539654 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:19:58.540289 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:19:58.540489 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:19:58.541944 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:19:58.542140 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:19:58.542314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:58.543215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:19:58.543215 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:58.543215 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:19:58.543215 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:19:58.543215 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:58.543215 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:19:58.543215 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:58.543215 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:19:58.544867 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:58.544867 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:19:58.544867 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:19:58.544867 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:19:58.544867 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:19:58.544867 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:19:58.544867 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:19:58.544867 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:58.544867 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:19:58.544867 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:58.544867 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:19:58.544867 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:58.545248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:19:58.545248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:58.545248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:19:58.545248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:58.545248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:19:58.545248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:19:58.545248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:58.545248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:19:58.545248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:58.545248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:19:58.545248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:19:58.545248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:58.545248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:19:58.545581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:58.545581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:19:58.547049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:58.547049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:19:58.547049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:19:58.547049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:19:58.547049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:19:58.547049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:19:58.547049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:19:58.547049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:19:58.547049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:19:58.547049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:19:58.547049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:19:58.547049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:19:58.547049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:19:58.547049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:19:58.547049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:19:58.547049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:19:58.547641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:19:58.547641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:19:58.547641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:58.547641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:19:58.547641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:58.547641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:19:58.547641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:58.547641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:19:58.547641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:58.547641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:19:58.547641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:58.547641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:19:58.547641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:19:58.547641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:58.547641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:19:58.547641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:58.547641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:19:58.547641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:19:58.547641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:58.548170 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:19:58.548170 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:58.548170 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:19:58.549318 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:19:58.549733 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:19:58.549974 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:19:58.550207 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:19:58.550597 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:19:58.550817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:58.551401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:19:58.551401 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:58.551401 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:19:58.551401 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:19:58.551401 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:58.551401 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:19:58.551401 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:58.551401 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:19:58.552364 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:58.552364 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:19:58.552364 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:19:58.552364 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:19:58.552364 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:58.552364 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:19:58.552364 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:58.552364 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:19:58.552364 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:58.552364 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:19:58.552364 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:58.552364 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:19:58.552364 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:58.552364 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:19:58.552364 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:19:58.552841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:58.552841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:19:58.552841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:58.552841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:19:58.552841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:19:58.552841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:58.552841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:19:58.552841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:58.552841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:19:58.553268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:58.553268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:19:58.553268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:19:58.553268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:19:58.553268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:19:58.553268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:19:58.553268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:19:58.553268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:19:58.553268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:19:58.553268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:19:58.553268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:58.553268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:19:58.553905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:58.553905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:19:58.553905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:58.553905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:19:58.553905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:58.553905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:19:58.553905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:58.553905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:19:58.553905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:19:58.553905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:58.553905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:19:58.553905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:58.553905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:19:58.553905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:19:58.554302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:58.554302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:19:58.554302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:58.554302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:19:58.554302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:58.554302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:19:58.554302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:19:58.554302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:19:58.554302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:19:58.554302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:19:58.554302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:19:58.554302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:19:58.554302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:19:58.554302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:19:58.554805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:19:58.554805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:19:58.554805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:19:58.554805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:19:58.554805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:19:58.554805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:19:58.554805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:19:58.554805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:19:58.554805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:19:58.554805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:19:58.554805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:19:58.554805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:58.554805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:19:58.554805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:58.554805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:19:58.554805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:58.554805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:19:58.554805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:58.554805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:19:58.555316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:58.555316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:19:58.555316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:19:58.555316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:58.555316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:19:58.555316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:58.555316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:19:58.555316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:19:58.555316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:58.555316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:19:58.555316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:58.555316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:19:58.557589 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:19:58.558107 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:19:58.558368 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:19:58.558677 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:19:58.560844 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:19:58.560844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:58.561118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:19:58.561118 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:58.561118 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:19:58.561118 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:19:58.561118 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:58.561118 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:19:58.561118 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:58.561118 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:19:58.562539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:58.562539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:19:58.562539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:19:58.562539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:19:58.562539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:58.562539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:19:58.562539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:58.562539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:19:58.562539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:58.562539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:19:58.562539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:58.562539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:19:58.562539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:58.562539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:19:58.562539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:19:58.562972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:58.562972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:19:58.562972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:58.562972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:19:58.562972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:19:58.562972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:58.562972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:19:58.562972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:58.562972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:19:58.564596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:58.564596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:19:58.564596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:19:58.564596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:19:58.564596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:19:58.564596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:19:58.564596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:19:58.564596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:19:58.564596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:19:58.564596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:58.564596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:19:58.564972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:58.564972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:19:58.564972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:58.564972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:19:58.564972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:58.564972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:19:58.564972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:58.564972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:19:58.564972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:19:58.564972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:58.564972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:19:58.564972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:58.564972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:19:58.564972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:19:58.565354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:58.565354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:19:58.565354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:58.565354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:19:58.566778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:58.566778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:19:58.566778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:19:58.566778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:19:58.566778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:19:58.566778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:19:58.566778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:19:58.566778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:19:58.566778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:19:58.566778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:19:58.566778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:19:58.566778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:19:58.566778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:19:58.566778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:19:58.566778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:19:58.566778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:19:58.566778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:19:58.567299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:19:58.567299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:19:58.567299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:19:58.567299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:58.567299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:19:58.567299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:58.567299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:19:58.567299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:58.567299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:19:58.567299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:58.567299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:19:58.567299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:58.567299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:19:58.567299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:19:58.567299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:58.567299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:19:58.567299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:58.567299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:19:58.567299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:19:58.567873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:58.567873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:19:58.567873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:58.567873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:19:58.567873 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:19:58.567873 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:19:58.567873 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:19:58.567873 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:19:58.569458 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:19:58.569458 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:19:58.569458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:58.569689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:19:58.569689 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:58.569689 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:19:58.569689 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:19:58.569689 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:58.569689 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:19:58.569689 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:58.569689 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:19:58.570853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:58.570853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:19:58.570853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:19:58.570853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:19:58.570853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:19:58.570853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:19:58.570853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:19:58.570853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:58.570853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:19:58.570853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:58.570853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:19:58.570853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:58.571206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:19:58.571206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:58.571206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:19:58.571206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:58.571206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:19:58.571206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:19:58.571206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:58.571206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:19:58.571206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:58.571206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:19:58.571206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:19:58.571206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:58.571206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:19:58.571554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:58.571554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:19:58.571854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:58.571854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:19:58.571854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:19:58.571854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:19:58.571854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:19:58.571854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:19:58.571854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:19:58.571854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:19:58.571854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:19:58.571854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:19:58.571854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:19:58.571854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:19:58.571854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:19:58.571854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:19:58.571854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:19:58.571854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:19:58.572316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:19:58.572316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:19:58.572316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:58.572316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:19:58.572316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:19:58.572316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:19:58.572316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:19:58.572316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:19:58.572316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:19:58.572316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:19:58.572316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:19:58.572316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:19:58.572316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:19:58.572316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:19:58.572316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:19:58.572316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:19:58.572316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:19:58.572316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:19:58.572316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:19:58.572829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:19:58.572829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:19:58.572829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:20:03.375434 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:20:03.376051 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:20:03.376051 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:20:03.376695 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:20:03.414547 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:20:03.415009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:03.416148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:20:03.416148 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:03.416148 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:20:03.416148 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:20:03.416148 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:03.416148 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:20:03.416148 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:03.416148 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:20:03.417078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:03.417078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:20:03.417078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:20:03.417078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:20:03.417078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:20:03.417078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:20:03.417078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:20:03.417078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:03.417078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:20:03.417078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:03.417078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:20:03.417078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:03.417479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:20:03.417479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:03.417479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:20:03.417479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:03.417479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:20:03.417479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:20:03.417479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:03.417479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:20:03.417479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:03.417479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:20:03.417479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:20:03.417479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:03.417479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:20:03.417877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:03.417877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:20:03.417877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:03.417877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:20:03.417877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:20:03.417877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:20:03.417877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:20:03.417877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:20:03.417877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:20:03.417877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:20:03.417877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:20:03.417877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:20:03.417877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:20:03.417877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:20:03.417877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:20:03.418361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:20:03.418361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:20:03.418361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:20:03.418361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:20:03.418361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:20:03.418361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:03.418361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:20:03.418361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:03.418361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:20:03.418361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:03.418361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:20:03.418361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:03.418361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:20:03.418361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:03.418361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:20:03.418361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:20:03.418361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:03.418361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:20:03.418361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:03.418975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:20:03.418975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:20:03.418975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:03.418975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:20:03.418975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:03.418975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:20:03.540471 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:20:03.541657 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:20:03.541657 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:20:03.542371 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:20:03.542831 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:20:03.543088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:03.544983 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:20:03.544983 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:03.544983 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:20:03.544983 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:20:03.544983 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:03.544983 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:20:03.544983 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:03.544983 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:20:03.545386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:03.545386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:20:03.545386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:20:03.545386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:20:03.545386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:20:03.545386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:20:03.545386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:20:03.545386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:03.545386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:20:03.545386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:03.545386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:20:03.545386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:03.545784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:20:03.545784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:03.545784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:20:03.545784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:03.545784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:20:03.545784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:20:03.545784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:03.545784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:20:03.545784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:03.545784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:20:03.545784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:20:03.545784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:03.545784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:20:03.546608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:03.546608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:20:03.546608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:03.546608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:20:03.546608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:20:03.546608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:20:03.546608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:20:03.546608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:20:03.546608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:20:03.546608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:20:03.546608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:20:03.546608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:20:03.546608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:20:03.546608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:20:03.546608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:20:03.547049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:20:03.547049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:20:03.547049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:20:03.547049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:20:03.547049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:20:03.547049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:03.547049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:20:03.547049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:03.547049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:20:03.547049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:03.547049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:20:03.547049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:03.547049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:20:03.547049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:03.547049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:20:03.547049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:20:03.547049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:03.547049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:20:03.547049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:03.547553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:20:03.547553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:20:03.547553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:03.547553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:20:03.547553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:03.547553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:20:03.549733 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:20:03.550003 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:20:03.550213 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:20:03.550407 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:20:03.551292 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:20:03.551842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:03.552824 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:20:03.552824 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:03.552824 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:20:03.552824 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:20:03.552824 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:03.552824 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:20:03.552824 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:03.552824 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:20:03.553613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:03.553613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:20:03.553613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:20:03.553613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:20:03.553613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:03.553613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:20:03.553613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:03.553613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:20:03.553613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:03.553613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:20:03.553613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:03.553613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:20:03.553613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:03.553613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:20:03.553613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:20:03.554187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:03.554187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:20:03.554187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:03.554187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:20:03.554187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:20:03.554187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:03.554187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:20:03.554187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:03.554187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:20:03.555046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:03.555046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:20:03.555046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:20:03.555046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:20:03.555046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:20:03.555046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:20:03.555046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:20:03.555046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:20:03.555046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:20:03.555046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:20:03.555046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:03.555046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:20:03.555451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:03.555451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:20:03.555451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:03.555451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:20:03.555451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:03.555451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:20:03.555451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:03.555451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:20:03.555451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:20:03.555451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:03.555451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:20:03.555451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:03.555451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:20:03.555451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:20:03.555907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:03.555907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:20:03.555907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:03.555907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:20:03.556826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:03.556826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:20:03.556826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:20:03.556826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:20:03.556826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:20:03.556826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:20:03.556826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:20:03.556826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:20:03.556826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:20:03.556826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:20:03.556826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:20:03.556826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:20:03.556826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:20:03.556826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:20:03.556826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:20:03.556826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:20:03.557284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:20:03.557284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:20:03.557284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:20:03.557284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:20:03.557284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:20:03.557284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:03.557284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:20:03.557284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:03.557284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:20:03.557284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:03.557284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:20:03.557284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:03.557284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:20:03.557284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:03.557284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:20:03.557284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:20:03.557284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:03.557284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:20:03.557789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:03.557789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:20:03.557789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:20:03.557789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:03.557789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:20:03.557789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:03.557789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:20:03.558053 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:20:03.558279 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:20:03.558403 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:20:03.558524 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:20:03.559093 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:20:03.559259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:03.559814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:20:03.559814 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:03.559814 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:20:03.559814 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:20:03.559814 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:03.559814 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:20:03.559814 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:03.559814 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:20:03.560553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:03.560553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:20:03.560553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:20:03.560553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:20:03.560553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:03.560553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:20:03.560553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:03.560553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:20:03.560553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:03.560553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:20:03.560553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:03.560553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:20:03.560553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:03.560553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:20:03.560553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:20:03.561029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:03.561029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:20:03.561029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:03.561029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:20:03.561029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:20:03.561029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:03.561029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:20:03.561029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:03.561029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:20:03.561430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:03.561430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:20:03.561430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:20:03.561430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:20:03.561430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:20:03.561430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:20:03.561430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:20:03.561430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:20:03.561430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:20:03.561430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:03.561430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:20:03.561793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:03.561793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:20:03.561793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:03.561793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:20:03.561793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:03.561793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:20:03.561793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:03.561793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:20:03.561793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:20:03.561793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:03.561793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:20:03.561793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:03.561793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:20:03.561793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:20:03.562147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:03.562147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:20:03.562147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:03.562147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:20:03.562328 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:03.562328 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:20:03.562328 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:20:03.562328 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:20:03.562328 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:20:03.562328 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:20:03.562328 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:20:03.562328 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:20:03.562328 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:20:03.562328 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:20:03.562328 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:20:03.562328 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:20:03.562328 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:20:03.562328 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:20:03.562328 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:20:03.562328 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:20:03.562328 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:20:03.562782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:20:03.562782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:20:03.562782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:20:03.562782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:03.562782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:20:03.562782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:03.562782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:20:03.562782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:03.562782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:20:03.562782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:03.562782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:20:03.562782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:03.562782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:20:03.562782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:20:03.562782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:03.562782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:20:03.562782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:03.562782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:20:03.562782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:20:03.563318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:03.563318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:20:03.563318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:03.563318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:20:03.567724 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:20:03.568084 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:20:03.568234 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:20:03.568636 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:20:03.569184 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:20:03.569703 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:20:03.570055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:03.570990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:20:03.570990 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:03.570990 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:20:03.570990 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:20:03.570990 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:03.570990 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:20:03.570990 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:03.570990 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:20:03.572223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:03.572223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:20:03.572223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:20:03.572223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:20:03.572223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:20:03.572223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:20:03.572223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:20:03.572223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:03.572223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:20:03.572223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:03.572223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:20:03.572223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:03.572602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:20:03.572602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:03.572602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:20:03.572602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:03.572602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:20:03.572602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:20:03.572602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:03.572602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:20:03.572602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:03.572602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:20:03.572602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:20:03.572602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:03.572602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:20:03.573095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:03.573095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:20:03.573199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:03.573199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:20:03.573199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:20:03.573199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:20:03.573199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:20:03.573199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:20:03.573199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:20:03.573199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:20:03.573199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:20:03.573199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:20:03.573199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:20:03.573199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:20:03.573199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:20:03.573199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:20:03.573199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:20:03.573199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:20:03.573642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:20:03.573642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:20:03.573642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:03.573642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:20:03.573642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:03.573642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:20:03.573642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:03.573642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:20:03.573642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:03.573642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:20:03.573642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:03.573642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:20:03.573642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:20:03.573642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:03.573642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:20:03.573642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:03.573642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:20:03.573642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:20:03.573642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:03.574183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:20:03.574183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:03.574183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:20:08.376149 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:20:08.376973 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:20:08.376973 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:20:08.377690 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:20:08.410952 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:20:08.411229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:08.411936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:20:08.411936 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:08.411936 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:20:08.411936 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:20:08.411936 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:08.411936 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:20:08.411936 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:08.411936 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:20:08.413533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:08.413533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:20:08.413533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:20:08.413533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:20:08.413533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:20:08.413533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:20:08.413533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:20:08.413533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:08.413533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:20:08.413533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:08.413533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:20:08.413533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:08.414841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:20:08.414841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:08.414841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:20:08.414841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:08.414841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:20:08.414841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:20:08.414841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:08.414841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:20:08.414841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:08.414841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:20:08.414841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:20:08.414841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:08.414841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:20:08.415556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:08.415556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:20:08.415556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:08.415556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:20:08.415556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:20:08.415556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:20:08.415556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:20:08.415556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:20:08.415556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:20:08.415556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:20:08.415556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:20:08.415556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:20:08.415556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:20:08.415556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:20:08.415556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:20:08.416292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:20:08.416292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:20:08.416292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:20:08.416292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:20:08.416292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:20:08.416292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:08.416292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:20:08.416292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:08.416292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:20:08.416292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:08.416292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:20:08.416292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:08.416292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:20:08.416292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:08.416292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:20:08.416292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:20:08.416292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:08.416292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:20:08.416292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:08.417121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:20:08.417121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:20:08.417121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:08.417121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:20:08.417121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:08.417121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:20:08.541241 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:20:08.541542 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:20:08.541673 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:20:08.543478 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:20:08.543837 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:20:08.544063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:08.544926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:20:08.544926 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:08.544926 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:20:08.544926 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:20:08.544926 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:08.544926 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:20:08.544926 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:08.544926 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:20:08.546034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:08.546034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:20:08.546034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:20:08.546034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:20:08.546034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:20:08.546034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:20:08.546034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:20:08.546034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:08.546034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:20:08.546034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:08.546034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:20:08.546034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:08.546544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:20:08.546544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:08.546544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:20:08.546544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:08.546544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:20:08.546544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:20:08.546544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:08.546544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:20:08.546544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:08.546544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:20:08.546544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:20:08.546544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:08.546544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:20:08.547108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:08.547108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:20:08.547108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:08.547108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:20:08.547108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:20:08.547108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:20:08.547108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:20:08.547108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:20:08.547108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:20:08.547108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:20:08.547108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:20:08.547108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:20:08.547108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:20:08.547108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:20:08.547108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:20:08.547861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:20:08.547861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:20:08.547861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:20:08.547861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:20:08.547861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:20:08.547861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:08.547861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:20:08.547861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:08.547861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:20:08.547861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:08.547861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:20:08.547861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:08.547861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:20:08.547861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:08.547861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:20:08.547861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:20:08.547861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:08.547861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:20:08.547861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:08.548596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:20:08.548596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:20:08.548596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:08.548596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:20:08.548596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:08.548596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:20:08.550054 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:20:08.551303 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:20:08.551435 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:20:08.551834 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:20:08.552541 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:20:08.552865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:08.554028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:20:08.554028 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:08.554028 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:20:08.554028 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:20:08.554028 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:08.554028 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:20:08.554028 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:08.554028 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:20:08.555958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:08.555958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:20:08.555958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:20:08.555958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:20:08.555958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:08.555958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:20:08.555958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:08.555958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:20:08.555958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:08.555958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:20:08.555958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:08.555958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:20:08.555958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:08.555958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:20:08.555958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:20:08.556352 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:08.556352 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:20:08.556352 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:08.556352 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:20:08.556352 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:20:08.556352 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:08.556352 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:20:08.556352 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:08.556352 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:20:08.557737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:08.557737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:20:08.557737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:20:08.557737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:20:08.557737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:20:08.557737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:20:08.557737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:20:08.557737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:20:08.557737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:20:08.557737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:20:08.557737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:08.557737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:20:08.558997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:08.558997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:20:08.558997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:08.558997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:20:08.558997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:08.558997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:20:08.558997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:08.558997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:20:08.558997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:20:08.558997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:08.558997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:20:08.558997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:08.558997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:20:08.558997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:20:08.560496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:08.560496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:20:08.560496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:08.560496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:20:08.560496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:08.560496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:20:08.560496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:20:08.560496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:20:08.560496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:20:08.560496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:20:08.560496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:20:08.560496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:20:08.560496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:20:08.560496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:20:08.561481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:20:08.561481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:20:08.561481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:20:08.561481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:20:08.561481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:20:08.561481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:20:08.561481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:20:08.561481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:20:08.561481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:20:08.561481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:20:08.561481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:20:08.561481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:08.561481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:20:08.561481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:08.561481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:20:08.561481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:08.561481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:20:08.561481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:08.561481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:20:08.562154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:08.562154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:20:08.562154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:20:08.562154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:08.562154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:20:08.562154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:08.562154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:20:08.562154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:20:08.562154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:08.562154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:20:08.562154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:08.562154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:20:08.562154 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:20:08.562154 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:20:08.562664 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:20:08.562664 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:20:08.562664 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:20:08.562664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:08.562664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:20:08.562664 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:08.562664 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:20:08.562664 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:20:08.562664 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:08.562664 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:20:08.562664 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:08.562664 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:20:08.563119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:08.563119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:20:08.563119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:20:08.563119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:20:08.563119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:08.563119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:20:08.563119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:08.563119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:20:08.563119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:08.563119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:20:08.563119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:08.563119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:20:08.563119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:08.563119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:20:08.563119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:20:08.563510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:08.563510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:20:08.563510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:08.563510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:20:08.563510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:20:08.563510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:08.563510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:20:08.563510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:08.563510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:20:08.563510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:08.563510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:20:08.563510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:20:08.563510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:20:08.563510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:20:08.563935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:20:08.563935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:20:08.563935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:20:08.563935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:20:08.563935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:08.563935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:20:08.563935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:08.563935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:20:08.563935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:08.563935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:20:08.563935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:08.563935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:20:08.563935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:08.564292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:20:08.564292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:20:08.564292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:08.564292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:20:08.564292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:08.564292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:20:08.564292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:20:08.564292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:08.564292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:20:08.564292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:08.564292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:20:08.564601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:08.564601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:20:08.564601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:20:08.564601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:20:08.564601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:20:08.564601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:20:08.564601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:20:08.564601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:20:08.564601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:20:08.564601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:20:08.564601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:20:08.564601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:20:08.564601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:20:08.564601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:20:08.564601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:20:08.564601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:20:08.564601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:20:08.565432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:20:08.565432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:20:08.565432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:20:08.565432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:08.565432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:20:08.565432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:08.565432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:20:08.565432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:08.565432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:20:08.565432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:08.565432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:20:08.565432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:08.565432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:20:08.565432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:20:08.565432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:08.565432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:20:08.565432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:08.565432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:20:08.565432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:20:08.566129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:08.566129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:20:08.566129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:08.566129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:20:08.568286 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:20:08.568912 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:20:08.569161 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:20:08.569437 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:20:08.570339 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:20:08.570698 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:20:08.570948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:08.572092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:20:08.572092 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:08.572092 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:20:08.572092 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:20:08.572092 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:08.572092 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:20:08.572092 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:08.572092 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:20:08.573470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:08.573470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:20:08.573470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:20:08.573470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:20:08.573470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:20:08.573470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:20:08.573470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:20:08.573470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:08.573470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:20:08.573470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:08.573470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:20:08.573470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:08.573705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:20:08.573705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:08.573705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:20:08.573705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:08.573705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:20:08.573705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:20:08.573705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:08.573705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:20:08.573705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:08.573705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:20:08.573705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:20:08.573705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:08.573705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:20:08.574024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:08.574024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:20:08.575060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:08.575060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:20:08.575060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:20:08.575060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:20:08.575060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:20:08.575060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:20:08.575060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:20:08.575060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:20:08.575060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:20:08.575060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:20:08.575060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:20:08.575060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:20:08.575060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:20:08.575060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:20:08.575060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:20:08.575060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:20:08.575410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:20:08.575410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:20:08.575410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:08.575410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:20:08.575410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:08.575410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:20:08.575410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:08.575410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:20:08.575410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:08.575410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:20:08.575410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:08.575410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:20:08.575410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:20:08.575410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:08.575410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:20:08.575410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:08.575410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:20:08.575410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:20:08.575410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:08.575983 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:20:08.575983 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:08.575983 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:20:13.378036 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:20:13.378036 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:20:13.378036 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:20:13.378036 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:20:13.411956 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:20:13.412397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:13.413523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:20:13.413523 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:13.413523 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:20:13.413523 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:20:13.413523 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:13.413523 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:20:13.413523 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:13.413523 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:20:13.414316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:13.414316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:20:13.414316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:20:13.414316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:20:13.414316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:20:13.414316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:20:13.414316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:20:13.414316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:13.414316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:20:13.414316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:13.414316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:20:13.414316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:13.414625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:20:13.414625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:13.414625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:20:13.414625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:13.414625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:20:13.414625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:20:13.414625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:13.414625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:20:13.414625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:13.414625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:20:13.414625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:20:13.414625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:13.414625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:20:13.414918 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:13.414918 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:20:13.415199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:13.415199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:20:13.415199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:20:13.415199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:20:13.415199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:20:13.415199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:20:13.415199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:20:13.415199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:20:13.415199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:20:13.415199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:20:13.415199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:20:13.415199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:20:13.415199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:20:13.415199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:20:13.415199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:20:13.415199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:20:13.415600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:20:13.415600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:20:13.415600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:13.415600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:20:13.415600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:13.415600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:20:13.415600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:13.415600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:20:13.415600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:13.415600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:20:13.415600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:13.415600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:20:13.415600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:20:13.415600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:13.415600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:20:13.415600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:13.415600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:20:13.415600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:20:13.415600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:13.416039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:20:13.416039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:13.416039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:20:13.541773 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:20:13.542986 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:20:13.542986 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:20:13.543457 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:20:13.543706 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:20:13.543942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:13.544735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:20:13.544735 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:13.544735 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:20:13.544735 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:20:13.544735 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:13.544735 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:20:13.544735 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:13.544735 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:20:13.546653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:13.546653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:20:13.546653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:20:13.546653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:20:13.546653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:20:13.546653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:20:13.546653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:20:13.546653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:13.546653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:20:13.546653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:13.546653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:20:13.546653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:13.547071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:20:13.547071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:13.547071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:20:13.547071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:13.547071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:20:13.547071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:20:13.547071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:13.547071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:20:13.547071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:13.547071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:20:13.547071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:20:13.547071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:13.547071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:20:13.547416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:13.547416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:20:13.549896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:13.549896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:20:13.549896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:20:13.549896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:20:13.549896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:20:13.549896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:20:13.549896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:20:13.549896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:20:13.549896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:20:13.549896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:20:13.549896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:20:13.549896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:20:13.549896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:20:13.549896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:20:13.549896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:20:13.549896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:20:13.551515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:20:13.551515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:20:13.551515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:13.551515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:20:13.551515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:13.551515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:20:13.551515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:13.551515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:20:13.551515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:13.551515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:20:13.551515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:13.551515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:20:13.551515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:20:13.551515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:13.551515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:20:13.551515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:13.551515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:20:13.551515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:20:13.551515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:13.552072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:20:13.552072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:13.552072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:20:13.552072 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:20:13.552072 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:20:13.552072 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:20:13.552484 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:20:13.553011 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:20:13.553254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:13.554009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:20:13.554009 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:13.554009 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:20:13.554009 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:20:13.554009 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:13.554009 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:20:13.554009 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:13.554009 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:20:13.554885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:13.554885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:20:13.554885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:20:13.554885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:20:13.554885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:13.554885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:20:13.554885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:13.554885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:20:13.554885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:13.554885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:20:13.554885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:13.554885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:20:13.554885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:13.554885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:20:13.554885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:20:13.555306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:13.555306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:20:13.555306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:13.555306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:20:13.555306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:20:13.555306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:13.555306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:20:13.555306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:13.555306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:20:13.555941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:13.555941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:20:13.555941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:20:13.555941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:20:13.555941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:20:13.555941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:20:13.555941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:20:13.555941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:20:13.555941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:20:13.555941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:20:13.555941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:13.555941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:20:13.557083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:13.557083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:20:13.557083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:13.557083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:20:13.557083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:13.557083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:20:13.557083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:13.557083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:20:13.557083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:20:13.557083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:13.557083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:20:13.557083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:13.557083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:20:13.557083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:20:13.557583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:13.557583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:20:13.557583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:13.557583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:20:13.557583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:13.557583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:20:13.557583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:20:13.557583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:20:13.557583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:20:13.557583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:20:13.557583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:20:13.557583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:20:13.557583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:20:13.557583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:20:13.558207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:20:13.558207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:20:13.558207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:20:13.558207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:20:13.558207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:20:13.558207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:20:13.558207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:20:13.558207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:20:13.558207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:20:13.558207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:20:13.558207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:20:13.558207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:13.558207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:20:13.558207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:13.558207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:20:13.558207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:13.558207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:20:13.558207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:13.558207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:20:13.559388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:13.559388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:20:13.559388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:20:13.559388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:13.559388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:20:13.559388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:13.559388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:20:13.559388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:20:13.559388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:13.559388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:20:13.559388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:13.559388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:20:13.560126 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:20:13.560360 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:20:13.560508 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:20:13.560864 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:20:13.561207 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:20:13.561457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:13.562169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:20:13.562169 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:13.562169 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:20:13.562169 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:20:13.562169 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:13.562169 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:20:13.562169 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:13.562169 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:20:13.563004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:13.563004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:20:13.563004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:20:13.563004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:20:13.563004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:13.563004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:20:13.563004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:13.563004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:20:13.563004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:13.563004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:20:13.563004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:13.563004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:20:13.563004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:13.563004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:20:13.563004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:20:13.563407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:13.563407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:20:13.563407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:13.563407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:20:13.563407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:20:13.563407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:13.563407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:20:13.563407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:13.563407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:20:13.564079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:13.564079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:20:13.564079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:20:13.564079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:20:13.564079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:20:13.564079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:20:13.564079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:20:13.564079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:20:13.564079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:20:13.564079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:13.564079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:20:13.564446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:13.564446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:20:13.564446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:13.564446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:20:13.564446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:13.564446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:20:13.564446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:13.564446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:20:13.564446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:20:13.564446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:13.564446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:20:13.564446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:13.564446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:20:13.564446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:20:13.564812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:13.564812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:20:13.564812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:13.564812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:20:13.565877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:13.565877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:20:13.565877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:20:13.565877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:20:13.565877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:20:13.565877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:20:13.565877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:20:13.565877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:20:13.565877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:20:13.565877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:20:13.565877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:20:13.565877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:20:13.565877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:20:13.565877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:20:13.565877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:20:13.565877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:20:13.565877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:20:13.566699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:20:13.566699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:20:13.566699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:20:13.566699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:13.566699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:20:13.566699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:13.566699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:20:13.566699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:13.566699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:20:13.566699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:13.566699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:20:13.566699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:13.566699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:20:13.566699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:20:13.566699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:13.566699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:20:13.566699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:13.566699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:20:13.566699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:20:13.567200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:13.567200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:20:13.567200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:13.567200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:20:13.568679 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:20:13.568996 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:20:13.569127 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:20:13.569326 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:20:13.570028 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:20:13.570414 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:20:13.570632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:13.571403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:20:13.571403 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:13.571403 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:20:13.571403 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:20:13.571403 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:13.571403 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:20:13.571403 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:13.571403 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:20:13.572529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:13.572529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:20:13.572529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:20:13.572529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:20:13.572529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:20:13.572529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:20:13.572529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:20:13.572529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:13.572529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:20:13.572529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:13.572529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:20:13.572529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:13.572892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:20:13.572892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:13.572892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:20:13.572892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:13.572892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:20:13.572892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:20:13.572892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:13.572892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:20:13.572892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:13.572892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:20:13.572892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:20:13.572892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:13.572892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:20:13.573191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:13.573191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:20:13.573872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:13.573872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:20:13.573872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:20:13.573872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:20:13.573872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:20:13.573872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:20:13.573872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:20:13.573872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:20:13.573872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:20:13.573872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:20:13.573872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:20:13.573872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:20:13.573872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:20:13.573872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:20:13.573872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:20:13.573872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:20:13.574401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:20:13.574401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:20:13.574401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:13.574401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:20:13.574401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:13.574401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:20:13.574401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:13.574401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:20:13.574401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:13.574401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:20:13.574401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:13.574401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:20:13.574401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:20:13.574401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:13.574401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:20:13.574401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:13.574401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:20:13.574401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:20:13.574401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:13.574899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:20:13.574899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:13.574899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:20:18.378014 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:20:18.379006 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:20:18.379006 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:20:18.380356 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:20:18.416838 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:20:18.416838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:18.416838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:20:18.416838 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:18.416838 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:20:18.416838 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:20:18.416838 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:18.416838 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:20:18.416838 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:18.416838 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:20:18.417845 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:18.417845 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:20:18.417845 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:20:18.417845 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:20:18.417845 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:20:18.417845 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:20:18.417845 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:20:18.417845 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:18.417845 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:20:18.417845 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:18.417845 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:20:18.417845 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:18.418191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:20:18.418191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:18.418191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:20:18.418191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:18.418191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:20:18.418191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:20:18.418191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:18.418191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:20:18.418191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:18.418191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:20:18.418191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:20:18.418191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:18.418191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:20:18.418553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:18.418553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:20:18.418856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:18.418856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:20:18.418856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:20:18.418856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:20:18.418856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:20:18.418856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:20:18.418856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:20:18.418856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:20:18.418856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:20:18.418856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:20:18.418856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:20:18.418856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:20:18.418856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:20:18.418856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:20:18.418856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:20:18.418856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:20:18.419205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:20:18.419205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:20:18.419205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:18.419205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:20:18.419205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:18.419205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:20:18.419205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:18.419205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:20:18.419205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:18.419205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:20:18.419205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:18.419205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:20:18.419205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:20:18.419205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:18.419205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:20:18.419205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:18.419205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:20:18.419205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:20:18.419205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:18.419651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:20:18.419651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:18.419651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:20:18.542413 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:20:18.542919 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:20:18.543047 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:20:18.543725 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:20:18.543938 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:20:18.544093 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:18.544691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:20:18.544691 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:18.544691 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:20:18.544691 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:20:18.544691 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:18.544691 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:20:18.544691 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:18.544691 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:20:18.545769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:18.545769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:20:18.545769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:20:18.545769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:20:18.545769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:20:18.545769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:20:18.545769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:20:18.545769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:18.545769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:20:18.545769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:18.545769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:20:18.545769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:18.546118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:20:18.546118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:18.546118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:20:18.546118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:18.546118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:20:18.546118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:20:18.546118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:18.546118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:20:18.546118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:18.546118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:20:18.546118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:20:18.546118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:18.546118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:20:18.546456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:18.546456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:20:18.547159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:18.547159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:20:18.547159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:20:18.547159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:20:18.547159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:20:18.547159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:20:18.547159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:20:18.547159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:20:18.547159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:20:18.547159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:20:18.547159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:20:18.547159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:20:18.547159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:20:18.547159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:20:18.547159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:20:18.547159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:20:18.547711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:20:18.547711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:20:18.547711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:18.547711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:20:18.547711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:18.547711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:20:18.547711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:18.547711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:20:18.547711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:18.547711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:20:18.547711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:18.547711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:20:18.547711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:20:18.547711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:18.547711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:20:18.547711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:18.547711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:20:18.547711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:20:18.547711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:18.548261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:20:18.548261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:18.548261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:20:18.551712 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:20:18.551992 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:20:18.552109 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:20:18.552233 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:20:18.553582 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:20:18.554155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:18.554922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:20:18.554922 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:18.554922 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:20:18.554922 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:20:18.554922 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:18.554922 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:20:18.554922 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:18.554922 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:20:18.555781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:18.555781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:20:18.555781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:20:18.555781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:20:18.555781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:18.555781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:20:18.555781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:18.555781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:20:18.555781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:18.555781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:20:18.555781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:18.555781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:20:18.555781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:18.555781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:20:18.555781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:20:18.556115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:18.556115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:20:18.556115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:18.556115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:20:18.556115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:20:18.556115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:18.556115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:20:18.556115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:18.556115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:20:18.556872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:18.556872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:20:18.556872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:20:18.556872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:20:18.556872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:20:18.556872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:20:18.556872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:20:18.556872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:20:18.556872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:20:18.556872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:20:18.556872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:18.556872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:20:18.557166 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:18.557166 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:20:18.557166 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:18.557166 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:20:18.557166 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:18.557166 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:20:18.557166 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:18.557166 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:20:18.557166 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:20:18.557166 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:18.557166 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:20:18.557166 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:18.557166 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:20:18.557166 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:20:18.557410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:18.557410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:20:18.557410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:18.557410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:20:18.558368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:18.558368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:20:18.558368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:20:18.558368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:20:18.558368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:20:18.558368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:20:18.558368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:20:18.558368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:20:18.558368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:20:18.558368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:20:18.558368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:20:18.558368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:20:18.558368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:20:18.558368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:20:18.558368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:20:18.558368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:20:18.558777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:20:18.558777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:20:18.558777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:20:18.558777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:20:18.558777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:20:18.558777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:18.558777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:20:18.558777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:18.558777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:20:18.558777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:18.558777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:20:18.558777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:18.558777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:20:18.558777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:18.558777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:20:18.558777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:20:18.558777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:18.558777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:20:18.559115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:18.559115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:20:18.559115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:20:18.559115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:18.559115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:20:18.559115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:18.559115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:20:18.560477 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:20:18.560718 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:20:18.560997 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:20:18.560997 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:20:18.561436 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:20:18.561855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:18.562440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:20:18.562440 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:18.562440 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:20:18.562440 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:20:18.562440 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:18.562440 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:20:18.562440 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:18.562440 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:20:18.563249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:18.563249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:20:18.563249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:20:18.563249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:20:18.563249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:18.563249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:20:18.563249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:18.563249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:20:18.563249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:18.563249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:20:18.563249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:18.563249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:20:18.563249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:18.563249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:20:18.563249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:20:18.563565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:18.563565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:20:18.563565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:18.563565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:20:18.563565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:20:18.563565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:18.563565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:20:18.563565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:18.563565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:20:18.564180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:18.564180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:20:18.564180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:20:18.564180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:20:18.564180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:20:18.564180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:20:18.564180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:20:18.564180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:20:18.564180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:20:18.564180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:18.564180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:20:18.564446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:18.564446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:20:18.564446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:18.564446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:20:18.564446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:18.564446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:20:18.564446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:18.564446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:20:18.564446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:20:18.564446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:18.564446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:20:18.564446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:18.564446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:20:18.564446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:20:18.564800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:18.564800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:20:18.564800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:18.564800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:20:18.565959 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:18.565959 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:20:18.565959 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:20:18.565959 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:20:18.565959 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:20:18.565959 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:20:18.565959 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:20:18.565959 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:20:18.565959 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:20:18.565959 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:20:18.565959 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:20:18.565959 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:20:18.565959 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:20:18.565959 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:20:18.565959 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:20:18.565959 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:20:18.565959 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:20:18.566341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:20:18.566341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:20:18.566341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:20:18.566341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:18.566341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:20:18.566341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:18.566341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:20:18.566341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:18.566341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:20:18.566341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:18.566341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:20:18.566341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:18.566341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:20:18.566341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:20:18.566341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:18.566341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:20:18.566341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:18.566341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:20:18.566341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:20:18.566721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:18.566721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:20:18.566721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:18.566721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:20:18.569153 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:20:18.569365 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:20:18.569516 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:20:18.569772 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:20:18.570048 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:20:18.571504 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:20:18.571504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:18.571504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:20:18.571504 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:18.571504 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:20:18.571504 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:20:18.571504 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:18.571504 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:20:18.571504 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:18.571504 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:20:18.571738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:18.571738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:20:18.571738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:20:18.571738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:20:18.571738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:20:18.571738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:20:18.571738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:20:18.571738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:18.571738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:20:18.571738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:18.571738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:20:18.571738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:18.572040 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:20:18.572040 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:18.572040 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:20:18.572040 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:18.572040 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:20:18.572040 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:20:18.572040 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:18.572040 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:20:18.572040 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:18.572040 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:20:18.572040 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:20:18.572040 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:18.572040 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:20:18.572289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:18.572289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:20:18.572763 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:18.572763 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:20:18.572763 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:20:18.572763 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:20:18.572763 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:20:18.572763 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:20:18.572763 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:20:18.572763 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:20:18.572763 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:20:18.572763 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:20:18.572763 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:20:18.572763 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:20:18.572763 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:20:18.572763 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:20:18.572763 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:20:18.572763 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:20:18.573159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:20:18.573159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:20:18.573159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:18.573159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:20:18.573159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:18.573159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:20:18.573159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:18.573159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:20:18.573159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:18.573159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:20:18.573159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:18.573159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:20:18.573159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:20:18.573159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:18.573159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:20:18.573159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:18.573159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:20:18.573159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:20:18.573159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:18.573488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:20:18.573488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:18.573488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:20:23.378536 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:20:23.379324 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:20:23.379324 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:20:23.383431 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:20:23.416350 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:20:23.416702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:23.417463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:20:23.417463 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:23.417463 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:20:23.417463 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:20:23.417463 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:23.417463 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:20:23.417463 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:23.417463 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:20:23.418656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:23.418656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:20:23.418656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:20:23.418656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:20:23.418656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:20:23.418656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:20:23.418656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:20:23.418656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:23.418656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:20:23.418656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:23.418656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:20:23.418656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:23.419282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:20:23.419282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:23.419282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:20:23.419282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:23.419282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:20:23.419282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:20:23.419282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:23.419282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:20:23.419282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:23.419282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:20:23.419282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:20:23.419282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:23.419282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:20:23.419688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:23.419688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:20:23.420055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:23.420055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:20:23.420055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:20:23.420055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:20:23.420055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:20:23.420055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:20:23.420055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:20:23.420055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:20:23.420055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:20:23.420055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:20:23.420055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:20:23.420055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:20:23.420055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:20:23.420055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:20:23.420055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:20:23.420055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:20:23.420500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:20:23.420500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:20:23.420500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:23.420500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:20:23.420500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:23.420500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:20:23.420500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:23.420500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:20:23.420500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:23.420500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:20:23.420500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:23.420500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:20:23.420500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:20:23.420500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:23.420500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:20:23.420500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:23.420500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:20:23.420500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:20:23.420500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:23.421145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:20:23.421145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:23.421145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:20:23.543355 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:20:23.543935 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:20:23.544025 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:20:23.545642 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:20:23.546118 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:20:23.546217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:23.547043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:20:23.547043 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:23.547043 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:20:23.547043 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:20:23.547043 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:23.547043 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:20:23.547043 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:23.547043 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:20:23.548318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:23.548318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:20:23.548318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:20:23.548318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:20:23.548318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:20:23.548318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:20:23.548318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:20:23.548318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:23.548318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:20:23.548318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:23.548318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:20:23.548318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:23.548800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:20:23.548800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:23.548800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:20:23.548800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:23.548800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:20:23.548800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:20:23.548800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:23.548800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:20:23.548800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:23.548800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:20:23.548800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:20:23.548800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:23.548800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:20:23.549148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:23.549148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:20:23.549485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:23.549485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:20:23.549485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:20:23.549485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:20:23.549485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:20:23.549485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:20:23.549485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:20:23.549485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:20:23.549485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:20:23.549485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:20:23.549485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:20:23.549485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:20:23.549485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:20:23.549485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:20:23.549485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:20:23.549485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:20:23.550144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:20:23.550144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:20:23.550144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:23.550144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:20:23.550144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:23.550144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:20:23.550144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:23.550144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:20:23.550144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:23.550144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:20:23.550144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:23.550144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:20:23.550144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:20:23.550144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:23.550144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:20:23.550144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:23.550144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:20:23.550144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:20:23.550144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:23.551073 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:20:23.551073 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:23.551073 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:20:23.552245 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:20:23.552498 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:20:23.552621 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:20:23.552950 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:20:23.554536 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:20:23.554536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:23.554536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:20:23.554536 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:23.554536 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:20:23.554536 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:20:23.554536 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:23.554536 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:20:23.554536 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:23.554536 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:20:23.554904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:23.554904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:20:23.554904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:20:23.554904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:20:23.554904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:23.554904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:20:23.554904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:23.554904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:20:23.554904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:23.554904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:20:23.554904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:23.554904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:20:23.554904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:23.554904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:20:23.554904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:20:23.555373 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:23.555373 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:20:23.555373 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:23.555373 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:20:23.555373 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:20:23.555373 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:23.555373 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:20:23.555373 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:23.555373 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:20:23.555869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:23.555869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:20:23.555869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:20:23.555869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:20:23.555869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:20:23.555869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:20:23.555869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:20:23.555869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:20:23.555869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:20:23.555869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:20:23.555869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:23.555869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:20:23.556529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:23.556529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:20:23.556529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:23.556529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:20:23.556529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:23.556529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:20:23.556529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:23.556529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:20:23.556529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:20:23.556529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:23.556529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:20:23.556529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:23.556529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:20:23.556529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:20:23.557125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:23.557125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:20:23.557125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:23.557125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:20:23.557883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:23.557883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:20:23.557883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:20:23.557883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:20:23.557883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:20:23.557883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:20:23.557883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:20:23.557883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:20:23.557883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:20:23.557883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:20:23.557883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:20:23.557883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:20:23.557883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:20:23.557883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:20:23.557883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:20:23.557883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:20:23.558330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:20:23.558330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:20:23.558330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:20:23.558330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:20:23.558330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:20:23.558330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:23.558330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:20:23.558330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:23.558330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:20:23.558330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:23.558330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:20:23.558330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:23.558330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:20:23.558330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:23.558330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:20:23.558330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:20:23.558330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:23.558330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:20:23.558824 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:23.558824 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:20:23.558824 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:20:23.558824 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:23.558824 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:20:23.558824 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:23.558824 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:20:23.561058 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:20:23.561302 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:20:23.561408 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:20:23.561542 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:20:23.562044 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:20:23.562335 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:23.563028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:20:23.563028 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:23.563028 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:20:23.563028 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:20:23.563028 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:23.563028 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:20:23.563028 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:23.563028 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:20:23.563911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:23.563911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:20:23.563911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:20:23.563911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:20:23.563911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:23.563911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:20:23.563911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:23.563911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:20:23.563911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:23.563911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:20:23.563911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:23.563911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:20:23.563911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:23.563911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:20:23.563911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:20:23.564357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:23.564357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:20:23.564357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:23.564357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:20:23.564357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:20:23.564357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:23.564357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:20:23.564357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:23.564357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:20:23.565099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:23.565099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:20:23.565099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:20:23.565099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:20:23.565099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:20:23.565099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:20:23.565099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:20:23.565099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:20:23.565099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:20:23.565099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:23.565099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:20:23.565473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:23.565473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:20:23.565473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:23.565473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:20:23.565473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:23.565473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:20:23.565473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:23.565473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:20:23.565473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:20:23.565473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:23.565473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:20:23.565473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:23.565473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:20:23.565473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:20:23.565871 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:23.565871 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:20:23.565871 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:23.565871 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:20:23.568563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:23.568563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:20:23.568563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:20:23.568563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:20:23.568563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:20:23.568563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:20:23.568563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:20:23.568563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:20:23.568563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:20:23.568563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:20:23.568563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:20:23.568563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:20:23.568563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:20:23.568563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:20:23.568563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:20:23.568563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:20:23.568563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:20:23.569221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:20:23.569221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:20:23.569221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:20:23.569221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:23.569221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:20:23.569221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:23.569221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:20:23.569221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:23.569221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:20:23.569221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:23.569221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:20:23.569221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:23.569221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:20:23.569221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:20:23.569221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:23.569221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:20:23.569221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:23.569221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:20:23.569221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:20:23.569739 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:23.569739 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:20:23.569739 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:23.569739 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:20:23.570222 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:20:23.570493 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:20:23.570613 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:20:23.570812 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:20:23.571226 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:20:23.571481 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:20:23.571760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:23.572257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:20:23.572257 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:23.572257 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:20:23.572257 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:20:23.572257 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:23.572257 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:20:23.572257 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:23.572257 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:20:23.573355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:23.573355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:20:23.573355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:20:23.573355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:20:23.573355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:20:23.573355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:20:23.573355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:20:23.573355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:23.573355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:20:23.573355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:23.573355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:20:23.573355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:23.573736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:20:23.573736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:23.573736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:20:23.573736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:23.573736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:20:23.573736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:20:23.573736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:23.573736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:20:23.573736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:23.573736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:20:23.573736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:20:23.573736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:23.573736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:20:23.574103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:23.574103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:20:23.574416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:23.574416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:20:23.574416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:20:23.574416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:20:23.574416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:20:23.574416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:20:23.574416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:20:23.574416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:20:23.574416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:20:23.574416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:20:23.574416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:20:23.574416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:20:23.574416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:20:23.574416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:20:23.574416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:20:23.574416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:20:23.574837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:20:23.574837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:20:23.574837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:23.574837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:20:23.574837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:23.574837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:20:23.574837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:23.574837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:20:23.574837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:23.574837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:20:23.574837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:23.574837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:20:23.574837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:20:23.574837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:23.574837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:20:23.574837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:23.574837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:20:23.574837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:20:23.574837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:23.575333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:20:23.575333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:23.575333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:20:28.379347 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:20:28.380909 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:20:28.380992 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:20:28.382491 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:20:28.419459 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:20:28.420031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:28.420952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:20:28.420952 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:28.420952 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:20:28.420952 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:20:28.420952 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:28.420952 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:20:28.420952 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:28.420952 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:20:28.421895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:28.421895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:20:28.421895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:20:28.421895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:20:28.421895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:20:28.421895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:20:28.421895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:20:28.421895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:28.421895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:20:28.421895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:28.421895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:20:28.421895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:28.422671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:20:28.422671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:28.422671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:20:28.422671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:28.422671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:20:28.422671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:20:28.422671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:28.422671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:20:28.422671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:28.422671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:20:28.422671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:20:28.422671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:28.422671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:20:28.423143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:28.423143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:20:28.423234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:28.423234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:20:28.423234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:20:28.423234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:20:28.423234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:20:28.423234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:20:28.423234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:20:28.423234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:20:28.423234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:20:28.423234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:20:28.423234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:20:28.423234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:20:28.423234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:20:28.423234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:20:28.423234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:20:28.423234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:20:28.423862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:20:28.423862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:20:28.423862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:28.423862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:20:28.423862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:28.423862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:20:28.423862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:28.423862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:20:28.423862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:28.423862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:20:28.423862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:28.423862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:20:28.423862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:20:28.423862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:28.423862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:20:28.423862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:28.423862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:20:28.423862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:20:28.423862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:28.424546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:20:28.424546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:28.424546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:20:28.543988 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:20:28.544586 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:20:28.544849 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:20:28.546040 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:20:28.546280 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:20:28.546449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:28.547263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:20:28.547263 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:28.547263 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:20:28.547263 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:20:28.547263 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:28.547263 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:20:28.547263 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:28.547263 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:20:28.549254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:28.549254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:20:28.549254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:20:28.549254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:20:28.549254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:20:28.549254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:20:28.549254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:20:28.549254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:28.549254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:20:28.549254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:28.549254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:20:28.549254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:28.549700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:20:28.549700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:28.549700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:20:28.549700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:28.549700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:20:28.549700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:20:28.549700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:28.549700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:20:28.549700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:28.549700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:20:28.549700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:20:28.549700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:28.549700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:20:28.550078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:28.550078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:20:28.551531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:28.551531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:20:28.551531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:20:28.551531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:20:28.551531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:20:28.551531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:20:28.551531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:20:28.551531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:20:28.551531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:20:28.551531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:20:28.551531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:20:28.551531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:20:28.551531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:20:28.551531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:20:28.551531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:20:28.551531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:20:28.552045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:20:28.552045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:20:28.552045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:28.552045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:20:28.552045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:28.552045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:20:28.552045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:28.552045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:20:28.552045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:28.552045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:20:28.552045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:28.552045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:20:28.552045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:20:28.552045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:28.552045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:20:28.552045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:28.552045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:20:28.552045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:20:28.552045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:28.552580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:20:28.552580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:28.552580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:20:28.552668 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:20:28.552992 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:20:28.553067 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:20:28.553162 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:20:28.553668 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:20:28.553791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:28.554560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:20:28.554560 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:28.554560 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:20:28.554560 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:20:28.554560 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:28.554560 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:20:28.554560 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:28.554560 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:20:28.556931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:28.556931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:20:28.556931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:20:28.556931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:20:28.556931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:28.556931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:20:28.556931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:28.556931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:20:28.556931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:28.556931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:20:28.556931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:28.556931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:20:28.556931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:28.556931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:20:28.556931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:20:28.557522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:28.557522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:20:28.557522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:28.557522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:20:28.557522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:20:28.557522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:28.557522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:20:28.557522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:28.557522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:20:28.557522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:28.557522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:20:28.557522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:20:28.557522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:20:28.557522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:20:28.558032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:20:28.558032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:20:28.558032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:20:28.558032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:20:28.558032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:20:28.558032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:28.558032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:20:28.558032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:28.558032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:20:28.558032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:28.558032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:20:28.558032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:28.558032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:20:28.558395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:28.558395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:20:28.558395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:20:28.558395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:28.558395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:20:28.558395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:28.558395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:20:28.558395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:20:28.558395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:28.558395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:20:28.558395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:28.558395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:20:28.558713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:28.558713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:20:28.558713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:20:28.558713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:20:28.558713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:20:28.558713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:20:28.558713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:20:28.558713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:20:28.558713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:20:28.558713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:20:28.558713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:20:28.558713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:20:28.558713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:20:28.558713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:20:28.558713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:20:28.558713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:20:28.559176 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:20:28.559176 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:20:28.559176 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:20:28.559176 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:20:28.559176 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:20:28.559176 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:28.559176 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:20:28.559176 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:28.559176 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:20:28.559176 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:28.559176 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:20:28.559176 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:28.559176 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:20:28.559176 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:28.559176 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:20:28.559176 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:20:28.559176 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:28.559176 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:20:28.559891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:28.559891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:20:28.559891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:20:28.559891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:28.559891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:20:28.559891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:28.559891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:20:28.561476 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:20:28.561908 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:20:28.562023 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:20:28.563625 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:20:28.563625 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:20:28.563625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:28.563841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:20:28.563841 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:28.563841 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:20:28.563841 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:20:28.563841 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:28.563841 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:20:28.563841 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:28.563841 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:20:28.564805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:28.564805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:20:28.564805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:20:28.564805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:20:28.564805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:28.564805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:20:28.564805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:28.564805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:20:28.564805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:28.564805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:20:28.564805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:28.564805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:20:28.564805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:28.564805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:20:28.564805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:20:28.565408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:28.565408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:20:28.565408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:28.565408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:20:28.565408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:20:28.565408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:28.565408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:20:28.565408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:28.565408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:20:28.566506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:28.566506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:20:28.566506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:20:28.566506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:20:28.566506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:20:28.566506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:20:28.566506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:20:28.566506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:20:28.566506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:20:28.566506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:28.566506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:20:28.566865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:28.566865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:20:28.566865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:28.566865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:20:28.566865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:28.566865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:20:28.566865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:28.566865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:20:28.566865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:20:28.566865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:28.566865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:20:28.566865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:28.566865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:20:28.566865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:20:28.567182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:28.567182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:20:28.567182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:28.567182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:20:28.568230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:28.568230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:20:28.568230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:20:28.568230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:20:28.568230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:20:28.568230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:20:28.568230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:20:28.568230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:20:28.568230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:20:28.568230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:20:28.568230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:20:28.568230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:20:28.568230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:20:28.568230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:20:28.568230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:20:28.568230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:20:28.568230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:20:28.568838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:20:28.568838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:20:28.568838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:20:28.568838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:28.568838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:20:28.568838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:28.568838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:20:28.568838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:28.568838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:20:28.568838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:28.568838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:20:28.568838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:28.568838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:20:28.568838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:20:28.568838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:28.568838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:20:28.568838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:28.568838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:20:28.568838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:20:28.569306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:28.569306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:20:28.569306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:28.569306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:20:28.570635 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:20:28.571003 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:20:28.571113 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:20:28.571292 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:20:28.571877 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:20:28.572282 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:20:28.572512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:28.575047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:20:28.575047 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:28.575047 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:20:28.575047 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:20:28.575047 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:28.575047 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:20:28.575047 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:28.575047 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:20:28.576006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:28.576006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:20:28.576006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:20:28.576006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:20:28.576006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:20:28.576006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:20:28.576006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:20:28.576006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:28.576006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:20:28.576006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:28.576006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:20:28.576006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:28.576397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:20:28.576397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:28.576397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:20:28.576397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:28.576397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:20:28.576397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:20:28.576397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:28.576397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:20:28.576397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:28.576397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:20:28.576397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:20:28.576397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:28.576397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:20:28.576712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:28.576712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:20:28.576846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:28.576846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:20:28.576846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:20:28.576846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:20:28.576846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:20:28.576846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:20:28.576846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:20:28.576846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:20:28.576846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:20:28.576846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:20:28.576846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:20:28.576846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:20:28.576846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:20:28.576846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:20:28.576846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:20:28.576846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:20:28.577261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:20:28.577261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:20:28.577261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:28.577261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:20:28.577261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:28.577261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:20:28.577261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:28.577261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:20:28.577261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:28.577261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:20:28.577261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:28.577261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:20:28.577261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:20:28.577261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:28.577261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:20:28.577261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:28.577261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:20:28.577261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:20:28.577261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:28.577900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:20:28.577900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:28.577900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:20:33.380172 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:20:33.380892 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:20:33.381119 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:20:33.382559 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:20:33.431794 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:20:33.432193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:33.432901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:20:33.432901 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:33.432901 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:20:33.432901 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:20:33.432901 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:33.432901 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:20:33.432901 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:33.432901 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:20:33.434675 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.052 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:33.434675 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:20:33.434675 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:20:33.434675 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:20:33.434675 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:20:33.434675 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:20:33.434675 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:20:33.434675 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:33.434675 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:20:33.434675 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:33.434675 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:20:33.434675 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:33.435002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:20:33.435002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:33.435002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:20:33.435002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:33.435002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:20:33.435002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:20:33.435002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:33.435002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:20:33.435002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:33.435002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:20:33.435002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:20:33.435002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:33.435002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:20:33.435287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:33.435287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:20:33.435478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:33.435478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:20:33.435478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:20:33.435478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:20:33.435478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:20:33.435478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:20:33.435478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:20:33.435478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:20:33.435478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:20:33.435478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:20:33.435478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:20:33.435478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:20:33.435478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:20:33.435478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:20:33.435478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:20:33.435478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:20:33.435835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:20:33.435835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:20:33.435835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:33.435835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:20:33.435835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:33.435835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:20:33.435835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:33.435835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:20:33.435835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:33.435835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:20:33.435835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:33.435835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:20:33.435835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:20:33.435835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:33.435835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:20:33.435835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:33.435835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:20:33.435835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:20:33.435835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:33.436220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:20:33.436220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:33.436220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:20:33.544790 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:20:33.545390 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:20:33.545585 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:20:33.546672 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:20:33.546895 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:20:33.547055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:33.547921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:20:33.547921 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:33.547921 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:20:33.547921 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:20:33.547921 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:33.547921 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:20:33.547921 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:33.547921 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:20:33.549303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:33.549303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:20:33.549303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:20:33.549303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:20:33.549303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:20:33.549303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:20:33.549303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:20:33.549303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:33.549303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:20:33.549303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:33.549303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:20:33.549303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:33.549709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:20:33.549709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:33.549709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:20:33.549709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:33.549709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:20:33.549709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:20:33.549709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:33.549709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:20:33.549709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:33.549709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:20:33.549709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:20:33.549709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:33.549709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:20:33.550097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:33.550097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:20:33.550529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:33.550529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:20:33.550529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:20:33.550529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:20:33.550529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:20:33.550529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:20:33.550529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:20:33.550529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:20:33.550529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:20:33.550529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:20:33.550529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:20:33.550529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:20:33.550529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:20:33.550529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:20:33.550529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:20:33.550529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:20:33.551032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:20:33.551032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:20:33.551032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:33.551032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:20:33.551032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:33.551032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:20:33.551032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:33.551032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:20:33.551032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:33.551032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:20:33.551032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:33.551032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:20:33.551032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:20:33.551032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:33.551032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:20:33.551032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:33.551032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:20:33.551032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:20:33.551032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:33.551588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:20:33.551588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:33.551588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:20:33.553127 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:20:33.553448 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:20:33.553612 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:20:33.553759 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:20:33.554188 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:20:33.554427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:33.555018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:20:33.555018 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:33.555018 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:20:33.555018 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:20:33.555018 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:33.555018 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:20:33.555018 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:33.555018 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:20:33.555968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:33.555968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:20:33.555968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:20:33.555968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:20:33.555968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:33.555968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:20:33.555968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:33.555968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:20:33.555968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:33.555968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:20:33.555968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:33.555968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:20:33.555968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:33.555968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:20:33.555968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:20:33.556463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:33.556463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:20:33.556463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:33.556463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:20:33.556463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:20:33.556463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:33.556463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:20:33.556463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:33.556463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:20:33.556949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:33.556949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:20:33.556949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:20:33.556949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:20:33.556949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:20:33.556949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:20:33.556949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:20:33.556949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:20:33.556949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:20:33.556949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:20:33.556949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:33.556949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:20:33.557350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:33.557350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:20:33.557350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:33.557350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:20:33.557350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:33.557350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:20:33.557350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:33.557350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:20:33.557350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:20:33.557350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:33.557350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:20:33.557350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:33.557350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:20:33.557350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:20:33.557844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:33.557844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:20:33.557844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:33.557844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:20:33.558249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:33.558249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:20:33.558249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:20:33.558249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:20:33.558249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:20:33.558249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:20:33.558249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:20:33.558249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:20:33.558249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:20:33.558249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:20:33.558249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:20:33.558249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:20:33.558249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:20:33.558249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:20:33.558249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:20:33.558249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:20:33.558676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:20:33.558676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:20:33.558676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:20:33.558676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:20:33.558676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:20:33.558676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:33.558676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:20:33.558676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:33.558676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:20:33.558676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:33.558676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:20:33.558676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:33.558676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:20:33.558676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:33.558676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:20:33.558676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:20:33.558676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:33.558676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:20:33.559201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:33.559201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:20:33.559201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:20:33.559201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:33.559201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:20:33.559201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:33.559201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:20:33.562074 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:20:33.562425 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:20:33.562540 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:20:33.562668 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:20:33.563222 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:20:33.563612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:33.564518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:20:33.564518 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:33.564518 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:20:33.564518 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:20:33.564518 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:33.564518 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:20:33.564518 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:33.564518 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:20:33.565631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:33.565631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:20:33.565631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:20:33.565631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:20:33.565631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:33.565631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:20:33.565631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:33.565631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:20:33.565631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:33.565631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:20:33.565631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:33.565631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:20:33.565631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:33.565631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:20:33.565631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:20:33.566289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:33.566289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:20:33.566289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:33.566289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:20:33.566289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:20:33.566289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:33.566289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:20:33.566289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:33.566289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:20:33.566758 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:33.566758 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:20:33.566758 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:20:33.566758 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:20:33.566758 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:20:33.566758 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:20:33.566758 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:20:33.566758 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:20:33.566758 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:20:33.566758 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:33.566758 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:20:33.567188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:33.567188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:20:33.567188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:33.567188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:20:33.567188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:33.567188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:20:33.567188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:33.567188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:20:33.567188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:20:33.567188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:33.567188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:20:33.567188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:33.567188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:20:33.567188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:20:33.567691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:33.567691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:20:33.567691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:33.567691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:20:33.568047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:33.568047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:20:33.568047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:20:33.568047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:20:33.568047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:20:33.568047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:20:33.568047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:20:33.568047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:20:33.568047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:20:33.568047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:20:33.568047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:20:33.568047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:20:33.568047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:20:33.568047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:20:33.568047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:20:33.568047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:20:33.568047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:20:33.568663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:20:33.568663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:20:33.568663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:20:33.568663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:33.568663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:20:33.568663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:33.568663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:20:33.568663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:33.568663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:20:33.568663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:33.568663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:20:33.568663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:33.568663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:20:33.568663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:20:33.568663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:33.568663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:20:33.568663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:33.568663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:20:33.568663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:20:33.569364 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:33.569364 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:20:33.569364 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:33.569364 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:20:33.571132 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:20:33.571381 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:20:33.571485 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:20:33.571696 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:20:33.572157 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:20:33.572513 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:20:33.572731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:33.573396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:20:33.573396 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:33.573396 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:20:33.573396 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:20:33.573396 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:33.573396 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:20:33.573396 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:33.573396 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:20:33.574416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:33.574416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:20:33.574416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:20:33.574416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:20:33.574416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:20:33.574416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:20:33.574416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:20:33.574416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:33.574416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:20:33.574416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:33.574416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:20:33.574416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:33.574899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:20:33.574899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:33.574899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:20:33.574899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:33.574899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:20:33.574899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:20:33.574899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:33.574899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:20:33.574899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:33.574899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:20:33.574899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:20:33.574899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:33.574899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:20:33.575344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:33.575344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:20:33.575734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:33.575734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:20:33.575734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:20:33.575734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:20:33.575734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:20:33.575734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:20:33.575734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:20:33.575734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:20:33.575734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:20:33.575734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:20:33.575734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:20:33.575734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:20:33.575734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:20:33.575734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:20:33.575734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:20:33.575734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:20:33.576259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:20:33.576259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:20:33.576259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:33.576259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:20:33.576259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:33.576259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:20:33.576259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:33.576259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:20:33.576259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:33.576259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:20:33.576259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:33.576259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:20:33.576259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:20:33.576259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:33.576259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:20:33.576259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:33.576259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:20:33.576259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:20:33.576259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:33.576934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:20:33.576934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:33.576934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:20:38.381006 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:20:38.381543 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:20:38.381543 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:20:38.382572 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:20:38.419633 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:20:38.419633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:38.419911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:20:38.419911 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:38.419911 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:20:38.419911 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:20:38.419911 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:38.419911 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:20:38.419911 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:38.419911 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:20:38.420458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:38.420458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:20:38.420458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:20:38.420458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:20:38.420458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:20:38.420458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:20:38.420458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:20:38.420458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:38.420458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:20:38.420458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:38.420458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:20:38.420458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:38.420907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:20:38.420907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:38.420907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:20:38.420907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:38.420907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:20:38.420907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:20:38.420907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:38.420907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:20:38.420907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:38.420907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:20:38.420907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:20:38.420907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:38.420907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:20:38.421291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:38.421291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:20:38.421291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:38.421291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:20:38.421291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:20:38.421291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:20:38.421291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:20:38.421291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:20:38.421291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:20:38.421291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:20:38.421291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:20:38.421291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:20:38.421291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:20:38.421291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:20:38.421291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:20:38.421865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:20:38.421865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:20:38.421865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:20:38.421865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:20:38.421865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:20:38.421865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:38.421865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:20:38.421865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:38.421865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:20:38.421865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:38.421865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:20:38.421865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:38.421865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:20:38.421865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:38.421865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:20:38.421865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:20:38.421865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:38.421865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:20:38.421865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:38.422510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:20:38.422510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:20:38.422510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:38.422510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:20:38.422510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:38.422510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:20:38.545544 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:20:38.546113 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:20:38.548077 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:20:38.548977 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:20:38.549155 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:20:38.551045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:38.551045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:20:38.551045 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:38.551045 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:20:38.551045 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:20:38.551045 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:38.551045 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:20:38.551045 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:38.551045 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:20:38.551045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:38.551045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:20:38.551045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:20:38.551045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:20:38.551045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:20:38.551045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:20:38.551816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:20:38.551816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:38.551816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:20:38.551816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:38.551816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:20:38.551816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:38.551816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:20:38.551816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:38.551816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:20:38.551816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:38.551816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:20:38.551816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:20:38.551816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:38.551816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:20:38.552365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:38.552365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:20:38.552365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:20:38.552365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:38.552365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:20:38.552365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:38.552365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:20:38.552365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:38.552365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:20:38.552365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:20:38.552365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:20:38.552365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:20:38.552365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:20:38.552880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:20:38.552880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:20:38.552880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:20:38.552880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:20:38.552880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:20:38.552880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:20:38.552880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:20:38.552880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:20:38.552880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:20:38.552880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:20:38.552880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:20:38.552880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:20:38.552880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:38.552880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:20:38.552880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:38.552880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:20:38.552880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:38.552880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:20:38.552880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:38.552880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:20:38.553637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:38.553637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:20:38.553637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:20:38.553637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:38.553637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:20:38.553637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:38.553637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:20:38.553637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:20:38.553637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:38.553637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:20:38.553637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:38.553637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:20:38.554121 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:20:38.554121 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:20:38.554237 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:20:38.554366 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:20:38.554821 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:20:38.554972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:38.557944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:20:38.557944 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:38.557944 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:20:38.557944 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:20:38.557944 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:38.557944 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:20:38.557944 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:38.557944 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:20:38.558571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:38.558571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:20:38.558571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:20:38.558571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:20:38.558571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:38.558571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:20:38.558571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:38.558571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:20:38.558571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:38.558571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:20:38.558571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:38.558571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:20:38.558571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:38.558571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:20:38.558571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:20:38.560252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:38.560252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:20:38.560252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:38.560252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:20:38.560252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:20:38.560252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:38.560252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:20:38.560252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:38.560252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:20:38.560252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:38.560252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:20:38.560252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:20:38.560252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:20:38.560252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:20:38.560610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:20:38.560610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:20:38.560610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:20:38.560610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:20:38.560610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:20:38.560610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:38.560610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:20:38.560610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:38.560610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:20:38.560610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:38.560610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:20:38.560610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:38.560610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:20:38.560943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:38.560943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:20:38.560943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:20:38.560943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:38.560943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:20:38.560943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:38.560943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:20:38.560943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:20:38.560943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:38.560943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:20:38.560943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:38.560943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:20:38.561327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:38.561327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:20:38.561327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:20:38.561327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:20:38.561327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:20:38.561327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:20:38.561327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:20:38.561327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:20:38.561327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:20:38.561327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:20:38.561327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:20:38.561327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:20:38.561327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:20:38.561327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:20:38.561327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:20:38.561327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:20:38.561618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:20:38.561618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:20:38.561618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:20:38.561618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:20:38.561618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:20:38.561618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:38.561618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:20:38.561618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:38.561618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:20:38.561618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:38.561618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:20:38.561618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:38.561618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:20:38.561618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:38.561618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:20:38.561618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:20:38.561618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:38.561618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:20:38.562219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:38.562219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:20:38.562219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:20:38.562219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:38.562219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:20:38.562219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:38.562219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:20:38.562575 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:20:38.563023 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:20:38.563092 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:20:38.563257 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:20:38.563949 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:20:38.564151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:38.564522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:20:38.564522 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:38.564522 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:20:38.564522 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:20:38.564522 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:38.564522 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:20:38.564522 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:38.564522 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:20:38.565332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:38.565332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:20:38.565332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:20:38.565332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:20:38.565332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:38.565332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:20:38.565332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:38.565332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:20:38.565332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:38.565332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:20:38.565332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:38.565332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:20:38.565332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:38.565332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:20:38.565332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:20:38.565686 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:38.565686 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:20:38.565686 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:38.565686 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:20:38.565686 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:20:38.565686 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:38.565686 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:20:38.565686 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:38.565686 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:20:38.566135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:38.566135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:20:38.566135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:20:38.566135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:20:38.566135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:20:38.566135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:20:38.566135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:20:38.566135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:20:38.566135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:20:38.566135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:38.566135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:20:38.566456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:38.566456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:20:38.566456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:38.566456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:20:38.566456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:38.566456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:20:38.566456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:38.566456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:20:38.566456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:20:38.566456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:38.566456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:20:38.566456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:38.566456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:20:38.566456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:20:38.566809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:38.566809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:20:38.566809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:38.566809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:20:38.566809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:38.566809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:20:38.566809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:20:38.566809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:20:38.566809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:20:38.566809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:20:38.566809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:20:38.566809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:20:38.566809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:20:38.566809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:20:38.567926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:20:38.567926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:20:38.567926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:20:38.567926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:20:38.567926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:20:38.567926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:20:38.567926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:20:38.567926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:20:38.567926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:20:38.567926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:20:38.567926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:38.567926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:20:38.567926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:38.567926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:20:38.567926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:38.567926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:20:38.567926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:38.567926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:20:38.567926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:38.568326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:20:38.568326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:20:38.568326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:38.568326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:20:38.568326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:38.568326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:20:38.568326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:20:38.568326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:38.568326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:20:38.568326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:38.568326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:20:38.595657 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:20:38.595657 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:20:38.595657 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:20:38.595657 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:20:38.595657 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:20:38.595657 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:20:38.596152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:38.596152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:20:38.596152 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:38.596152 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:20:38.596152 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:20:38.596152 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:38.596152 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:20:38.596152 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:38.596152 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:20:38.596152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:38.596152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:20:38.596152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:20:38.596152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:20:38.596152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:20:38.596152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:20:38.596885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:20:38.596885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:38.596885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:20:38.596885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:38.596885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:20:38.596885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:38.596885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:20:38.596885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:38.596885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:20:38.596885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:38.596885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:20:38.596885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:20:38.596885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:38.596885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:20:38.597586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:38.597586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:20:38.597586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:20:38.597586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:38.597586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:20:38.597586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:38.597586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:20:38.597586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:38.597586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:20:38.597586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:20:38.597586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:20:38.597586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:20:38.597586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:20:38.598229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:20:38.598229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:20:38.598229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:20:38.598229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:20:38.598229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:20:38.598229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:20:38.598229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:20:38.598229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:20:38.598229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:20:38.598229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:20:38.598229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:20:38.598229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:20:38.598229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:38.598229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:20:38.598229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:38.598229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:20:38.598229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:38.598229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:20:38.598229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:38.598229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:20:38.599657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:38.599657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:20:38.599657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:20:38.599657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:38.599657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:20:38.599657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:38.599657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:20:38.599657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:20:38.599657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:38.599657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:20:38.599657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:38.599657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:20:43.381648 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:20:43.382226 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:20:43.382298 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:20:43.383374 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:20:43.417718 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:20:43.418163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:43.419150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:20:43.419150 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:43.419150 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:20:43.419150 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:20:43.419150 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:43.419150 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:20:43.419150 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:43.419150 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:20:43.420200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:43.420200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:20:43.420200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:20:43.420200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:20:43.420200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:20:43.420200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:20:43.420200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:20:43.420200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:43.420200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:20:43.420200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:43.420200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:20:43.420200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:43.421014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:20:43.421014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:43.421014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:20:43.421014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:43.421014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:20:43.421014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:20:43.421014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:43.421014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:20:43.421014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:43.421014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:20:43.421014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:20:43.421014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:43.421014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:20:43.421765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:43.421765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:20:43.421765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:43.421765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:20:43.421765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:20:43.421765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:20:43.421765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:20:43.421765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:20:43.421765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:20:43.421765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:20:43.421765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:20:43.421765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:20:43.421765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:20:43.421765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:20:43.421765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:20:43.422432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:20:43.422432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:20:43.422432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:20:43.422432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:20:43.422432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:20:43.422432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:43.422432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:20:43.422432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:43.422432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:20:43.422432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:43.422432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:20:43.422432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:43.422432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:20:43.422432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:43.422432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:20:43.422432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:20:43.422432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:43.422432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:20:43.422432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:43.423327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:20:43.423327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:20:43.423327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:43.423327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:20:43.423327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:43.423327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:20:43.546307 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:20:43.546799 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:20:43.546952 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:20:43.547906 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:20:43.548124 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:20:43.548289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:43.549046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:20:43.549046 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:43.549046 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:20:43.549046 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:20:43.549046 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:43.549046 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:20:43.549046 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:43.549046 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:20:43.550313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:43.550313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:20:43.550313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:20:43.550313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:20:43.550313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:20:43.550313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:20:43.550313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:20:43.550313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:43.550313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:20:43.550313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:43.550313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:20:43.550313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:43.550671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:20:43.550671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:43.550671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:20:43.550671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:43.550671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:20:43.550671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:20:43.550671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:43.550671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:20:43.550671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:43.550671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:20:43.550671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:20:43.550671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:43.550671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:20:43.551004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:43.551004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:20:43.551664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:43.551664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:20:43.551664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:20:43.551664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:20:43.551664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:20:43.551664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:20:43.551664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:20:43.551664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:20:43.551664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:20:43.551664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:20:43.551664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:20:43.551664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:20:43.551664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:20:43.551664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:20:43.551664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:20:43.551664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:20:43.552226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:20:43.552226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:20:43.552226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:43.552226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:20:43.552226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:43.552226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:20:43.552226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:43.552226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:20:43.552226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:43.552226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:20:43.552226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:43.552226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:20:43.552226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:20:43.552226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:43.552226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:20:43.552226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:43.552226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:20:43.552226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:20:43.552226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:43.552671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:20:43.552671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:43.552671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:20:43.555512 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:20:43.556240 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:20:43.556643 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:20:43.556643 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:20:43.557257 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:20:43.557474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:43.558108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:20:43.558108 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:43.558108 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:20:43.558108 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:20:43.558108 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:43.558108 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:20:43.558108 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:43.558108 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:20:43.558978 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:43.558978 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:20:43.558978 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:20:43.558978 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:20:43.558978 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:43.558978 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:20:43.558978 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:43.558978 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:20:43.558978 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:43.558978 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:20:43.558978 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:43.558978 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:20:43.558978 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:43.558978 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:20:43.558978 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:20:43.559654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:43.559654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:20:43.559654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:43.559654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:20:43.559654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:20:43.559654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:43.559654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:20:43.559654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:43.559654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:20:43.559996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:43.559996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:20:43.559996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:20:43.559996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:20:43.559996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:20:43.559996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:20:43.559996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:20:43.559996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:20:43.559996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:20:43.559996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:20:43.559996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:43.559996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:20:43.560445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:43.560445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:20:43.560445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:43.560445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:20:43.560445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:43.560445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:20:43.560445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:43.560445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:20:43.560445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:20:43.560445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:43.560445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:20:43.560445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:43.560445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:20:43.560445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:20:43.560953 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:43.560953 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:20:43.560953 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:43.560953 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:20:43.561133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:43.561133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:20:43.561133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:20:43.561133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:20:43.561133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:20:43.561133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:20:43.561133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:20:43.561133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:20:43.561133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:20:43.561133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:20:43.561133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:20:43.561133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:20:43.561133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:20:43.561133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:20:43.561133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:20:43.561133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:20:43.561729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:20:43.561729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:20:43.561729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:20:43.561729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:20:43.561729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:20:43.561729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:43.561729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:20:43.561729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:43.561729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:20:43.561729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:43.561729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:20:43.561729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:43.561729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:20:43.561729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:43.561729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:20:43.561729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:20:43.561729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:43.561729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:20:43.562224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:43.562224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:20:43.562224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:20:43.562224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:43.562224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:20:43.562224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:43.562224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:20:43.563025 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:20:43.563282 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:20:43.563405 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:20:43.563544 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:20:43.564057 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:20:43.564313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:43.564943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:20:43.564943 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:43.564943 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:20:43.564943 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:20:43.564943 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:43.564943 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:20:43.564943 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:43.564943 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:20:43.565951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:43.565951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:20:43.565951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:20:43.565951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:20:43.565951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:43.565951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:20:43.565951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:43.565951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:20:43.565951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:43.565951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:20:43.565951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:43.565951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:20:43.565951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:43.565951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:20:43.565951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:20:43.566402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:43.566402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:20:43.566402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:43.566402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:20:43.566402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:20:43.566402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:43.566402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:20:43.566402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:43.566402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:20:43.567071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:43.567071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:20:43.567071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:20:43.567071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:20:43.567071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:20:43.567071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:20:43.567071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:20:43.567071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:20:43.567071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:20:43.567071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:43.567071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:20:43.567593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:43.567593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:20:43.567593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:43.567593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:20:43.567593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:43.567593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:20:43.567593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:43.567593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:20:43.567593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:20:43.567593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:43.567593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:20:43.567593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:43.567593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:20:43.567593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:20:43.567979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:43.567979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:20:43.567979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:43.567979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:20:43.568199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:43.568199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:20:43.568199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:20:43.568199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:20:43.568199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:20:43.568199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:20:43.568199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:20:43.568199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:20:43.568199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:20:43.568199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:20:43.568199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:20:43.568199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:20:43.568199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:20:43.568199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:20:43.568199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:20:43.568199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:20:43.568199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:20:43.569052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:20:43.569052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:20:43.569052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:20:43.569052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:43.569052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:20:43.569052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:43.569052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:20:43.569052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:43.569052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:20:43.569052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:43.569052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:20:43.569052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:43.569052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:20:43.569052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:20:43.569052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:43.569052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:20:43.569052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:43.569052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:20:43.569052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:20:43.569936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:43.569936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:20:43.569936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:43.569936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:20:43.571833 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:20:43.572122 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:20:43.572283 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:20:43.572491 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:20:43.573250 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:20:43.573639 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:20:43.573895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:43.574714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:20:43.574714 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:43.574714 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:20:43.574714 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:20:43.574714 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:43.574714 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:20:43.574714 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:43.574714 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:20:43.575827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:43.575827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:20:43.575827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:20:43.575827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:20:43.575827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:20:43.575827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:20:43.575827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:20:43.575827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:43.575827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:20:43.575827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:43.575827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:20:43.575827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:43.576241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:20:43.576241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:43.576241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:20:43.576241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:43.576241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:20:43.576241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:20:43.576241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:43.576241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:20:43.576241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:43.576241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:20:43.576241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:20:43.576241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:43.576241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:20:43.576570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:43.576570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:20:43.577041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:43.577041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:20:43.577041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:20:43.577041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:20:43.577041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:20:43.577041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:20:43.577041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:20:43.577041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:20:43.577041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:20:43.577041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:20:43.577041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:20:43.577041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:20:43.577041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:20:43.577041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:20:43.577041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:20:43.577041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:20:43.577474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:20:43.577474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:20:43.577474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:43.577474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:20:43.577474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:43.577474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:20:43.577474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:43.577474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:20:43.577474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:43.577474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:20:43.577474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:43.577474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:20:43.577474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:20:43.577474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:43.577474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:20:43.577474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:43.577474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:20:43.577474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:20:43.577474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:43.578011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:20:43.578011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:43.578011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:20:48.382306 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:20:48.383010 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:20:48.383010 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:20:48.383331 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:20:48.409980 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:20:48.410242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:48.411036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:20:48.411036 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:48.411036 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:20:48.411036 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:20:48.411036 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:48.411036 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:20:48.411036 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:48.411036 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:20:48.412073 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.028 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:48.412073 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:20:48.412073 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:20:48.412073 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:20:48.412073 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:20:48.412073 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:20:48.412073 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:20:48.412073 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:48.412073 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:20:48.412073 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:48.412073 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:20:48.412073 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:48.412330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:20:48.412330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:48.412330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:20:48.412330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:48.412330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:20:48.412330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:20:48.412330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:48.412330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:20:48.412330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:48.412330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:20:48.412330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:20:48.412330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:48.412330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:20:48.412571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:48.412571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:20:48.413000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:48.413000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:20:48.413000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:20:48.413000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:20:48.413000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:20:48.413000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:20:48.413000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:20:48.413000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:20:48.413000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:20:48.413000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:20:48.413000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:20:48.413000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:20:48.413000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:20:48.413000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:20:48.413000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:20:48.413000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:20:48.413487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:20:48.413487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:20:48.413487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:48.413487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:20:48.413487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:48.413487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:20:48.413487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:48.413487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:20:48.413487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:48.413487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:20:48.413487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:48.413487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:20:48.413487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:20:48.413487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:48.413487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:20:48.413487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:48.413487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:20:48.413487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:20:48.413487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:48.414109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:20:48.414109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:48.414109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:20:48.547061 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:20:48.547663 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:20:48.547981 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:20:48.550360 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:20:48.550695 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:20:48.550695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:48.552149 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:20:48.552149 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:48.552149 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:20:48.552149 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:20:48.552149 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:48.552149 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:20:48.552149 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:48.552149 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:20:48.555541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:48.555541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:20:48.555541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:20:48.555541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:20:48.555541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:20:48.555541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:20:48.555541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:20:48.555541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:48.555541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:20:48.555541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:48.555541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:20:48.555541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:48.556138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:20:48.556138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:48.556138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:20:48.556138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:48.556138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:20:48.556138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:20:48.556138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:48.556138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:20:48.556138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:48.556138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:20:48.556138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:20:48.556138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:48.556138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:20:48.556657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:48.556657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:20:48.556657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:48.556657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:20:48.556657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:20:48.556657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:20:48.556657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:20:48.556657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:20:48.556657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:20:48.556657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:20:48.556657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:20:48.556657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:20:48.556657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:20:48.556657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:20:48.556657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:20:48.557174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:20:48.557174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:20:48.557174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:20:48.557174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:20:48.557174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:20:48.557174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:48.557174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:20:48.557174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:48.557174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:20:48.557174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:48.557174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:20:48.557174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:48.557174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:20:48.557174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:48.557174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:20:48.557174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:20:48.557174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:48.557174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:20:48.557174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:48.557664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:20:48.557664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:20:48.557664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:48.557664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:20:48.557664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:48.557664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:20:48.557664 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:20:48.557664 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:20:48.557664 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:20:48.557664 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:20:48.558085 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:20:48.558413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:48.559248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:20:48.559248 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:48.559248 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:20:48.559248 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:20:48.559248 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:48.559248 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:20:48.559248 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:48.559248 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:20:48.560323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:48.560323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:20:48.560323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:20:48.560323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:20:48.560323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:48.560323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:20:48.560323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:48.560323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:20:48.560323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:48.560323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:20:48.560323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:48.560323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:20:48.560323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:48.560323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:20:48.560323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:20:48.560726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:48.560726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:20:48.560726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:48.560726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:20:48.560726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:20:48.560726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:48.560726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:20:48.560726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:48.560726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:20:48.561318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:48.561318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:20:48.561318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:20:48.561318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:20:48.561318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:20:48.561318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:20:48.561318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:20:48.561318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:20:48.561318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:20:48.561318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:20:48.561318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:48.561318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:20:48.561660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:48.561660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:20:48.561660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:48.561660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:20:48.561660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:48.561660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:20:48.561660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:48.561660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:20:48.561660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:20:48.561660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:48.561660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:20:48.561660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:48.561660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:20:48.561660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:20:48.562053 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:48.562053 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:20:48.562053 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:48.562053 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:20:48.562523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:48.562523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:20:48.562523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:20:48.562523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:20:48.562523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:20:48.562523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:20:48.562523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:20:48.562523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:20:48.562523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:20:48.562523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:20:48.562523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:20:48.562523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:20:48.562523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:20:48.562523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:20:48.562523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:20:48.562523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:20:48.563167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:20:48.563167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:20:48.563167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:20:48.563167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:20:48.563167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:20:48.563167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:48.563167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:20:48.563167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:48.563167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:20:48.563167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:48.563167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:20:48.563167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:48.563167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:20:48.563167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:48.563167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:20:48.563167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:20:48.563167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:48.563167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:20:48.563631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:48.563631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:20:48.563631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:20:48.563631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:48.563631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:20:48.563631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:48.563631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:20:48.563631 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:20:48.563900 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:20:48.564390 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:20:48.564390 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:20:48.564661 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:20:48.564947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:48.566594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:20:48.566594 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:48.566594 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:20:48.566594 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:20:48.566594 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:48.566594 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:20:48.566594 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:48.566594 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:20:48.566594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:48.566594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:20:48.566594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:20:48.566594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:20:48.566594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:48.566594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:20:48.566594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:48.566594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:20:48.566594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:48.566594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:20:48.567138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:48.567138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:20:48.567138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:48.567138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:20:48.567138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:20:48.567138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:48.567138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:20:48.567138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:48.567138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:20:48.567138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:20:48.567138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:48.567138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:20:48.567138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:48.567138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:20:48.567500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:48.567500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:20:48.567500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:20:48.567500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:20:48.567500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:20:48.567500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:20:48.567500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:20:48.567500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:20:48.567500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:20:48.567500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:48.567500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:20:48.567876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:48.567876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:20:48.567876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:48.567876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:20:48.567876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:48.567876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:20:48.567876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:48.567876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:20:48.567876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:20:48.567876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:48.567876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:20:48.567876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:48.567876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:20:48.567876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:20:48.568242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:48.568242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:20:48.568242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:48.568242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:20:48.568698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:48.568698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:20:48.568698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:20:48.568698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:20:48.568698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:20:48.568698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:20:48.568698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:20:48.568698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:20:48.568698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:20:48.568698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:20:48.568698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:20:48.568698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:20:48.568698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:20:48.568698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:20:48.568698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:20:48.568698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:20:48.568698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:20:48.569178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:20:48.569178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:20:48.569178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:20:48.569178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:48.569178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:20:48.569178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:48.569178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:20:48.569178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:48.569178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:20:48.569178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:48.569178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:20:48.569178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:48.569178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:20:48.569178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:20:48.569178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:48.569178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:20:48.569178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:48.569178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:20:48.569178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:20:48.569651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:48.569651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:20:48.569651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:48.569651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:20:48.572627 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:20:48.572737 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:20:48.572927 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:20:48.573159 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:20:48.574243 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:20:48.574936 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:20:48.575041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:48.576226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:20:48.576226 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:48.576226 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:20:48.576226 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:20:48.576226 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:48.576226 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:20:48.576226 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:48.576226 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:20:48.577332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:48.577332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:20:48.577332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:20:48.577332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:20:48.577332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:20:48.577332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:20:48.577332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:20:48.577332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:48.577332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:20:48.577332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:48.577332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:20:48.577332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:48.577571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:20:48.577571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:48.577571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:20:48.577571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:48.577571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:20:48.577571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:20:48.577571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:48.577571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:20:48.577571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:48.577571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:20:48.577571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:20:48.577571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:48.577571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:20:48.577836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:48.577836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:20:48.578417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:48.578417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:20:48.578417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:20:48.578417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:20:48.578417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:20:48.578417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:20:48.578417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:20:48.578417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:20:48.578417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:20:48.578417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:20:48.578417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:20:48.578417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:20:48.578417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:20:48.578417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:20:48.578417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:20:48.578417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:20:48.578830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:20:48.578830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:20:48.578830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:48.578830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:20:48.578830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:48.578830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:20:48.578830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:48.578830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:20:48.578830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:48.578830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:20:48.578830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:48.578830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:20:48.578830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:20:48.578830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:48.578830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:20:48.578830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:48.578830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:20:48.578830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:20:48.578830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:48.579180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:20:48.579180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:48.579180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:20:53.383161 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:20:53.384136 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:20:53.384136 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:20:53.385055 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:20:53.424311 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:20:53.424655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:53.425520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:20:53.425520 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:53.425520 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:20:53.425520 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:20:53.425520 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:53.425520 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:20:53.425520 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:53.425520 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:20:53.426548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:53.426548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:20:53.426548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:20:53.426548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:20:53.426548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:20:53.426548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:20:53.426548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:20:53.426548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:53.426548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:20:53.426548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:53.426548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:20:53.426548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:53.427210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:20:53.427210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:53.427210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:20:53.427210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:53.427210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:20:53.427210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:20:53.427210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:53.427210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:20:53.427210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:53.427210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:20:53.427210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:20:53.427210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:53.427210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:20:53.427532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:53.427532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:20:53.427532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:53.427532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:20:53.427532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:20:53.427532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:20:53.427532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:20:53.427532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:20:53.427532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:20:53.427532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:20:53.427532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:20:53.427532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:20:53.427532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:20:53.427532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:20:53.427532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:20:53.428016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:20:53.428016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:20:53.428016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:20:53.428016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:20:53.428016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:20:53.428016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:53.428016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:20:53.428016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:53.428016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:20:53.428016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:53.428016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:20:53.428016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:53.428016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:20:53.428016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:53.428016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:20:53.428016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:20:53.428016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:53.428016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:20:53.428016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:53.428514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:20:53.428514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:20:53.428514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:53.428514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:20:53.428514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:53.428514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:20:53.547916 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:20:53.548410 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:20:53.550433 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:20:53.550433 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:20:53.550433 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:20:53.550433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:53.550433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:20:53.550433 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:53.550433 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:20:53.550433 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:20:53.550433 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:53.550433 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:20:53.550433 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:53.550433 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:20:53.551593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:53.551593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:20:53.551593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:20:53.551593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:20:53.551593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:20:53.551593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:20:53.551593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:20:53.551593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:53.551593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:20:53.551593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:53.551593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:20:53.551593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:53.552066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:20:53.552066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:53.552066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:20:53.552066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:53.552066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:20:53.552066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:20:53.552066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:53.552066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:20:53.552066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:53.552066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:20:53.552066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:20:53.552066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:53.552066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:20:53.552405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:53.552405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:20:53.552786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:53.552786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:20:53.552786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:20:53.552786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:20:53.552786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:20:53.552786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:20:53.552786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:20:53.552786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:20:53.552786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:20:53.552786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:20:53.552786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:20:53.552786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:20:53.552786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:20:53.552786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:20:53.552786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:20:53.552786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:20:53.553195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:20:53.553195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:20:53.553195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:53.553195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:20:53.553195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:53.553195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:20:53.553195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:53.553195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:20:53.553195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:53.553195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:20:53.553195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:53.553195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:20:53.553195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:20:53.553195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:53.553195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:20:53.553195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:53.553195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:20:53.553195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:20:53.553195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:53.553821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:20:53.553821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:53.553821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:20:53.556813 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:20:53.557055 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:20:53.557169 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:20:53.557307 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:20:53.558125 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:20:53.558435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:53.559567 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:20:53.559567 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:53.559567 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:20:53.559567 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:20:53.559567 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:53.559567 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:20:53.559567 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:53.559567 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:20:53.560407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:53.560407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:20:53.560407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:20:53.560407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:20:53.560407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:53.560407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:20:53.560407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:53.560407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:20:53.560407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:53.560407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:20:53.560407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:53.560407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:20:53.560407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:53.560407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:20:53.560407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:20:53.560813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:53.560813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:20:53.560813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:53.560813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:20:53.560813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:20:53.560813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:53.560813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:20:53.560813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:53.560813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:20:53.561364 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:53.561364 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:20:53.561364 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:20:53.561364 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:20:53.561364 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:20:53.561364 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:20:53.561364 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:20:53.561364 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:20:53.561364 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:20:53.561364 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:20:53.561364 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:53.561364 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:20:53.561661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:53.561661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:20:53.561661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:53.561661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:20:53.561661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:53.561661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:20:53.561661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:53.561661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:20:53.561661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:20:53.561661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:53.561661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:20:53.561661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:53.561661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:20:53.561661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:20:53.562565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:53.562565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:20:53.562565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:53.562565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:20:53.562565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:53.562565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:20:53.562565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:20:53.562565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:20:53.562565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:20:53.562565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:20:53.562565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:20:53.562565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:20:53.562565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:20:53.562565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:20:53.562923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:20:53.562923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:20:53.562923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:20:53.562923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:20:53.562923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:20:53.562923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:20:53.562923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:20:53.562923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:20:53.562923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:20:53.562923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:20:53.562923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:20:53.562923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:53.562923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:20:53.562923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:53.562923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:20:53.562923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:53.562923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:20:53.562923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:53.562923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:20:53.563306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:53.563306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:20:53.563306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:20:53.563306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:53.563306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:20:53.563306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:53.563306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:20:53.563306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:20:53.563306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:53.563306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:20:53.563306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:53.563306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:20:53.564560 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:20:53.564560 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:20:53.564560 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:20:53.564560 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:20:53.565101 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:20:53.565340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:53.566066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:20:53.566066 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:53.566066 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:20:53.566066 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:20:53.566066 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:53.566066 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:20:53.566066 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:53.566066 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:20:53.567063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:53.567063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:20:53.567063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:20:53.567063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:20:53.567063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:53.567063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:20:53.567063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:53.567063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:20:53.567063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:53.567063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:20:53.567063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:53.567063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:20:53.567063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:53.567063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:20:53.567063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:20:53.567539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:53.567539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:20:53.567539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:53.567539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:20:53.567539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:20:53.567539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:53.567539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:20:53.567539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:53.567539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:20:53.568777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:53.568777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:20:53.568777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:20:53.568777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:20:53.568777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:20:53.568777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:20:53.568777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:20:53.568777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:20:53.568777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:20:53.568777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:53.568777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:20:53.569174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:53.569174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:20:53.569174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:53.569174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:20:53.569174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:53.569174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:20:53.569174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:53.569174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:20:53.569174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:20:53.569174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:53.569174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:20:53.569174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:53.569174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:20:53.569174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:20:53.569587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:53.569587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:20:53.569587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:53.569587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:20:53.569587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:53.569587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:20:53.569587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:20:53.569587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:20:53.569587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:20:53.569587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:20:53.569587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:20:53.569587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:20:53.569587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:20:53.569587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:20:53.570037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:20:53.570037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:20:53.570037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:20:53.570037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:20:53.570037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:20:53.570037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:20:53.570037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:20:53.570037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:20:53.570037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:20:53.570037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:20:53.570037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:53.570037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:20:53.570037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:53.570037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:20:53.570037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:53.570037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:20:53.570037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:53.570037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:20:53.570037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:53.570540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:20:53.570540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:20:53.570540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:53.570540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:20:53.570540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:53.570540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:20:53.570540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:20:53.570540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:53.570540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:20:53.570540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:53.570540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:20:53.573258 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:20:53.573258 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:20:53.573352 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:20:53.573475 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:20:53.574095 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:20:53.574617 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:20:53.574617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:53.575051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:20:53.575051 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:53.575051 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:20:53.575051 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:20:53.575051 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:53.575051 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:20:53.575051 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:53.575051 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:20:53.576257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:53.576257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:20:53.576257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:20:53.576257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:20:53.576257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:20:53.576257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:20:53.576257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:20:53.576257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:53.576257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:20:53.576257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:53.576257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:20:53.576257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:53.576549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:20:53.576549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:53.576549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:20:53.576549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:53.576549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:20:53.576549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:20:53.576549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:53.576549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:20:53.576549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:53.576549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:20:53.576549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:20:53.576549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:53.576549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:20:53.576859 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:53.576859 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:20:53.577264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:53.577264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:20:53.577264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:20:53.577264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:20:53.577264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:20:53.577264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:20:53.577264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:20:53.577264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:20:53.577264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:20:53.577264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:20:53.577264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:20:53.577264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:20:53.577264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:20:53.577264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:20:53.577264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:20:53.577264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:20:53.577581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:20:53.577581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:20:53.577581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:53.577581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:20:53.577581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:53.577581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:20:53.577581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:53.577581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:20:53.577581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:53.577581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:20:53.577581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:53.577581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:20:53.577581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:20:53.577581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:53.577581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:20:53.577581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:53.577581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:20:53.577581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:20:53.577581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:53.578024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:20:53.578024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:53.578024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:20:58.384059 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:20:58.387396 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:20:58.387396 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:20:58.387396 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:20:58.389658 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG dbcounter [-] [106314] Writing DB stats neutron:SELECT=260 {{(pid=106314) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 09 09:20:58.429287 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:20:58.429856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:58.432928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:20:58.432928 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:58.432928 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:20:58.432928 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:20:58.432928 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:58.432928 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:20:58.432928 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:58.432928 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:20:58.432928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.046 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:58.432928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:20:58.432928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:20:58.432928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:20:58.432928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:20:58.432928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:20:58.432928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:20:58.433385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:58.433385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:20:58.433385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:58.433385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:20:58.433385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:58.433385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:20:58.433385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:58.433385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:20:58.433385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:58.433385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:20:58.433385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:20:58.433385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:58.433385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:20:58.433385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:58.433648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:20:58.433648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:20:58.433648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:58.433648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:20:58.433648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:58.433648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:20:58.433648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:58.433648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:20:58.433648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:20:58.433648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:20:58.433648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:20:58.433648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:20:58.433648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:20:58.433648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:20:58.434012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:20:58.434012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:20:58.434012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:20:58.434012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:20:58.434012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:20:58.434012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:20:58.434012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:20:58.434012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:20:58.434012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:20:58.434012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:20:58.434012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:58.434012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:20:58.434012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:58.434012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:20:58.434012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:58.434012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:20:58.434012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:58.434012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:20:58.434012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:58.434012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:20:58.434012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:20:58.434856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:58.434856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:20:58.434856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:58.434856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:20:58.434856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:20:58.434856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:58.434856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:20:58.434856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:58.434856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:20:58.548973 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:20:58.550307 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:20:58.550307 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:20:58.550958 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:20:58.551206 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:20:58.551376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:58.555091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:20:58.555091 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:58.555091 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:20:58.555091 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:20:58.555091 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:58.555091 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:20:58.555091 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:58.555091 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:20:58.555091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:58.555091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:20:58.555091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:20:58.555091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:20:58.555091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:20:58.555091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:20:58.555091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:20:58.556101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:58.556101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:20:58.556101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:58.556101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:20:58.556101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:58.556101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:20:58.556101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:58.556101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:20:58.556101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:58.556101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:20:58.556101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:20:58.556101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:58.556101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:20:58.556101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:58.557100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:20:58.557100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:20:58.557100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:58.557100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:20:58.557100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:58.557100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:20:58.557100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:58.557100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:20:58.557100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:20:58.557100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:20:58.557100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:20:58.557100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:20:58.557100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:20:58.557100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:20:58.557562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:20:58.557562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:20:58.557562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:20:58.557562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:20:58.557562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:20:58.557562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:20:58.557562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:20:58.557562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:20:58.557562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:20:58.557562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:20:58.557562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:58.557562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:20:58.557562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:58.557562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:20:58.557562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:58.557562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:20:58.557562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:58.557562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:20:58.557562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:58.557562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:20:58.558173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:20:58.558173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:58.558173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:20:58.558173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:58.558173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:20:58.558173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:20:58.558173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:58.558173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:20:58.558173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:58.558173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:20:58.558173 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:20:58.559302 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:20:58.562783 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:20:58.562783 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:20:58.562783 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:20:58.562783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:58.562783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:20:58.562783 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:58.562783 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:20:58.562783 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:20:58.562783 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:58.562783 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:20:58.562783 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:58.562783 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:20:58.563182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:58.563182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:20:58.563182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:20:58.563182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:20:58.563182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:58.563182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:20:58.563182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:58.563182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:20:58.563182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:58.563182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:20:58.563182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:58.563182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:20:58.563182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:58.563182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:20:58.563182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:20:58.563600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:58.563600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:20:58.563600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:58.563600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:20:58.563600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:20:58.563600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:58.563600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:20:58.563600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:58.563600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:20:58.563862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:58.563862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:20:58.563862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:20:58.563862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:20:58.563862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:20:58.563862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:20:58.563862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:20:58.563862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:20:58.563862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:20:58.563862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:20:58.563862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:58.563862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:20:58.564195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:58.564195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:20:58.564195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:58.564195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:20:58.564195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:58.564195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:20:58.564195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:58.564195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:20:58.564195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:20:58.564195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:58.564195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:20:58.564195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:58.564195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:20:58.564195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:20:58.564530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:58.564530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:20:58.564530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:58.564530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:20:58.564935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:58.564935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:20:58.564935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:20:58.564935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:20:58.564935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:20:58.564935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:20:58.564935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:20:58.564935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:20:58.564935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:20:58.564935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:20:58.564935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:20:58.564935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:20:58.564935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:20:58.564935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:20:58.564935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:20:58.564935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:20:58.565294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:20:58.565294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:20:58.565294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:20:58.565294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:20:58.565294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:20:58.565294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:58.565294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:20:58.565294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:58.565294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:20:58.565294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:58.565294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:20:58.565294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:58.565294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:20:58.565294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:58.565294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:20:58.565294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:20:58.565294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:58.565294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:20:58.565612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:58.565612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:20:58.565612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:20:58.565612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:58.565612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:20:58.565612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:58.565612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:20:58.566320 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:20:58.566677 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:20:58.566800 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:20:58.566943 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:20:58.567481 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:20:58.567851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:58.568438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:20:58.568438 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:58.568438 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:20:58.568438 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:20:58.568438 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:58.568438 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:20:58.568438 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:58.568438 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:20:58.570321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:58.570321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:20:58.570321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:20:58.570321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:20:58.570321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:58.570321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:20:58.570321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:58.570321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:20:58.570321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:58.570321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:20:58.570321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:58.570321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:20:58.570321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:58.570321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:20:58.570321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:20:58.570767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:58.570767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:20:58.570767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:58.570767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:20:58.570767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:20:58.570767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:58.570767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:20:58.570767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:58.570767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:20:58.571262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:58.571262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:20:58.571262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:20:58.571262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:20:58.571262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:20:58.571262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:20:58.571262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:20:58.571262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:20:58.571262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:20:58.571262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:58.571262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:20:58.571549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:58.571549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:20:58.571549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:58.571549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:20:58.571549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:58.571549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:20:58.571549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:58.571549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:20:58.571549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:20:58.571549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:58.571549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:20:58.571549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:58.571549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:20:58.571549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:20:58.571838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:58.571838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:20:58.571838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:58.571838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:20:58.572447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:58.572447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:20:58.572447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:20:58.572447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:20:58.572447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:20:58.572447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:20:58.572447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:20:58.572447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:20:58.572447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:20:58.572447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:20:58.572447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:20:58.572447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:20:58.572447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:20:58.572447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:20:58.572447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:20:58.572447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:20:58.572447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:20:58.572841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:20:58.572841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:20:58.572841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:20:58.572841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:58.572841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:20:58.572841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:58.572841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:20:58.572841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:58.572841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:20:58.572841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:58.572841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:20:58.572841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:58.572841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:20:58.572841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:20:58.572841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:58.572841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:20:58.572841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:58.572841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:20:58.572841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:20:58.573221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:58.573221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:20:58.573221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:58.573221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:20:58.573486 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:20:58.573933 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:20:58.574061 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:20:58.574239 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:20:58.574854 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:20:58.575145 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:20:58.575311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:58.575911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:20:58.575911 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:58.575911 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:20:58.575911 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:20:58.575911 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:58.575911 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:20:58.575911 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:58.575911 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:20:58.576792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:58.576792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:20:58.576792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:20:58.576792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:20:58.576792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:20:58.576792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:20:58.576792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:20:58.576792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:58.576792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:20:58.576792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:58.576792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:20:58.576792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:58.577069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:20:58.577069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:58.577069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:20:58.577069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:58.577069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:20:58.577069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:20:58.577069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:58.577069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:20:58.577069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:58.577069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:20:58.577069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:20:58.577069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:58.577069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:20:58.577330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:58.577330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:20:58.577849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:58.577849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:20:58.577849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:20:58.577849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:20:58.577849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:20:58.577849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:20:58.577849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:20:58.577849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:20:58.577849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:20:58.577849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:20:58.577849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:20:58.577849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:20:58.577849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:20:58.577849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:20:58.577849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:20:58.577849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:20:58.578188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:20:58.578188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:20:58.578188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:58.578188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:20:58.578188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:20:58.578188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:20:58.578188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:20:58.578188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:20:58.578188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:20:58.578188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:20:58.578188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:20:58.578188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:20:58.578188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:20:58.578188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:20:58.578188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:20:58.578188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:20:58.578188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:20:58.578188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:20:58.578188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:20:58.578582 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:20:58.578582 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:20:58.578582 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:21:03.385716 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:21:03.386267 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:21:03.386267 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:21:03.387554 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:21:03.436503 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:21:03.436700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:03.437444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:21:03.437444 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:03.437444 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:21:03.437444 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:21:03.437444 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:03.437444 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:21:03.437444 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:03.437444 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:21:03.438904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.051 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:03.438904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:21:03.438904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:21:03.438904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:21:03.438904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:21:03.438904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:21:03.438904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:21:03.438904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:03.438904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:21:03.438904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:03.438904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:21:03.438904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:03.439295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:21:03.439295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:03.439295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:21:03.439295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:03.439295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:21:03.439295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:21:03.439295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:03.439295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:21:03.439295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:03.439295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:21:03.439295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:21:03.439295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:03.439295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:21:03.439574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:03.439574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:21:03.440682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:03.440682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:21:03.440682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:21:03.440682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:21:03.440682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:21:03.440682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:21:03.440682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:21:03.440682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:21:03.440682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:21:03.440682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:21:03.440682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:21:03.440682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:21:03.440682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:21:03.440682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:21:03.440682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:21:03.440682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:21:03.441089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:21:03.441089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:21:03.441089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:03.441089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:21:03.441089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:03.441089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:21:03.441089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:03.441089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:21:03.441089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:03.441089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:21:03.441089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:03.441089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:21:03.441089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:21:03.441089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:03.441089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:21:03.441089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:03.441089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:21:03.441089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:21:03.441089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:03.441466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:21:03.441466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:03.441466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:21:03.549600 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:21:03.550267 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:21:03.550466 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:21:03.551648 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:21:03.551959 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:21:03.552238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:03.553164 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:21:03.553164 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:03.553164 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:21:03.553164 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:21:03.553164 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:03.553164 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:21:03.553164 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:03.553164 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:21:03.555053 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:03.555053 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:21:03.555053 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:21:03.555053 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:21:03.555053 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:21:03.555053 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:21:03.555053 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:21:03.555053 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:03.555053 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:21:03.555053 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:03.555053 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:21:03.555053 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:03.555726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:21:03.555726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:03.555726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:21:03.555726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:03.555726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:21:03.555726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:21:03.555726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:03.555726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:21:03.555726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:03.555726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:21:03.555726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:21:03.555726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:03.555726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:21:03.556237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:03.556237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:21:03.557106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:03.557106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:21:03.557106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:21:03.557106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:21:03.557106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:21:03.557106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:21:03.557106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:21:03.557106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:21:03.557106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:21:03.557106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:21:03.557106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:21:03.557106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:21:03.557106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:21:03.557106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:21:03.557106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:21:03.557106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:21:03.558046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:21:03.558046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:21:03.558046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:03.558046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:21:03.558046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:03.558046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:21:03.558046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:03.558046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:21:03.558046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:03.558046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:21:03.558046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:03.558046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:21:03.558046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:21:03.558046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:03.558046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:21:03.558046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:03.558046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:21:03.558046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:21:03.558046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:03.558604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:21:03.558604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:03.558604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:21:03.559382 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:21:03.559713 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:21:03.559865 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:21:03.559968 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:21:03.560438 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:21:03.560663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:03.561391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:21:03.561391 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:03.561391 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:21:03.561391 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:21:03.561391 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:03.561391 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:21:03.561391 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:03.561391 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:21:03.562323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:03.562323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:21:03.562323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:21:03.562323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:21:03.562323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:03.562323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:21:03.562323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:03.562323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:21:03.562323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:03.562323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:21:03.562323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:03.562323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:21:03.562323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:03.562323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:21:03.562323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:21:03.562849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:03.562849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:21:03.562849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:03.562849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:21:03.562849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:21:03.562849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:03.562849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:21:03.562849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:03.562849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:21:03.563760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:03.563760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:21:03.563760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:21:03.563760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:21:03.563760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:21:03.563760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:21:03.563760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:21:03.563760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:21:03.563760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:21:03.563760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:21:03.563760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:03.563760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:21:03.564174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:03.564174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:21:03.564174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:03.564174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:21:03.564174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:03.564174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:21:03.564174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:03.564174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:21:03.564174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:21:03.564174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:03.564174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:21:03.564174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:03.564174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:21:03.564174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:21:03.564583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:03.564583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:21:03.564583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:03.564583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:21:03.565146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:03.565146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:21:03.565146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:21:03.565146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:21:03.565146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:21:03.565146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:21:03.565146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:21:03.565146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:21:03.565146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:21:03.565146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:21:03.565146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:21:03.565146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:21:03.565146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:21:03.565146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:21:03.565146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:21:03.565146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:21:03.565703 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:21:03.565703 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:21:03.565703 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:21:03.565703 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:21:03.565703 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:21:03.565703 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:03.565703 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:21:03.565703 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:03.565703 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:21:03.565703 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:03.565703 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:21:03.565703 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:03.565703 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:21:03.565703 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:03.565703 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:21:03.565703 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:21:03.565703 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:03.565703 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:21:03.566288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:03.566288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:21:03.566288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:21:03.566288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:03.566288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:21:03.566288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:03.566288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:21:03.567081 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:21:03.567332 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:21:03.567428 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:21:03.567642 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:21:03.568099 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:21:03.568620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:03.568849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:21:03.568849 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:03.568849 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:21:03.568849 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:21:03.568849 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:03.568849 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:21:03.568849 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:03.568849 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:21:03.569669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:03.569669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:21:03.569669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:21:03.569669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:21:03.569669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:03.569669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:21:03.569669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:03.569669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:21:03.569669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:03.569669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:21:03.569669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:03.569669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:21:03.569669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:03.569669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:21:03.569669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:21:03.570216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:03.570216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:21:03.570216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:03.570216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:21:03.570216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:21:03.570216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:03.570216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:21:03.570216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:03.570216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:21:03.570625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:03.570625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:21:03.570625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:21:03.570625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:21:03.570625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:21:03.570625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:21:03.570625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:21:03.570625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:21:03.570625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:21:03.570625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:03.570625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:21:03.571073 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:03.571073 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:21:03.571073 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:03.571073 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:21:03.571073 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:03.571073 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:21:03.571073 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:03.571073 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:21:03.571073 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:21:03.571073 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:03.571073 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:21:03.571073 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:03.571073 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:21:03.571073 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:21:03.571547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:03.571547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:21:03.571547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:03.571547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:21:03.572120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:03.572120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:21:03.572120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:21:03.572120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:21:03.572120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:21:03.572120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:21:03.572120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:21:03.572120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:21:03.572120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:21:03.572120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:21:03.572120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:21:03.572120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:21:03.572120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:21:03.572120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:21:03.572120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:21:03.572120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:21:03.572120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:21:03.572633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:21:03.572633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:21:03.572633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:21:03.572633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:03.572633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:21:03.572633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:03.572633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:21:03.572633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:03.572633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:21:03.572633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:03.572633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:21:03.572633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:03.572633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:21:03.572633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:21:03.572633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:03.572633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:21:03.572633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:03.572633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:21:03.572633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:21:03.573205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:03.573205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:21:03.573205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:03.573205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:21:03.574562 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:21:03.574891 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:21:03.575040 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:21:03.575348 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:21:03.578030 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:21:03.578460 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:21:03.578529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:03.579435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:21:03.579435 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:03.579435 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:21:03.579435 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:21:03.579435 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:03.579435 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:21:03.579435 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:03.579435 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:21:03.580705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:03.580705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:21:03.580705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:21:03.580705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:21:03.580705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:21:03.580705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:21:03.580705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:21:03.580705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:03.580705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:21:03.580705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:03.580705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:21:03.580705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:03.581124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:21:03.581124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:03.581124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:21:03.581124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:03.581124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:21:03.581124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:21:03.581124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:03.581124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:21:03.581124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:03.581124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:21:03.581124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:21:03.581124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:03.581124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:21:03.581490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:03.581490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:21:03.581807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:03.581807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:21:03.581807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:21:03.581807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:21:03.581807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:21:03.581807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:21:03.581807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:21:03.581807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:21:03.581807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:21:03.581807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:21:03.581807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:21:03.581807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:21:03.581807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:21:03.581807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:21:03.581807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:21:03.581807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:21:03.582257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:21:03.582257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:21:03.582257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:03.582257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:21:03.582257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:03.582257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:21:03.582257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:03.582257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:21:03.582257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:03.582257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:21:03.582257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:03.582257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:21:03.582257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:21:03.582257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:03.582257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:21:03.582257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:03.582257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:21:03.582257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:21:03.582257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:03.582737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:21:03.582737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:03.582737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:21:08.386078 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:21:08.387046 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:21:08.387046 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:21:08.387200 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:21:08.420511 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:21:08.420688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:08.421386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:21:08.421386 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:08.421386 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:21:08.421386 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:21:08.421386 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:08.421386 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:21:08.421386 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:08.421386 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:21:08.422529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:08.422529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:21:08.422529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:21:08.422529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:21:08.422529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:21:08.422529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:21:08.422529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:21:08.422529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:08.422529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:21:08.422529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:08.422529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:21:08.422529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:08.423033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:21:08.423033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:08.423033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:21:08.423033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:08.423033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:21:08.423033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:21:08.423033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:08.423033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:21:08.423033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:08.423033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:21:08.423033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:21:08.423033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:08.423033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:21:08.423572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:08.423572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:21:08.423661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:08.423661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:21:08.423661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:21:08.423661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:21:08.423661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:21:08.423661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:21:08.423661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:21:08.423661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:21:08.423661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:21:08.423661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:21:08.423661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:21:08.423661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:21:08.423661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:21:08.423661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:21:08.423661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:21:08.423661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:21:08.424201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:21:08.424201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:21:08.424201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:08.424201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:21:08.424201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:08.424201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:21:08.424201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:08.424201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:21:08.424201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:08.424201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:21:08.424201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:08.424201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:21:08.424201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:21:08.424201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:08.424201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:21:08.424201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:08.424201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:21:08.424201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:21:08.424201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:08.424898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:21:08.424898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:08.424898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:21:08.550410 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:21:08.552183 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:21:08.552183 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:21:08.552609 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:21:08.552976 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:21:08.553097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:08.553813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:21:08.553813 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:08.553813 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:21:08.553813 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:21:08.553813 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:08.553813 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:21:08.553813 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:08.553813 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:21:08.554862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:08.554862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:21:08.554862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:21:08.554862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:21:08.554862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:21:08.554862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:21:08.554862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:21:08.554862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:08.554862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:21:08.554862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:08.554862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:21:08.554862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:08.555361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:21:08.555361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:08.555361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:21:08.555361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:08.555361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:21:08.555361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:21:08.555361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:08.555361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:21:08.555361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:08.555361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:21:08.555361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:21:08.555361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:08.555361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:21:08.555902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:08.555902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:21:08.555976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:08.555976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:21:08.555976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:21:08.555976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:21:08.555976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:21:08.555976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:21:08.555976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:21:08.555976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:21:08.555976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:21:08.555976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:21:08.555976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:21:08.555976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:21:08.555976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:21:08.555976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:21:08.555976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:21:08.555976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:21:08.556488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:21:08.556488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:21:08.556488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:08.556488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:21:08.556488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:08.556488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:21:08.556488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:08.556488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:21:08.556488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:08.556488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:21:08.556488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:08.556488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:21:08.556488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:21:08.556488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:08.556488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:21:08.556488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:08.556488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:21:08.556488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:21:08.556488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:08.557186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:21:08.557186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:08.557186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:21:08.560023 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:21:08.560390 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:21:08.560480 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:21:08.560613 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:21:08.561162 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:21:08.561346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:08.561883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:21:08.561883 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:08.561883 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:21:08.561883 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:21:08.561883 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:08.561883 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:21:08.561883 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:08.561883 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:21:08.563077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:08.563077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:21:08.563077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:21:08.563077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:21:08.563077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:08.563077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:21:08.563077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:08.563077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:21:08.563077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:08.563077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:21:08.563077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:08.563077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:21:08.563077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:08.563077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:21:08.563077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:21:08.563504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:08.563504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:21:08.563504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:08.563504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:21:08.563504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:21:08.563504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:08.563504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:21:08.563504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:08.563504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:21:08.564345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:08.564345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:21:08.564345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:21:08.564345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:21:08.564345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:21:08.564345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:21:08.564345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:21:08.564345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:21:08.564345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:21:08.564345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:21:08.564345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:08.564345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:21:08.564855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:08.564855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:21:08.564855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:08.564855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:21:08.564855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:08.564855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:21:08.564855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:08.564855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:21:08.564855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:21:08.564855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:08.564855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:21:08.564855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:08.564855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:21:08.564855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:21:08.567050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:08.567050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:21:08.567050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:08.567050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:21:08.567050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:08.567050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:21:08.567050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:21:08.567050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:21:08.567050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:21:08.567050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:21:08.567050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:21:08.567050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:21:08.567050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:21:08.567050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:21:08.567622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:21:08.567622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:21:08.567622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:21:08.567622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:21:08.567622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:21:08.567622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:21:08.567622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:21:08.567622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:21:08.567622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:21:08.567622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:21:08.567622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:21:08.567622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:08.567622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:21:08.567622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:08.567622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:21:08.567622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:08.567622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:21:08.567622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:08.567622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:21:08.569129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:08.569129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:21:08.569129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:21:08.569129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:08.569129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:21:08.569129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:08.569129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:21:08.569129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:21:08.569129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:08.569129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:21:08.569129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:08.569129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:21:08.569129 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:21:08.569129 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:21:08.569701 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:21:08.569701 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:21:08.569701 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:21:08.569701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:08.569701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:21:08.569701 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:08.569701 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:21:08.569701 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:21:08.569701 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:08.569701 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:21:08.569701 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:08.569701 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:21:08.570933 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:08.570933 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:21:08.570933 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:21:08.570933 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:21:08.570933 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:08.570933 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:21:08.570933 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:08.570933 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:21:08.570933 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:08.570933 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:21:08.570933 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:08.570933 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:21:08.570933 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:08.570933 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:21:08.570933 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:21:08.571439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:08.571439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:21:08.571439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:08.571439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:21:08.571439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:21:08.571439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:08.571439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:21:08.571439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:08.571439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:21:08.572581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:08.572581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:21:08.572581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:21:08.572581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:21:08.572581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:21:08.572581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:21:08.572581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:21:08.572581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:21:08.572581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:21:08.572581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:08.572581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:21:08.573015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:08.573015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:21:08.573015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:08.573015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:21:08.573015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:08.573015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:21:08.573015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:08.573015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:21:08.573015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:21:08.573015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:08.573015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:21:08.573015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:08.573015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:21:08.573015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:21:08.573385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:08.573385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:21:08.573385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:08.573385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:21:08.574040 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:08.574040 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:21:08.574040 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:21:08.574040 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:21:08.574040 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:21:08.574040 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:21:08.574040 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:21:08.574040 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:21:08.574040 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:21:08.574040 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:21:08.574040 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:21:08.574040 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:21:08.574040 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:21:08.574040 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:21:08.574040 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:21:08.574040 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:21:08.574040 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:21:08.574543 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:21:08.574543 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:21:08.574543 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:21:08.574543 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:08.574543 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:21:08.574543 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:08.574543 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:21:08.574543 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:08.574543 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:21:08.574543 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:08.574543 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:21:08.574543 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:08.574543 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:21:08.574543 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:21:08.574543 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:08.574543 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:21:08.574543 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:08.574543 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:21:08.574543 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:21:08.575082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:08.575082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:21:08.575082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:08.575082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:21:08.575082 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:21:08.575304 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:21:08.575416 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:21:08.575605 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:21:08.575989 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:21:08.576166 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:21:08.576842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:08.577301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:21:08.577301 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:08.577301 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:21:08.577301 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:21:08.577301 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:08.577301 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:21:08.577301 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:08.577301 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:21:08.578172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:08.578172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:21:08.578172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:21:08.578172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:21:08.578172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:21:08.578172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:21:08.578172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:21:08.578172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:08.578172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:21:08.578172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:08.578172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:21:08.578172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:08.578534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:21:08.578534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:08.578534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:21:08.578534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:08.578534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:21:08.578534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:21:08.578534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:08.578534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:21:08.578534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:08.578534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:21:08.578534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:21:08.578534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:08.578534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:21:08.578908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:08.578908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:21:08.579208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:08.579208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:21:08.579208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:21:08.579208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:21:08.579208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:21:08.579208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:21:08.579208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:21:08.579208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:21:08.579208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:21:08.579208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:21:08.579208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:21:08.579208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:21:08.579208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:21:08.579208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:21:08.579208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:21:08.579208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:21:08.579761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:21:08.579761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:21:08.579761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:08.579761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:21:08.579761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:08.579761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:21:08.579761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:08.579761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:21:08.579761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:08.579761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:21:08.579761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:08.579761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:21:08.579761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:21:08.579761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:08.579761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:21:08.579761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:08.579761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:21:08.579761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:21:08.579761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:08.580567 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:21:08.580567 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:08.580567 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:21:13.386666 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:21:13.387311 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:21:13.387490 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:21:13.388266 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:21:13.426328 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:21:13.426648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:13.427421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:21:13.427421 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:13.427421 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:21:13.427421 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:21:13.427421 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:13.427421 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:21:13.427421 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:13.427421 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:21:13.428555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:13.428555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:21:13.428555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:21:13.428555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:21:13.428555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:21:13.428555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:21:13.428555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:21:13.428555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:13.428555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:21:13.428555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:13.428555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:21:13.428555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:13.429703 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:21:13.429703 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:13.429703 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:21:13.429703 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:13.429703 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:21:13.429703 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:21:13.429703 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:13.429703 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:21:13.429703 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:13.429703 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:21:13.429703 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:21:13.429703 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:13.429703 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:21:13.430359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:13.430359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:21:13.430359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:13.430359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:21:13.430359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:21:13.430359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:21:13.430359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:21:13.430359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:21:13.430359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:21:13.430359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:21:13.430359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:21:13.430359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:21:13.430359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:21:13.430359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:21:13.430359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:21:13.431156 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:21:13.431156 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:21:13.431156 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:21:13.431156 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:21:13.431156 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:21:13.431156 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:13.431156 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:21:13.431156 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:13.431156 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:21:13.431156 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:13.431156 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:21:13.431156 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:13.431156 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:21:13.431156 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:13.431156 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:21:13.431156 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:21:13.431156 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:13.431156 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:21:13.431156 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:13.431579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:21:13.431579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:21:13.431579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:13.431579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:21:13.431579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:13.431579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:21:13.551814 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:21:13.552291 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:21:13.552291 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:21:13.553050 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:21:13.553277 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:21:13.553438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:13.554127 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:21:13.554127 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:13.554127 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:21:13.554127 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:21:13.554127 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:13.554127 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:21:13.554127 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:13.554127 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:21:13.555136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:13.555136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:21:13.555136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:21:13.555136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:21:13.555136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:21:13.555136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:21:13.555136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:21:13.555136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:13.555136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:21:13.555136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:13.555136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:21:13.555136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:13.555857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:21:13.555857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:13.555857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:21:13.555857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:13.555857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:21:13.555857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:21:13.555857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:13.555857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:21:13.555857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:13.555857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:21:13.555857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:21:13.555857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:13.555857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:21:13.556525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:13.556525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:21:13.556525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:13.556525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:21:13.556525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:21:13.556525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:21:13.556525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:21:13.556525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:21:13.556525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:21:13.556525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:21:13.556525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:21:13.556525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:21:13.556525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:21:13.556525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:21:13.556525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:21:13.557078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:21:13.557078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:21:13.557078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:21:13.557078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:21:13.557078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:21:13.557078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:13.557078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:21:13.557078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:13.557078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:21:13.557078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:13.557078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:21:13.557078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:13.557078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:21:13.557078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:13.557078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:21:13.557078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:21:13.557078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:13.557078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:21:13.557078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:13.557656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:21:13.557656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:21:13.557656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:13.557656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:21:13.557656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:13.557656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:21:13.560759 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:21:13.561253 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:21:13.561619 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:21:13.561729 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:21:13.562522 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:21:13.562849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:13.563523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:21:13.563523 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:13.563523 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:21:13.563523 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:21:13.563523 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:13.563523 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:21:13.563523 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:13.563523 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:21:13.564416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:13.564416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:21:13.564416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:21:13.564416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:21:13.564416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:13.564416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:21:13.564416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:13.564416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:21:13.564416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:13.564416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:21:13.564416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:13.564416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:21:13.564416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:13.564416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:21:13.564416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:21:13.566337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:13.566337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:21:13.566337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:13.566337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:21:13.566337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:21:13.566337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:13.566337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:21:13.566337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:13.566337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:21:13.566337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:13.566337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:21:13.566337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:21:13.566337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:21:13.566337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:21:13.567281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:21:13.567281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:21:13.567281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:21:13.567281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:21:13.567281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:21:13.567281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:13.567281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:21:13.567281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:13.567281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:21:13.567281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:13.567281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:21:13.567281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:13.567281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:21:13.567512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:13.567512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:21:13.567512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:21:13.567512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:13.567512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:21:13.567512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:13.567512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:21:13.567512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:21:13.567512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:13.567512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:21:13.567512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:13.567512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:21:13.567836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:13.567836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:21:13.567836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:21:13.567836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:21:13.567836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:21:13.567836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:21:13.567836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:21:13.567836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:21:13.567836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:21:13.567836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:21:13.567836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:21:13.567836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:21:13.567836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:21:13.567836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:21:13.567836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:21:13.567836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:21:13.568205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:21:13.568205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:21:13.568205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:21:13.568205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:21:13.568205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:21:13.568205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:13.568205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:21:13.568205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:13.568205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:21:13.568205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:13.568205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:21:13.568205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:13.568205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:21:13.568205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:13.568205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:21:13.568205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:21:13.568205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:13.568205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:21:13.568596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:13.568596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:21:13.568596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:21:13.568596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:13.568596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:21:13.568596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:13.568596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:21:13.568596 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:21:13.568596 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:21:13.568596 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:21:13.568812 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:21:13.569373 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:21:13.569445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:13.570294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:21:13.570294 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:13.570294 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:21:13.570294 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:21:13.570294 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:13.570294 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:21:13.570294 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:13.570294 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:21:13.570939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:13.570939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:21:13.570939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:21:13.570939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:21:13.570939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:13.570939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:21:13.570939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:13.570939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:21:13.570939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:13.570939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:21:13.570939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:13.570939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:21:13.570939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:13.570939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:21:13.570939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:21:13.571647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:13.571647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:21:13.571647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:13.571647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:21:13.571647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:21:13.571647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:13.571647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:21:13.571647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:13.571647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:21:13.572568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:13.572568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:21:13.572568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:21:13.572568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:21:13.572568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:21:13.572568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:21:13.572568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:21:13.572568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:21:13.572568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:21:13.572568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:13.572568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:21:13.573011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:13.573011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:21:13.573011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:13.573011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:21:13.573011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:13.573011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:21:13.573011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:13.573011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:21:13.573011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:21:13.573011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:13.573011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:21:13.573011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:13.573011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:21:13.573011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:21:13.574854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:13.574854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:21:13.574854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:13.574854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:21:13.574854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:13.574854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:21:13.574854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:21:13.574854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:21:13.574854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:21:13.574854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:21:13.574854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:21:13.574854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:21:13.574854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:21:13.574854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:21:13.575111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:21:13.575111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:21:13.575111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:21:13.575111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:21:13.575111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:21:13.575111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:21:13.575111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:21:13.575111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:21:13.575111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:21:13.575111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:21:13.575111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:13.575111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:21:13.575111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:13.575111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:21:13.575111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:13.575111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:21:13.575111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:13.575111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:21:13.575111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:13.575492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:21:13.575492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:21:13.575492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:13.575492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:21:13.575492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:13.575492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:21:13.575492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:21:13.575492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:13.575492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:21:13.575492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:13.575492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:21:13.575492 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:21:13.575724 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:21:13.575724 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:21:13.576074 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:21:13.577487 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:21:13.578080 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:21:13.578250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:13.579047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:21:13.579047 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:13.579047 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:21:13.579047 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:21:13.579047 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:13.579047 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:21:13.579047 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:13.579047 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:21:13.580123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:13.580123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:21:13.580123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:21:13.580123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:21:13.580123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:21:13.580123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:21:13.580123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:21:13.580123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:13.580123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:21:13.580123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:13.580123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:21:13.580123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:13.580386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:21:13.580386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:13.580386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:21:13.580386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:13.580386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:21:13.580386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:21:13.580386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:13.580386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:21:13.580386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:13.580386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:21:13.580386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:21:13.580386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:13.580386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:21:13.580624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:13.580624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:21:13.581128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:13.581128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:21:13.581128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:21:13.581128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:21:13.581128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:21:13.581128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:21:13.581128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:21:13.581128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:21:13.581128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:21:13.581128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:21:13.581128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:21:13.581128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:21:13.581128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:21:13.581128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:21:13.581128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:21:13.581128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:21:13.581424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:21:13.581424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:21:13.581424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:13.581424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:21:13.581424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:13.581424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:21:13.581424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:13.581424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:21:13.581424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:13.581424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:21:13.581424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:13.581424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:21:13.581424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:21:13.581424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:13.581424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:21:13.581424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:13.581424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:21:13.581424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:21:13.581424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:13.582367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:21:13.582367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:13.582367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:21:18.387311 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:21:18.388042 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:21:18.388042 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:21:18.388834 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:21:18.431560 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:21:18.431861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:18.432451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:21:18.432451 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:18.432451 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:21:18.432451 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:21:18.432451 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:18.432451 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:21:18.432451 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:18.432451 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:21:18.433496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.045 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:18.433496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:21:18.433496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:21:18.433496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:21:18.433496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:21:18.433496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:21:18.433496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:21:18.433496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:18.433496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:21:18.433496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:18.433496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:21:18.433496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:18.434130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:21:18.434130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:18.434130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:21:18.434130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:18.434130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:21:18.434130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:21:18.434130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:18.434130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:21:18.434130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:18.434130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:21:18.434130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:21:18.434130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:18.434130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:21:18.434704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:18.434704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:21:18.434704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:18.434704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:21:18.434704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:21:18.434704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:21:18.434704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:21:18.434704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:21:18.434704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:21:18.434704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:21:18.434704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:21:18.434704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:21:18.434704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:21:18.434704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:21:18.434704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:21:18.435399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:21:18.435399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:21:18.435399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:21:18.435399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:21:18.435399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:21:18.435399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:18.435399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:21:18.435399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:18.435399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:21:18.435399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:18.435399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:21:18.435399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:18.435399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:21:18.435399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:18.435399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:21:18.435399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:21:18.435399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:18.435399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:21:18.435399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:18.436373 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:21:18.436373 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:21:18.436373 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:18.436373 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:21:18.436373 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:18.436373 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:21:18.552289 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:21:18.552943 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:21:18.553183 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:21:18.554376 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:21:18.554529 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:21:18.554661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:18.555270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:21:18.555270 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:18.555270 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:21:18.555270 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:21:18.555270 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:18.555270 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:21:18.555270 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:18.555270 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:21:18.557061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:18.557061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:21:18.557061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:21:18.557061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:21:18.557061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:21:18.557061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:21:18.557061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:21:18.557061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:18.557061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:21:18.557061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:18.557061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:21:18.557061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:18.557668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:21:18.557668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:18.557668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:21:18.557668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:18.557668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:21:18.557668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:21:18.557668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:18.557668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:21:18.557668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:18.557668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:21:18.557668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:21:18.557668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:18.557668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:21:18.558230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:18.558230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:21:18.558921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:18.558921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:21:18.558921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:21:18.558921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:21:18.558921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:21:18.558921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:21:18.558921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:21:18.558921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:21:18.558921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:21:18.558921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:21:18.558921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:21:18.558921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:21:18.558921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:21:18.558921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:21:18.558921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:21:18.558921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:21:18.559650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:21:18.559650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:21:18.559650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:18.559650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:21:18.559650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:18.559650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:21:18.559650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:18.559650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:21:18.559650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:18.559650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:21:18.559650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:18.559650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:21:18.559650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:21:18.559650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:18.559650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:21:18.559650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:18.559650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:21:18.559650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:21:18.559650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:18.560535 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:21:18.560535 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:18.560535 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:21:18.561236 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:21:18.561506 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:21:18.561625 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:21:18.561767 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:21:18.562259 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:21:18.562431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:18.562990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:21:18.562990 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:18.562990 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:21:18.562990 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:21:18.562990 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:18.562990 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:21:18.562990 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:18.562990 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:21:18.563765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:18.563765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:21:18.563765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:21:18.563765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:21:18.563765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:18.563765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:21:18.563765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:18.563765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:21:18.563765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:18.563765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:21:18.563765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:18.563765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:21:18.563765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:18.563765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:21:18.563765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:21:18.564278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:18.564278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:21:18.564278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:18.564278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:21:18.564278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:21:18.564278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:18.564278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:21:18.564278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:18.564278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:21:18.565688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:18.565688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:21:18.565688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:21:18.565688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:21:18.565688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:21:18.565688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:21:18.565688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:21:18.565688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:21:18.565688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:21:18.565688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:21:18.565688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:18.565688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:21:18.566134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:18.566134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:21:18.566134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:18.566134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:21:18.566134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:18.566134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:21:18.566134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:18.566134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:21:18.566134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:21:18.566134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:18.566134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:21:18.566134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:18.566134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:21:18.566134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:21:18.566536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:18.566536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:21:18.566536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:18.566536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:21:18.566536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:18.566536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:21:18.566536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:21:18.566536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:21:18.566536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:21:18.566536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:21:18.566536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:21:18.566536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:21:18.566536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:21:18.566536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:21:18.567004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:21:18.567004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:21:18.567004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:21:18.567004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:21:18.567004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:21:18.567004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:21:18.567004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:21:18.567004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:21:18.567004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:21:18.567004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:21:18.567004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:21:18.567004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:18.567004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:21:18.567004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:18.567004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:21:18.567004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:18.567004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:21:18.567004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:18.567004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:21:18.568425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:18.568425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:21:18.568425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:21:18.568425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:18.568425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:21:18.568425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:18.568425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:21:18.568425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:21:18.568425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:18.568425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:21:18.568425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:18.568425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:21:18.568425 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:21:18.569213 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:21:18.569213 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:21:18.569213 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:21:18.569914 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:21:18.569914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:18.570510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:21:18.570510 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:18.570510 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:21:18.570510 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:21:18.570510 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:18.570510 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:21:18.570510 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:18.570510 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:21:18.571293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:18.571293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:21:18.571293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:21:18.571293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:21:18.571293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:18.571293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:21:18.571293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:18.571293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:21:18.571293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:18.571293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:21:18.571293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:18.571293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:21:18.571293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:18.571293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:21:18.571293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:21:18.571935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:18.571935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:21:18.571935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:18.571935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:21:18.571935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:21:18.571935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:18.571935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:21:18.571935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:18.571935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:21:18.572404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:18.572404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:21:18.572404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:21:18.572404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:21:18.572404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:21:18.572404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:21:18.572404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:21:18.572404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:21:18.572404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:21:18.572404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:18.572404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:21:18.572806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:18.572806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:21:18.572806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:18.572806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:21:18.572806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:18.572806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:21:18.572806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:18.572806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:21:18.572806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:21:18.572806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:18.572806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:21:18.572806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:18.572806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:21:18.572806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:21:18.573177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:18.573177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:21:18.573177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:18.573177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:21:18.573831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:18.573831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:21:18.573831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:21:18.573831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:21:18.573831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:21:18.573831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:21:18.573831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:21:18.573831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:21:18.573831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:21:18.573831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:21:18.573831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:21:18.573831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:21:18.573831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:21:18.573831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:21:18.573831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:21:18.573831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:21:18.573831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:21:18.574436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:21:18.574436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:21:18.574436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:21:18.574436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:18.574436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:21:18.574436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:18.574436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:21:18.574436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:18.574436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:21:18.574436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:18.574436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:21:18.574436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:18.574436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:21:18.574436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:21:18.574436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:18.574436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:21:18.574436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:18.574436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:21:18.574436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:21:18.575223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:18.575223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:21:18.575223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:18.575223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:21:18.575824 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:21:18.576242 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:21:18.576362 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:21:18.576525 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:21:18.576846 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:21:18.577086 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:21:18.577157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:18.577605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:21:18.577605 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:18.577605 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:21:18.577605 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:21:18.577605 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:18.577605 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:21:18.577605 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:18.577605 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:21:18.578540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:18.578540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:21:18.578540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:21:18.578540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:21:18.578540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:21:18.578540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:21:18.578540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:21:18.578540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:18.578540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:21:18.578540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:18.578540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:21:18.578540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:18.578997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:21:18.578997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:18.578997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:21:18.578997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:18.578997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:21:18.578997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:21:18.578997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:18.578997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:21:18.578997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:18.578997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:21:18.578997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:21:18.578997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:18.578997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:21:18.579389 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:18.579389 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:21:18.579475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:18.579475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:21:18.579475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:21:18.579475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:21:18.579475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:21:18.579475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:21:18.579475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:21:18.579475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:21:18.579475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:21:18.579475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:21:18.579475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:21:18.579475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:21:18.579475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:21:18.579475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:21:18.579475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:21:18.579475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:21:18.580061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:21:18.580061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:21:18.580061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:18.580061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:21:18.580061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:18.580061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:21:18.580061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:18.580061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:21:18.580061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:18.580061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:21:18.580061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:18.580061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:21:18.580061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:21:18.580061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:18.580061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:21:18.580061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:18.580061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:21:18.580061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:21:18.580061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:18.580948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:21:18.580948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:18.580948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:21:23.388254 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:21:23.389155 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:21:23.389272 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:21:23.391526 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:21:23.428457 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:21:23.428457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:23.429171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:21:23.429171 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:23.429171 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:21:23.429171 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:21:23.429171 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:23.429171 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:21:23.429171 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:23.429171 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:21:23.430257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:23.430257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:21:23.430257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:21:23.430257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:21:23.430257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:21:23.430257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:21:23.430257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:21:23.430257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:23.430257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:21:23.430257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:23.430257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:21:23.430257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:23.430584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:21:23.430584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:23.430584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:21:23.430584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:23.430584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:21:23.430584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:21:23.430584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:23.430584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:21:23.430584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:23.430584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:21:23.430584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:21:23.430584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:23.430584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:21:23.430875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:23.430875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:21:23.431384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:23.431384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:21:23.431384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:21:23.431384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:21:23.431384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:21:23.431384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:21:23.431384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:21:23.431384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:21:23.431384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:21:23.431384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:21:23.431384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:21:23.431384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:21:23.431384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:21:23.431384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:21:23.431384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:21:23.431384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:21:23.431773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:21:23.431773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:21:23.431773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:23.431773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:21:23.431773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:23.431773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:21:23.431773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:23.431773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:21:23.431773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:23.431773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:21:23.431773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:23.431773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:21:23.431773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:21:23.431773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:23.431773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:21:23.431773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:23.431773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:21:23.431773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:21:23.431773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:23.432197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:21:23.432197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:23.432197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:21:23.559879 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:21:23.559879 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:21:23.559879 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:21:23.559879 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:21:23.559879 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:21:23.559879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:23.560521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:21:23.560521 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:23.560521 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:21:23.560521 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:21:23.560521 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:23.560521 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:21:23.560521 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:23.560521 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:21:23.560521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:23.560521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:21:23.560521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:21:23.560521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:21:23.560521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:21:23.560521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:21:23.560521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:21:23.562076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:23.562076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:21:23.562076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:23.562076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:21:23.562076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:23.562076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:21:23.562076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:23.562076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:21:23.562076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:23.562076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:21:23.562076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:21:23.562076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:23.562076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:21:23.562076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:23.562498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:21:23.562498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:21:23.562498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:23.562498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:21:23.562498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:23.562498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:21:23.562498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:23.562498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:21:23.562498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:21:23.562498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:21:23.562498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:21:23.562498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:21:23.562498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:21:23.562498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:21:23.563693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:21:23.563693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:21:23.563693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:21:23.563693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:21:23.563693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:21:23.563693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:21:23.563693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:21:23.563693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:21:23.563693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:21:23.563693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:21:23.563693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:23.563693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:21:23.563693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:23.563693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:21:23.563693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:23.563693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:21:23.563693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:23.563693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:21:23.563693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:23.563693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:21:23.564348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:21:23.564348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:23.564348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:21:23.564348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:23.564348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:21:23.564348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:21:23.564348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:23.564348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:21:23.564348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:23.564348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:21:23.564348 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:21:23.564348 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:21:23.564734 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:21:23.564734 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:21:23.564734 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:21:23.564734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:23.564734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:21:23.564734 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:23.564734 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:21:23.564734 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:21:23.564734 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:23.564734 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:21:23.564734 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:23.564734 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:21:23.565188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:23.565188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:21:23.565188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:21:23.565188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:21:23.565188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:23.565188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:21:23.565188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:23.565188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:21:23.565188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:23.565188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:21:23.565188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:23.565188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:21:23.565188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:23.565188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:21:23.565188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:21:23.565715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:23.565715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:21:23.565715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:23.565715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:21:23.565715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:21:23.565715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:23.565715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:21:23.565715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:23.565715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:21:23.566019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:23.566019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:21:23.566019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:21:23.566019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:21:23.566019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:21:23.566019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:21:23.566019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:21:23.566019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:21:23.566019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:21:23.566019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:21:23.566019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:23.566019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:21:23.566336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:23.566336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:21:23.566336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:23.566336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:21:23.566336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:23.566336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:21:23.566336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:23.566336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:21:23.566336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:21:23.566336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:23.566336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:21:23.566336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:23.566336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:21:23.566336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:21:23.566940 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:23.566940 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:21:23.566940 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:23.566940 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:21:23.567060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:23.567060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:21:23.567060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:21:23.567060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:21:23.567060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:21:23.567060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:21:23.567060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:21:23.567060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:21:23.567060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:21:23.567060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:21:23.567060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:21:23.567060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:21:23.567060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:21:23.567060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:21:23.567060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:21:23.567060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:21:23.567811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:21:23.567811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:21:23.567811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:21:23.567811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:21:23.567811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:21:23.567811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:23.567811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:21:23.567811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:23.567811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:21:23.567811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:23.567811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:21:23.567811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:23.567811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:21:23.567811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:23.567811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:21:23.567811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:21:23.567811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:23.567811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:21:23.568979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:23.568979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:21:23.568979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:21:23.568979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:23.568979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:21:23.568979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:23.568979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:21:23.569173 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:21:23.569398 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:21:23.569523 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:21:23.569857 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:21:23.570569 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:21:23.570815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:23.571537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:21:23.571537 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:23.571537 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:21:23.571537 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:21:23.571537 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:23.571537 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:21:23.571537 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:23.571537 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:21:23.572415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:23.572415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:21:23.572415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:21:23.572415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:21:23.572415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:23.572415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:21:23.572415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:23.572415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:21:23.572415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:23.572415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:21:23.572415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:23.572415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:21:23.572415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:23.572415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:21:23.572415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:21:23.572720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:23.572720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:21:23.572720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:23.572720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:21:23.572720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:21:23.572720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:23.572720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:21:23.572720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:23.572720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:21:23.573422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:23.573422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:21:23.573422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:21:23.573422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:21:23.573422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:21:23.573422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:21:23.573422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:21:23.573422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:21:23.573422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:21:23.573422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:23.573422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:21:23.573649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:23.573649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:21:23.573649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:23.573649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:21:23.573649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:23.573649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:21:23.573649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:23.573649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:21:23.573649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:21:23.573649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:23.573649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:21:23.573649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:23.573649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:21:23.573649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:21:23.573955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:23.573955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:21:23.573955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:23.573955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:21:23.574636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:23.574636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:21:23.574636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:21:23.574636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:21:23.574636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:21:23.574636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:21:23.574636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:21:23.574636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:21:23.574636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:21:23.574636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:21:23.574636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:21:23.574636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:21:23.574636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:21:23.574636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:21:23.574636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:21:23.574636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:21:23.574636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:21:23.575015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:21:23.575015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:21:23.575015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:21:23.575015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:23.575015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:21:23.575015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:23.575015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:21:23.575015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:23.575015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:21:23.575015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:23.575015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:21:23.575015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:23.575015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:21:23.575015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:21:23.575015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:23.575015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:21:23.575015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:23.575015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:21:23.575015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:21:23.581802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:23.581802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:21:23.581802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:23.581802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:21:23.581802 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:21:23.581802 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:21:23.581802 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:21:23.581802 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:21:23.582134 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:21:23.582134 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:21:23.582134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:23.582134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:21:23.582134 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:23.582134 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:21:23.582134 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:21:23.582134 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:23.582134 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:21:23.582134 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:23.582134 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:21:23.582134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:23.582479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:21:23.582479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:21:23.582479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:21:23.582479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:21:23.582479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:21:23.582479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:21:23.582479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:23.582479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:21:23.582479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:23.582479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:21:23.582479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:23.582479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:21:23.582479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:23.582479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:21:23.583405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:23.583405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:21:23.583405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:21:23.583405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:23.583405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:21:23.583405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:23.583405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:21:23.583405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:21:23.583405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:23.583405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:21:23.583405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:23.583405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:21:23.583784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:23.583784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:21:23.583784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:21:23.583784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:21:23.583784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:21:23.583784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:21:23.583784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:21:23.583784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:21:23.583784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:21:23.583784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:21:23.583784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:21:23.583784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:21:23.583784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:21:23.583784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:21:23.583784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:21:23.583784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:21:23.584368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:21:23.584368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:21:23.584368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:23.584368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:21:23.584368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:23.584368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:21:23.584368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:23.584368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:21:23.584368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:23.584368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:21:23.584368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:23.584368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:21:23.584368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:21:23.584368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:23.584368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:21:23.584368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:23.584368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:21:23.584368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:21:23.584368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:23.584921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:21:23.584921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:23.584921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:21:28.389384 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:21:28.390255 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:21:28.390255 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:21:28.391292 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:21:28.441537 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:21:28.441980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:28.443130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:21:28.443130 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:28.443130 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:21:28.443130 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:21:28.443130 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:28.443130 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:21:28.443130 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:28.443130 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:21:28.444482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.053 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:28.444482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:21:28.444482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:21:28.444482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:21:28.444482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:21:28.444482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:21:28.444482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:21:28.444482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:28.444482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:21:28.444482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:28.444482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:21:28.444482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:28.445468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:21:28.445468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:28.445468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:21:28.445468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:28.445468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:21:28.445468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:21:28.445468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:28.445468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:21:28.445468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:28.445468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:21:28.445468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:21:28.445468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:28.445468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:21:28.449244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:28.449244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:21:28.449244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:28.449244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:21:28.449244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:21:28.449244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:21:28.449244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:21:28.449244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:21:28.449244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:21:28.449244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:21:28.449244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:21:28.449244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:21:28.449244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:21:28.449244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:21:28.449244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:21:28.450351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:21:28.450351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:21:28.450351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:21:28.450351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:21:28.450351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:21:28.450351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:28.450351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:21:28.450351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:28.450351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:21:28.450351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:28.450351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:21:28.450351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:28.450351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:21:28.450351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:28.450351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:21:28.450351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:21:28.450351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:28.450351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:21:28.450351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:28.451672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:21:28.451672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:21:28.451672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:28.451672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:21:28.451672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:28.451672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:21:28.554107 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:21:28.554554 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:21:28.554784 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:21:28.556249 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:21:28.556650 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:21:28.556963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:28.558140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:21:28.558140 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:28.558140 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:21:28.558140 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:21:28.558140 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:28.558140 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:21:28.558140 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:28.558140 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:21:28.560111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:28.560111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:21:28.560111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:21:28.560111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:21:28.560111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:21:28.560111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:21:28.560111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:21:28.560111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:28.560111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:21:28.560111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:28.560111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:21:28.560111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:28.560515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:21:28.560515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:28.560515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:21:28.560515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:28.560515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:21:28.560515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:21:28.560515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:28.560515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:21:28.560515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:28.560515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:21:28.560515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:21:28.560515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:28.560515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:21:28.560887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:28.560887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:21:28.562287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:28.562287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:21:28.562287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:21:28.562287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:21:28.562287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:21:28.562287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:21:28.562287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:21:28.562287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:21:28.562287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:21:28.562287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:21:28.562287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:21:28.562287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:21:28.562287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:21:28.562287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:21:28.562287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:21:28.562287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:21:28.562827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:21:28.562827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:21:28.562827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:28.562827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:21:28.562827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:28.562827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:21:28.562827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:28.562827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:21:28.562827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:28.562827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:21:28.562827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:28.562827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:21:28.562827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:21:28.562827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:28.562827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:21:28.562827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:28.562827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:21:28.562827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:21:28.562827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:28.563444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:21:28.563444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:28.563444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:21:28.563444 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:21:28.563444 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:21:28.563667 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:21:28.563667 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:21:28.564334 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:21:28.564565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:28.565198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:21:28.565198 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:28.565198 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:21:28.565198 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:21:28.565198 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:28.565198 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:21:28.565198 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:28.565198 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:21:28.566298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:28.566298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:21:28.566298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:21:28.566298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:21:28.566298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:28.566298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:21:28.566298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:28.566298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:21:28.566298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:28.566298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:21:28.566298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:28.566298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:21:28.566298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:28.566298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:21:28.566298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:21:28.567095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:28.567095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:21:28.567095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:28.567095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:21:28.567095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:21:28.567095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:28.567095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:21:28.567095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:28.567095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:21:28.567455 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:28.567455 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:21:28.567455 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:21:28.567455 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:21:28.567455 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:21:28.567455 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:21:28.567455 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:21:28.567455 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:21:28.567455 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:21:28.567455 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:21:28.567455 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:28.567455 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:21:28.567933 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:28.567933 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:21:28.567933 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:28.567933 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:21:28.567933 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:28.567933 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:21:28.567933 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:28.567933 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:21:28.567933 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:21:28.567933 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:28.567933 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:21:28.567933 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:28.567933 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:21:28.567933 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:21:28.568375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:28.568375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:21:28.568375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:28.568375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:21:28.569209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:28.569209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:21:28.569209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:21:28.569209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:21:28.569209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:21:28.569209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:21:28.569209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:21:28.569209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:21:28.569209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:21:28.569209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:21:28.569209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:21:28.569209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:21:28.569209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:21:28.569209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:21:28.569209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:21:28.569209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:21:28.569888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:21:28.569888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:21:28.569888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:21:28.569888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:21:28.569888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:21:28.569888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:28.569888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:21:28.569888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:28.569888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:21:28.569888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:28.569888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:21:28.569888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:28.569888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:21:28.569888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:28.569888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:21:28.569888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:21:28.569888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:28.569888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:21:28.570526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:28.570526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:21:28.570526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:21:28.570526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:28.570526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:21:28.570526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:28.570526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:21:28.570526 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:21:28.570526 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:21:28.570526 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:21:28.570986 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:21:28.571271 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:21:28.571672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:28.572679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:21:28.572679 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:28.572679 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:21:28.572679 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:21:28.572679 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:28.572679 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:21:28.572679 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:28.572679 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:21:28.574099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:28.574099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:21:28.574099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:21:28.574099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:21:28.574099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:28.574099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:21:28.574099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:28.574099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:21:28.574099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:28.574099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:21:28.574099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:28.574099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:21:28.574099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:28.574099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:21:28.574099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:21:28.574684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:28.574684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:21:28.574684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:28.574684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:21:28.574684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:21:28.574684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:28.574684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:21:28.574684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:28.574684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:21:28.577083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:28.577083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:21:28.577083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:21:28.577083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:21:28.577083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:21:28.577083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:21:28.577083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:21:28.577083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:21:28.577083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:21:28.577083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:28.577083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:21:28.577528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:28.577528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:21:28.577528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:28.577528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:21:28.577528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:28.577528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:21:28.577528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:28.577528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:21:28.577528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:21:28.577528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:28.577528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:21:28.577528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:28.577528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:21:28.577528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:21:28.577939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:28.577939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:21:28.577939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:28.577939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:21:28.580510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:28.580510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:21:28.580510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:21:28.580510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:21:28.580510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:21:28.580510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:21:28.580510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:21:28.580510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:21:28.580510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:21:28.580510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:21:28.580510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:21:28.580510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:21:28.580510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:21:28.580510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:21:28.580510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:21:28.580510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:21:28.580510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:21:28.581625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:21:28.581625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:21:28.581625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:21:28.581625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:28.581625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:21:28.581625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:28.581625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:21:28.581625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:28.581625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:21:28.581625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:28.581625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:21:28.581625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:28.581625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:21:28.581625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:21:28.581625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:28.581625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:21:28.581625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:28.581625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:21:28.581625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:21:28.582410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:28.582410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:21:28.582410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:28.582410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:21:28.582410 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:21:28.582410 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:21:28.582410 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:21:28.582410 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:21:28.582977 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:21:28.582977 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:21:28.582977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:28.586062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:21:28.586062 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:28.586062 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:21:28.586062 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:21:28.586062 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:28.586062 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:21:28.586062 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:28.586062 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:21:28.586062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:28.586062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:21:28.586062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:21:28.586062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:21:28.586062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:21:28.586062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:21:28.586062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:21:28.587141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:28.587141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:21:28.587141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:28.587141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:21:28.587141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:28.587141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:21:28.587141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:28.587141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:21:28.587141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:28.587141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:21:28.587141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:21:28.587141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:28.587141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:21:28.587141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:28.587720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:21:28.587720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:21:28.587720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:28.587720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:21:28.587720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:28.587720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:21:28.587720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:28.587720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:21:28.587720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:21:28.587720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:21:28.587720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:21:28.587720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:21:28.587720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:21:28.587720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:21:28.588348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:21:28.588348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:21:28.588348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:21:28.588348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:21:28.588348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:21:28.588348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:21:28.588348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:21:28.588348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:21:28.588348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:21:28.588348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:21:28.588348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:28.588348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:21:28.588348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:28.588348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:21:28.588348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:28.588348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:21:28.588348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:28.588348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:21:28.588348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:28.588348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:21:28.588732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:21:28.588732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:28.588732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:21:28.588732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:28.588732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:21:28.588732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:21:28.588732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:28.588732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:21:28.588732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:28.588732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:21:33.390173 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:21:33.390786 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:21:33.390786 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:21:33.391995 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:21:33.435063 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:21:33.435344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:33.436610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:21:33.436610 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:33.436610 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:21:33.436610 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:21:33.436610 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:33.436610 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:21:33.436610 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:33.436610 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:21:33.437370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.046 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:33.437370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:21:33.437370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:21:33.437370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:21:33.437370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:21:33.437370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:21:33.437370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:21:33.437370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:33.437370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:21:33.437370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:33.437370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:21:33.437370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:33.437946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:21:33.437946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:33.437946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:21:33.437946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:33.437946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:21:33.437946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:21:33.437946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:33.437946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:21:33.437946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:33.437946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:21:33.437946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:21:33.437946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:33.437946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:21:33.438405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:33.438405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:21:33.438405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:33.438405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:21:33.438405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:21:33.438405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:21:33.438405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:21:33.438405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:21:33.438405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:21:33.438405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:21:33.438405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:21:33.438405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:21:33.438405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:21:33.438405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:21:33.438405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:21:33.438784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:21:33.438784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:21:33.438784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:21:33.438784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:21:33.438784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:21:33.438784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:33.438784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:21:33.438784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:33.438784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:21:33.438784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:33.438784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:21:33.438784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:33.438784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:21:33.438784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:33.438784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:21:33.438784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:21:33.438784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:33.438784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:21:33.438784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:33.439178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:21:33.439178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:21:33.439178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:33.439178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:21:33.439178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:33.439178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:21:33.554424 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:21:33.554954 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:21:33.555774 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:21:33.556196 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:21:33.556330 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:21:33.556537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:33.557230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:21:33.557230 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:33.557230 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:21:33.557230 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:21:33.557230 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:33.557230 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:21:33.557230 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:33.557230 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:21:33.561217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:33.561217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:21:33.561217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:21:33.561217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:21:33.561217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:21:33.561217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:21:33.561217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:21:33.561217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:33.561217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:21:33.561217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:33.561217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:21:33.561217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:33.562179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:21:33.562179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:33.562179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:21:33.562179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:33.562179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:21:33.562179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:21:33.562179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:33.562179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:21:33.562179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:33.562179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:21:33.562179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:21:33.562179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:33.562179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:21:33.562651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:33.562651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:21:33.562651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:33.562651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:21:33.562651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:21:33.562651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:21:33.562651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:21:33.562651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:21:33.562651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:21:33.562651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:21:33.562651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:21:33.562651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:21:33.562651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:21:33.562651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:21:33.562651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:21:33.563344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:21:33.563344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:21:33.563344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:21:33.563344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:21:33.563344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:21:33.563344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:33.563344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:21:33.563344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:33.563344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:21:33.563344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:33.563344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:21:33.563344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:33.563344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:21:33.563344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:33.563344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:21:33.563344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:21:33.563344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:33.563344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:21:33.563344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:33.563895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:21:33.563895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:21:33.563895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:33.563895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:21:33.563895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:33.563895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:21:33.563895 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:21:33.563895 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:21:33.565419 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:21:33.565419 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:21:33.565929 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:21:33.566167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:33.566789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:21:33.566789 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:33.566789 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:21:33.566789 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:21:33.566789 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:33.566789 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:21:33.566789 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:33.566789 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:21:33.569839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:33.569839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:21:33.569839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:21:33.569839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:21:33.569839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:33.569839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:21:33.569839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:33.569839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:21:33.569839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:33.569839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:21:33.569839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:33.569839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:21:33.569839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:33.569839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:21:33.569839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:21:33.570444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:33.570444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:21:33.570444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:33.570444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:21:33.570444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:21:33.570444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:33.570444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:21:33.570444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:33.570444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:21:33.570444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:33.570444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:21:33.570444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:21:33.570444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:21:33.570444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:21:33.570896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:21:33.570896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:21:33.570896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:21:33.570896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:21:33.570896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:21:33.570896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:33.570896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:21:33.570896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:33.570896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:21:33.570896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:33.570896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:21:33.570896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:33.570896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:21:33.571373 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:33.571373 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:21:33.571373 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:21:33.571373 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:33.571373 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:21:33.571373 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:33.571373 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:21:33.571373 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:21:33.571373 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:33.571373 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:21:33.571373 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:33.571373 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:21:33.571902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:33.571902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:21:33.571902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:21:33.571902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:21:33.571902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:21:33.571902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:21:33.571902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:21:33.571902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:21:33.571902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:21:33.571902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:21:33.571902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:21:33.571902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:21:33.571902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:21:33.571902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:21:33.571902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:21:33.571902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:21:33.572339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:21:33.572339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:21:33.572339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:21:33.572339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:21:33.572339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:21:33.572339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:33.572339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:21:33.572339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:33.572339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:21:33.572339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:33.572339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:21:33.572339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:33.572339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:21:33.572339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:33.572339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:21:33.572339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:21:33.572339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:33.572339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:21:33.572800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:33.572800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:21:33.572800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:21:33.572800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:33.572800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:21:33.572800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:33.572800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:21:33.573348 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:21:33.574169 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:21:33.574169 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:21:33.574169 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:21:33.574775 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:21:33.575228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:33.575938 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:21:33.575938 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:33.575938 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:21:33.575938 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:21:33.575938 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:33.575938 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:21:33.575938 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:33.575938 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:21:33.579321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:33.579321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:21:33.579321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:21:33.579321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:21:33.579321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:33.579321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:21:33.579321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:33.579321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:21:33.579321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:33.579321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:21:33.579321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:33.579321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:21:33.579321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:33.579321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:21:33.579321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:21:33.579814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:33.579814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:21:33.579814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:33.579814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:21:33.579814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:21:33.579814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:33.579814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:21:33.579814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:33.579814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:21:33.579814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:33.579814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:21:33.579814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:21:33.579814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:21:33.579814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:21:33.580183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:21:33.580183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:21:33.580183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:21:33.580183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:21:33.580183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:33.580183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:21:33.580183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:33.580183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:21:33.580183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:33.580183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:21:33.580183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:33.580183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:21:33.580183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:33.580466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:21:33.580466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:21:33.580466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:33.580466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:21:33.580466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:33.580466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:21:33.580466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:21:33.580466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:33.580466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:21:33.580466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:33.580466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:21:33.580466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:33.580891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:21:33.580891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:21:33.580891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:21:33.580891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:21:33.580891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:21:33.580891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:21:33.580891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:21:33.580891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:21:33.580891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:21:33.580891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:21:33.580891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:21:33.580891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:21:33.580891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:21:33.580891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:21:33.580891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:21:33.580891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:21:33.580891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:21:33.580891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:21:33.580891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:21:33.581410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:33.581410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:21:33.581410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:33.581410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:21:33.581410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:33.581410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:21:33.581410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:33.581410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:21:33.581410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:33.581410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:21:33.581410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:21:33.581410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:33.581410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:21:33.581410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:33.581410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:21:33.581410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:21:33.581410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:33.581410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:21:33.581410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:33.581410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:21:33.582013 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:21:33.582309 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:21:33.582518 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:21:33.582907 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:21:33.583613 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:21:33.583912 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:21:33.584069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:33.586644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:21:33.586644 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:33.586644 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:21:33.586644 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:21:33.586644 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:33.586644 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:21:33.586644 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:33.586644 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:21:33.586644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:33.586644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:21:33.586644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:21:33.586644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:21:33.586644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:21:33.586644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:21:33.586644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:21:33.587054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:33.587054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:21:33.587054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:33.587054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:21:33.587054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:33.587054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:21:33.587054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:33.587054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:21:33.587054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:33.587054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:21:33.587054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:21:33.587054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:33.587054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:21:33.587054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:33.587316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:21:33.587316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:21:33.587316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:33.587316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:21:33.587316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:33.587316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:21:33.587316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:33.587316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:21:33.587316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:21:33.587316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:21:33.587316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:21:33.587316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:21:33.587316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:21:33.587316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:21:33.587579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:21:33.587579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:21:33.587579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:21:33.587579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:21:33.587579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:21:33.587579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:21:33.587579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:21:33.587579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:21:33.587579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:21:33.587579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:21:33.587579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:33.587579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:21:33.587579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:33.587579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:21:33.587579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:33.587579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:21:33.587579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:33.587579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:21:33.587579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:33.587579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:21:33.588008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:21:33.588008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:33.588008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:21:33.588008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:33.588008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:21:33.588008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:21:33.588008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:33.588008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:21:33.588008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:33.588008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:21:38.391009 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:21:38.391851 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:21:38.391851 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:21:38.392338 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:21:38.440800 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:21:38.440800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:38.440800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:21:38.440800 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:38.440800 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:21:38.440800 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:21:38.440800 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:38.440800 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:21:38.440800 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:38.440800 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:21:38.441471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.049 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:38.441471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:21:38.441471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:21:38.441471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:21:38.441471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:21:38.441471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:21:38.441471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:21:38.441471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:38.441471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:21:38.441471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:38.441471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:21:38.441471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:38.441872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:21:38.441872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:38.441872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:21:38.441872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:38.441872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:21:38.441872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:21:38.441872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:38.441872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:21:38.441872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:38.441872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:21:38.441872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:21:38.441872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:38.441872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:21:38.442216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:38.442216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:21:38.442216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:38.442216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:21:38.442216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:21:38.442216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:21:38.442216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:21:38.442216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:21:38.442216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:21:38.442216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:21:38.442216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:21:38.442216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:21:38.442216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:21:38.442216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:21:38.442216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:21:38.442624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:21:38.442624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:21:38.442624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:21:38.442624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:21:38.442624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:21:38.442624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:38.442624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:21:38.442624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:38.442624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:21:38.442624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:38.442624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:21:38.442624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:38.442624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:21:38.442624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:38.442624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:21:38.442624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:21:38.442624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:38.442624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:21:38.442624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:38.443138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:21:38.443138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:21:38.443138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:38.443138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:21:38.443138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:38.443138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:21:38.555481 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:21:38.556589 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:21:38.556816 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:21:38.558300 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:21:38.558710 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:21:38.559032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:38.560053 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:21:38.560053 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:38.560053 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:21:38.560053 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:21:38.560053 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:38.560053 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:21:38.560053 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:38.560053 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:21:38.561668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:38.561668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:21:38.561668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:21:38.561668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:21:38.561668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:21:38.561668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:21:38.561668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:21:38.561668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:38.561668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:21:38.561668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:38.561668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:21:38.561668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:38.562092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:21:38.562092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:38.562092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:21:38.562092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:38.562092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:21:38.562092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:21:38.562092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:38.562092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:21:38.562092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:38.562092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:21:38.562092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:21:38.562092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:38.562092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:21:38.562429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:38.562429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:21:38.563489 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:38.563489 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:21:38.563489 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:21:38.563489 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:21:38.563489 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:21:38.563489 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:21:38.563489 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:21:38.563489 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:21:38.563489 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:21:38.563489 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:21:38.563489 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:21:38.563489 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:21:38.563489 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:21:38.563489 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:21:38.563489 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:21:38.563489 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:21:38.564211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:21:38.564211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:21:38.564211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:38.564211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:21:38.564211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:38.564211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:21:38.564211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:38.564211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:21:38.564211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:38.564211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:21:38.564211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:38.564211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:21:38.564211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:21:38.564211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:38.564211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:21:38.564211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:38.564211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:21:38.564211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:21:38.564211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:38.564973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:21:38.564973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:38.564973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:21:38.564973 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:21:38.564973 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:21:38.564973 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:21:38.564973 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:21:38.565336 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:21:38.567286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:38.567286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:21:38.567286 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:38.567286 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:21:38.567286 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:21:38.567286 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:38.567286 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:21:38.567286 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:38.567286 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:21:38.567286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:38.567286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:21:38.567286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:21:38.567286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:21:38.567286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:38.567286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:21:38.567286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:38.567286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:21:38.568204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:38.568204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:21:38.568204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:38.568204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:21:38.568204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:38.568204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:21:38.568204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:21:38.568204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:38.568204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:21:38.568204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:38.568204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:21:38.568204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:21:38.568204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:38.568204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:21:38.568204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:38.568204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:21:38.568923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:38.568923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:21:38.568923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:21:38.568923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:21:38.568923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:21:38.568923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:21:38.568923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:21:38.568923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:21:38.568923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:21:38.568923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:21:38.568923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:38.568923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:21:38.569460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:38.569460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:21:38.569460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:38.569460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:21:38.569460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:38.569460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:21:38.569460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:38.569460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:21:38.569460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:21:38.569460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:38.569460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:21:38.569460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:38.569460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:21:38.569460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:21:38.570042 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:38.570042 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:21:38.570042 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:38.570042 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:21:38.570042 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:38.570042 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:21:38.570042 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:21:38.570042 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:21:38.570042 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:21:38.570042 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:21:38.570042 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:21:38.570042 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:21:38.570042 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:21:38.570042 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:21:38.570600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:21:38.570600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:21:38.570600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:21:38.570600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:21:38.570600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:21:38.570600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:21:38.570600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:21:38.570600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:21:38.570600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:21:38.570600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:21:38.570600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:21:38.570600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:38.570600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:21:38.570600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:38.570600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:21:38.570600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:38.570600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:21:38.570600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:38.570600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:21:38.571898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:38.571898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:21:38.571898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:21:38.571898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:38.571898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:21:38.571898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:38.571898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:21:38.571898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:21:38.571898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:38.571898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:21:38.571898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:38.571898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:21:38.573894 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:21:38.574214 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:21:38.574362 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:21:38.574506 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:21:38.575040 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:21:38.575248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:38.575845 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:21:38.575845 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:38.575845 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:21:38.575845 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:21:38.575845 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:38.575845 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:21:38.575845 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:38.575845 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:21:38.576646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:38.576646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:21:38.576646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:21:38.576646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:21:38.576646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:38.576646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:21:38.576646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:38.576646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:21:38.576646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:38.576646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:21:38.576646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:38.576646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:21:38.576646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:38.576646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:21:38.576646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:21:38.577111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:38.577111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:21:38.577111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:38.577111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:21:38.577111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:21:38.577111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:38.577111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:21:38.577111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:38.577111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:21:38.577582 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:38.577582 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:21:38.577582 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:21:38.577582 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:21:38.577582 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:21:38.577582 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:21:38.577582 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:21:38.577582 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:21:38.577582 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:21:38.577582 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:38.577582 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:21:38.577950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:38.577950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:21:38.577950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:38.577950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:21:38.577950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:38.577950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:21:38.577950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:38.577950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:21:38.577950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:21:38.577950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:38.577950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:21:38.577950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:38.577950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:21:38.577950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:21:38.578320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:38.578320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:21:38.578320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:38.578320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:21:38.578733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:38.578733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:21:38.578733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:21:38.578733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:21:38.578733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:21:38.578733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:21:38.578733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:21:38.578733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:21:38.578733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:21:38.578733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:21:38.578733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:21:38.578733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:21:38.578733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:21:38.578733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:21:38.578733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:21:38.578733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:21:38.578733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:21:38.579331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:21:38.579331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:21:38.579331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:21:38.579331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:38.579331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:21:38.579331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:38.579331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:21:38.579331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:38.579331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:21:38.579331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:38.579331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:21:38.579331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:38.579331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:21:38.579331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:21:38.579331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:38.579331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:21:38.579331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:38.579331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:21:38.579331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:21:38.579874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:38.579874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:21:38.579874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:38.579874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:21:38.582416 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:21:38.582704 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:21:38.582845 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:21:38.583010 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:21:38.583862 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:21:38.584395 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:21:38.584723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:38.585276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:21:38.585276 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:38.585276 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:21:38.585276 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:21:38.585276 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:38.585276 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:21:38.585276 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:38.585276 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:21:38.586165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:38.586165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:21:38.586165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:21:38.586165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:21:38.586165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:21:38.586165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:21:38.586165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:21:38.586165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:38.586165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:21:38.586165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:38.586165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:21:38.586165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:38.586546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:21:38.586546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:38.586546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:21:38.586546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:38.586546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:21:38.586546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:21:38.586546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:38.586546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:21:38.586546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:38.586546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:21:38.586546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:21:38.586546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:38.586546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:21:38.586891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:38.586891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:21:38.587229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:38.587229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:21:38.587229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:21:38.587229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:21:38.587229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:21:38.587229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:21:38.587229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:21:38.587229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:21:38.587229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:21:38.587229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:21:38.587229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:21:38.587229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:21:38.587229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:21:38.587229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:21:38.587229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:21:38.587229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:21:38.587709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:21:38.587709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:21:38.587709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:38.587709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:21:38.587709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:38.587709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:21:38.587709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:38.587709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:21:38.587709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:38.587709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:21:38.587709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:38.587709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:21:38.587709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:21:38.587709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:38.587709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:21:38.587709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:38.587709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:21:38.587709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:21:38.587709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:38.588207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:21:38.588207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:38.588207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:21:43.392640 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:21:43.392640 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:21:43.392640 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:21:43.393571 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:21:43.435014 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:21:43.435237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:43.435837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:21:43.435837 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:43.435837 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:21:43.435837 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:21:43.435837 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:43.435837 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:21:43.435837 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:43.435837 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:21:43.436702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.043 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:43.436702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:21:43.436702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:21:43.436702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:21:43.436702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:21:43.436702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:21:43.436702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:21:43.436702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:43.436702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:21:43.436702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:43.436702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:21:43.436702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:43.437369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:21:43.437369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:43.437369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:21:43.437369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:43.437369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:21:43.437369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:21:43.437369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:43.437369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:21:43.437369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:43.437369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:21:43.437369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:21:43.437369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:43.437369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:21:43.438046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:43.438046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:21:43.438046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:43.438046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:21:43.438046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:21:43.438046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:21:43.438046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:21:43.438046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:21:43.438046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:21:43.438046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:21:43.438046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:21:43.438046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:21:43.438046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:21:43.438046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:21:43.438046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:21:43.438872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:21:43.438872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:21:43.438872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:21:43.438872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:21:43.438872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:21:43.438872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:43.438872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:21:43.438872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:43.438872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:21:43.438872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:43.438872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:21:43.438872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:43.438872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:21:43.438872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:43.438872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:21:43.438872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:21:43.438872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:43.438872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:21:43.438872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:43.439520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:21:43.439520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:21:43.439520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:43.439520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:21:43.439520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:43.439520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:21:43.556297 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:21:43.559797 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:21:43.559797 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:21:43.559797 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:21:43.559797 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:21:43.559797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:43.559797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:21:43.559797 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:43.559797 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:21:43.559797 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:21:43.560210 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:43.560210 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:21:43.560210 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:43.560210 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:21:43.560210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:43.560210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:21:43.560210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:21:43.560210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:21:43.560210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:21:43.560210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:21:43.560210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:21:43.560210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:43.560210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:21:43.560971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:43.560971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:21:43.560971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:43.560971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:21:43.560971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:43.560971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:21:43.560971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:43.560971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:21:43.560971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:21:43.560971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:43.560971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:21:43.560971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:43.560971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:21:43.560971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:21:43.561709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:43.561709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:21:43.561709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:43.561709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:21:43.561709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:43.561709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:21:43.561709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:21:43.561709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:21:43.561709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:21:43.561709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:21:43.561709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:21:43.561709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:21:43.561709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:21:43.561709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:21:43.562184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:21:43.562184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:21:43.562184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:21:43.562184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:21:43.562184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:21:43.562184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:21:43.562184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:21:43.562184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:21:43.562184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:43.562184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:21:43.562184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:43.562184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:21:43.562184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:43.562184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:21:43.562184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:43.562184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:21:43.562184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:43.562184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:21:43.562184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:21:43.562710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:43.562710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:21:43.562710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:43.562710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:21:43.562710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:21:43.562710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:43.562710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:21:43.562710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:43.562710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:21:43.564512 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:21:43.565995 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:21:43.566123 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:21:43.566254 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:21:43.568207 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:21:43.568477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:43.569286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:21:43.569286 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:43.569286 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:21:43.569286 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:21:43.569286 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:43.569286 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:21:43.569286 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:43.569286 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:21:43.570174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:43.570174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:21:43.570174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:21:43.570174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:21:43.570174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:43.570174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:21:43.570174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:43.570174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:21:43.570174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:43.570174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:21:43.570174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:43.570174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:21:43.570174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:43.570174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:21:43.570174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:21:43.570680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:43.570680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:21:43.570680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:43.570680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:21:43.570680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:21:43.570680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:43.570680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:21:43.570680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:43.570680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:21:43.571105 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:43.571105 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:21:43.571105 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:21:43.571105 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:21:43.571105 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:21:43.571105 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:21:43.571105 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:21:43.571105 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:21:43.571105 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:21:43.571105 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:21:43.571105 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:43.571105 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:21:43.571512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:43.571512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:21:43.571512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:43.571512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:21:43.571512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:43.571512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:21:43.571512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:43.571512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:21:43.571512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:21:43.571512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:43.571512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:21:43.571512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:43.571512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:21:43.571512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:21:43.571894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:43.571894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:21:43.571894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:43.571894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:21:43.572216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:43.572216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:21:43.572216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:21:43.572216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:21:43.572216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:21:43.572216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:21:43.572216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:21:43.572216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:21:43.572216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:21:43.572216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:21:43.572216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:21:43.572216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:21:43.572216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:21:43.572216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:21:43.572216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:21:43.572216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:21:43.572660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:21:43.572660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:21:43.572660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:21:43.572660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:21:43.572660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:21:43.572660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:43.572660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:21:43.572660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:43.572660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:21:43.572660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:43.572660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:21:43.572660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:43.572660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:21:43.572660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:43.572660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:21:43.572660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:21:43.572660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:43.572660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:21:43.573177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:43.573177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:21:43.573177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:21:43.573177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:43.573177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:21:43.573177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:43.573177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:21:43.574507 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:21:43.574772 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:21:43.574909 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:21:43.575024 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:21:43.575604 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:21:43.575854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:43.576226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:21:43.576226 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:43.576226 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:21:43.576226 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:21:43.576226 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:43.576226 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:21:43.576226 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:43.576226 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:21:43.576781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:43.576781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:21:43.576781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:21:43.576781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:21:43.576781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:43.576781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:21:43.576781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:43.576781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:21:43.576781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:43.576781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:21:43.576781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:43.576781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:21:43.576781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:43.576781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:21:43.576781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:21:43.577205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:43.577205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:21:43.577205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:43.577205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:21:43.577205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:21:43.577205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:43.577205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:21:43.577205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:43.577205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:21:43.577455 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:43.577455 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:21:43.577455 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:21:43.577455 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:21:43.577455 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:21:43.577455 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:21:43.577455 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:21:43.577455 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:21:43.577455 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:21:43.577455 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:43.577455 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:21:43.577823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:43.577823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:21:43.577823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:43.577823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:21:43.577823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:43.577823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:21:43.577823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:43.577823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:21:43.577823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:21:43.577823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:43.577823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:21:43.577823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:43.577823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:21:43.577823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:21:43.578219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:43.578219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:21:43.578219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:43.578219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:21:43.578340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:43.578340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:21:43.578340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:21:43.578340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:21:43.578340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:21:43.578340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:21:43.578340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:21:43.578340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:21:43.578340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:21:43.578340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:21:43.578340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:21:43.578340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:21:43.578340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:21:43.578340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:21:43.578340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:21:43.578340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:21:43.578340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:21:43.578787 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:21:43.578787 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:21:43.578787 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:21:43.578787 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:43.578787 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:21:43.578787 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:43.578787 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:21:43.578787 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:43.578787 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:21:43.578787 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:43.578787 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:21:43.578787 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:43.578787 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:21:43.578787 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:21:43.578787 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:43.578787 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:21:43.578787 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:43.578787 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:21:43.578787 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:21:43.579249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:43.579249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:21:43.579249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:43.579249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:21:43.583946 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:21:43.583946 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:21:43.583946 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:21:43.583946 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:21:43.583946 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:21:43.584245 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:21:43.584245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:43.584874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:21:43.584874 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:43.584874 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:21:43.584874 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:21:43.584874 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:43.584874 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:21:43.584874 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:43.584874 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:21:43.585664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:43.585664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:21:43.585664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:21:43.585664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:21:43.585664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:21:43.585664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:21:43.585664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:21:43.585664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:43.585664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:21:43.585664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:43.585664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:21:43.585664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:43.586009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:21:43.586009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:43.586009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:21:43.586009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:43.586009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:21:43.586009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:21:43.586009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:43.586009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:21:43.586009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:43.586009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:21:43.586009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:21:43.586009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:43.586009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:21:43.586240 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:43.586240 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:21:43.586972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:43.586972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:21:43.586972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:21:43.586972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:21:43.586972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:21:43.586972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:21:43.586972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:21:43.586972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:21:43.586972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:21:43.586972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:21:43.586972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:21:43.586972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:21:43.586972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:21:43.586972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:21:43.586972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:21:43.586972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:21:43.587425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:21:43.587425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:21:43.587425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:43.587425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:21:43.587425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:43.587425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:21:43.587425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:43.587425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:21:43.587425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:43.587425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:21:43.587425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:43.587425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:21:43.587425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:21:43.587425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:43.587425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:21:43.587425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:43.587425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:21:43.587425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:21:43.587425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:43.587937 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:21:43.587937 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:43.587937 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:21:48.392676 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:21:48.393820 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:21:48.393820 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:21:48.394285 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:21:48.432773 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:21:48.433021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:48.433845 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:21:48.433845 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:48.433845 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:21:48.433845 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:21:48.433845 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:48.433845 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:21:48.433845 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:48.433845 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:21:48.436296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:48.436296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:21:48.436296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:21:48.436296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:21:48.436296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:21:48.436296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:21:48.436296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:21:48.436296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:48.436296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:21:48.436296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:48.436296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:21:48.436296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:48.436941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:21:48.436941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:48.436941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:21:48.436941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:48.436941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:21:48.436941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:21:48.436941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:48.436941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:21:48.436941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:48.436941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:21:48.436941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:21:48.436941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:48.436941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:21:48.437516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:48.437516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:21:48.437516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:48.437516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:21:48.437516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:21:48.437516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:21:48.437516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:21:48.437516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:21:48.437516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:21:48.437516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:21:48.437516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:21:48.437516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:21:48.437516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:21:48.437516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:21:48.437516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:21:48.438485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:21:48.438485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:21:48.438485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:21:48.438485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:21:48.438485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:21:48.438485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:48.438485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:21:48.438485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:48.438485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:21:48.438485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:48.438485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:21:48.438485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:48.438485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:21:48.438485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:48.438485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:21:48.438485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:21:48.438485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:48.438485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:21:48.438485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:48.439027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:21:48.439027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:21:48.439027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:48.439027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:21:48.439027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:48.439027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:21:48.556938 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:21:48.557566 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:21:48.557753 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:21:48.558931 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:21:48.559170 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:21:48.559320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:48.559945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:21:48.559945 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:48.559945 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:21:48.559945 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:21:48.559945 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:48.559945 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:21:48.559945 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:48.559945 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:21:48.561002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:48.561002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:21:48.561002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:21:48.561002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:21:48.561002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:21:48.561002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:21:48.561002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:21:48.561002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:48.561002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:21:48.561002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:48.561002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:21:48.561002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:48.561901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:21:48.561901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:48.561901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:21:48.561901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:48.561901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:21:48.561901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:21:48.561901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:48.561901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:21:48.561901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:48.561901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:21:48.561901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:21:48.561901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:48.561901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:21:48.562207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:48.562207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:21:48.562207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:48.562207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:21:48.562207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:21:48.562207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:21:48.562207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:21:48.562207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:21:48.562207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:21:48.562207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:21:48.562207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:21:48.562207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:21:48.562207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:21:48.562207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:21:48.562207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:21:48.562600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:21:48.562600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:21:48.562600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:21:48.562600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:21:48.562600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:21:48.562600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:48.562600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:21:48.562600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:48.562600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:21:48.562600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:48.562600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:21:48.562600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:48.562600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:21:48.562600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:48.562600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:21:48.562600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:21:48.562600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:48.562600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:21:48.562600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:48.563077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:21:48.563077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:21:48.563077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:48.563077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:21:48.563077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:48.563077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:21:48.565665 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:21:48.566162 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:21:48.566358 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:21:48.566560 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:21:48.567213 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:21:48.567408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:48.568041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:21:48.568041 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:48.568041 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:21:48.568041 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:21:48.568041 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:48.568041 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:21:48.568041 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:48.568041 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:21:48.569302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:48.569302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:21:48.569302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:21:48.569302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:21:48.569302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:48.569302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:21:48.569302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:48.569302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:21:48.569302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:48.569302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:21:48.569302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:48.569302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:21:48.569302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:48.569302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:21:48.569302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:21:48.569711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:48.569711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:21:48.569711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:48.569711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:21:48.569711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:21:48.569711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:48.569711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:21:48.569711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:48.569711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:21:48.570718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:48.570718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:21:48.570718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:21:48.570718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:21:48.570718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:21:48.570718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:21:48.570718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:21:48.570718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:21:48.570718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:21:48.570718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:21:48.570718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:48.570718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:21:48.571181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:48.571181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:21:48.571181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:48.571181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:21:48.571181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:48.571181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:21:48.571181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:48.571181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:21:48.571181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:21:48.571181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:48.571181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:21:48.571181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:48.571181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:21:48.571181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:21:48.571555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:48.571555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:21:48.571555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:48.571555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:21:48.572378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:48.572378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:21:48.572378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:21:48.572378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:21:48.572378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:21:48.572378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:21:48.572378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:21:48.572378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:21:48.572378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:21:48.572378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:21:48.572378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:21:48.572378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:21:48.572378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:21:48.572378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:21:48.572378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:21:48.572378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:21:48.572805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:21:48.572805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:21:48.572805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:21:48.572805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:21:48.572805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:21:48.572805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:48.572805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:21:48.572805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:48.572805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:21:48.572805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:48.572805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:21:48.572805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:48.572805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:21:48.572805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:48.572805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:21:48.572805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:21:48.572805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:48.572805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:21:48.573704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:48.573704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:21:48.573704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:21:48.573704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:48.573704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:21:48.573704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:48.573704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:21:48.574966 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:21:48.575229 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:21:48.575341 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:21:48.575509 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:21:48.575984 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:21:48.576224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:48.576731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:21:48.576731 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:48.576731 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:21:48.576731 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:21:48.576731 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:48.576731 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:21:48.576731 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:48.576731 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:21:48.577479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:48.577479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:21:48.577479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:21:48.577479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:21:48.577479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:48.577479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:21:48.577479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:48.577479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:21:48.577479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:48.577479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:21:48.577479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:48.577479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:21:48.577479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:48.577479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:21:48.577479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:21:48.577818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:48.577818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:21:48.577818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:48.577818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:21:48.577818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:21:48.577818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:48.577818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:21:48.577818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:48.577818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:21:48.578344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:48.578344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:21:48.578344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:21:48.578344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:21:48.578344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:21:48.578344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:21:48.578344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:21:48.578344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:21:48.578344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:21:48.578344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:48.578344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:21:48.578575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:48.578575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:21:48.578575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:48.578575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:21:48.578575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:48.578575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:21:48.578575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:48.578575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:21:48.578575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:21:48.578575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:48.578575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:21:48.578575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:48.578575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:21:48.578575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:21:48.578881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:48.578881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:21:48.578881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:48.578881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:21:48.579383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:48.579383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:21:48.579383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:21:48.579383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:21:48.579383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:21:48.579383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:21:48.579383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:21:48.579383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:21:48.579383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:21:48.579383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:21:48.579383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:21:48.579383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:21:48.579383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:21:48.579383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:21:48.579383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:21:48.579383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:21:48.579383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:21:48.579771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:21:48.579771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:21:48.579771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:21:48.579771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:48.579771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:21:48.579771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:48.579771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:21:48.579771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:48.579771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:21:48.579771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:48.579771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:21:48.579771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:48.579771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:21:48.579771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:21:48.579771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:48.579771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:21:48.579771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:48.579771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:21:48.579771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:21:48.580769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:48.580769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:21:48.580769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:48.580769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:21:48.583355 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:21:48.583705 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:21:48.583850 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:21:48.584013 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:21:48.584443 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:21:48.584758 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:21:48.584982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:48.585571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:21:48.585571 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:48.585571 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:21:48.585571 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:21:48.585571 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:48.585571 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:21:48.585571 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:48.585571 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:21:48.586429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:48.586429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:21:48.586429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:21:48.586429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:21:48.586429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:21:48.586429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:21:48.586429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:21:48.586429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:48.586429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:21:48.586429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:48.586429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:21:48.586429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:48.586933 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:21:48.586933 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:48.586933 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:21:48.586933 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:48.586933 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:21:48.586933 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:21:48.586933 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:48.586933 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:21:48.586933 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:48.586933 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:21:48.586933 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:21:48.586933 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:48.586933 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:21:48.587280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:48.587280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:21:48.587421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:48.587421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:21:48.587421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:21:48.587421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:21:48.587421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:21:48.587421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:21:48.587421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:21:48.587421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:21:48.587421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:21:48.587421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:21:48.587421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:21:48.587421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:21:48.587421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:21:48.587421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:21:48.587421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:21:48.587421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:21:48.587905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:21:48.587905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:21:48.587905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:48.587905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:21:48.587905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:48.587905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:21:48.587905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:48.587905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:21:48.587905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:48.587905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:21:48.587905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:48.587905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:21:48.587905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:21:48.587905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:48.587905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:21:48.587905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:48.587905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:21:48.587905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:21:48.587905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:48.588377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:21:48.588377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:48.588377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:21:53.393350 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:21:53.394041 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:21:53.394280 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:21:53.395502 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:21:53.424872 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:21:53.425156 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:53.425836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:21:53.425836 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:53.425836 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:21:53.425836 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:21:53.425836 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:53.425836 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:21:53.425836 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:53.425836 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:21:53.426600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.032 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:53.426600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:21:53.426600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:21:53.426600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:21:53.426600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:21:53.426600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:21:53.426600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:21:53.426600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:53.426600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:21:53.426600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:53.426600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:21:53.426600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:53.427224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:21:53.427224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:53.427224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:21:53.427224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:53.427224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:21:53.427224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:21:53.427224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:53.427224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:21:53.427224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:53.427224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:21:53.427224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:21:53.427224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:53.427224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:21:53.427820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:53.427820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:21:53.427820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:53.427820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:21:53.427820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:21:53.427820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:21:53.427820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:21:53.427820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:21:53.427820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:21:53.427820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:21:53.427820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:21:53.427820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:21:53.427820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:21:53.427820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:21:53.427820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:21:53.428297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:21:53.428297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:21:53.428297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:21:53.428297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:21:53.428297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:21:53.428297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:53.428297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:21:53.428297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:53.428297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:21:53.428297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:53.428297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:21:53.428297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:53.428297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:21:53.428297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:53.428297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:21:53.428297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:21:53.428297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:53.428297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:21:53.428297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:53.428930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:21:53.428930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:21:53.428930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:53.428930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:21:53.428930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:53.428930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:21:53.557677 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:21:53.558157 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:21:53.558248 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:21:53.559058 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:21:53.559225 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:21:53.559225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:53.559903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:21:53.559903 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:53.559903 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:21:53.559903 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:21:53.559903 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:53.559903 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:21:53.559903 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:53.559903 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:21:53.560797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:53.560797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:21:53.560797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:21:53.560797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:21:53.560797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:21:53.560797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:21:53.560797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:21:53.560797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:53.560797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:21:53.560797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:53.560797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:21:53.560797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:53.561068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:21:53.561068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:53.561068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:21:53.561068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:53.561068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:21:53.561068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:21:53.561068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:53.561068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:21:53.561068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:53.561068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:21:53.561068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:21:53.561068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:53.561068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:21:53.561552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:53.561552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:21:53.562019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:53.562019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:21:53.562019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:21:53.562019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:21:53.562019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:21:53.562019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:21:53.562019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:21:53.562019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:21:53.562019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:21:53.562019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:21:53.562019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:21:53.562019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:21:53.562019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:21:53.562019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:21:53.562019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:21:53.562019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:21:53.562830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:21:53.562830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:21:53.562830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:53.562830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:21:53.562830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:53.562830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:21:53.562830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:53.562830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:21:53.562830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:53.562830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:21:53.562830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:53.562830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:21:53.562830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:21:53.562830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:53.562830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:21:53.562830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:53.562830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:21:53.562830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:21:53.562830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:53.564501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:21:53.564501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:53.564501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:21:53.566364 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:21:53.566723 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:21:53.567036 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:21:53.567161 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:21:53.567591 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:21:53.567832 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:53.568299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:21:53.568299 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:53.568299 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:21:53.568299 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:21:53.568299 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:53.568299 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:21:53.568299 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:53.568299 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:21:53.569023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:53.569023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:21:53.569023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:21:53.569023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:21:53.569023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:53.569023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:21:53.569023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:53.569023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:21:53.569023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:53.569023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:21:53.569023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:53.569023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:21:53.569023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:53.569023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:21:53.569023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:21:53.569632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:53.569632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:21:53.569632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:53.569632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:21:53.569632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:21:53.569632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:53.569632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:21:53.569632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:53.569632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:21:53.569954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:53.569954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:21:53.569954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:21:53.569954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:21:53.569954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:21:53.569954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:21:53.569954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:21:53.569954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:21:53.569954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:21:53.569954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:21:53.569954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:53.569954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:21:53.570334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:53.570334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:21:53.570334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:53.570334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:21:53.570334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:53.570334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:21:53.570334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:53.570334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:21:53.570334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:21:53.570334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:53.570334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:21:53.570334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:53.570334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:21:53.570334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:21:53.570721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:53.570721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:21:53.570721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:53.570721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:21:53.570948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:53.570948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:21:53.570948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:21:53.570948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:21:53.570948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:21:53.570948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:21:53.570948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:21:53.570948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:21:53.570948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:21:53.570948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:21:53.570948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:21:53.570948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:21:53.570948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:21:53.570948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:21:53.570948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:21:53.570948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:21:53.571392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:21:53.571392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:21:53.571392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:21:53.571392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:21:53.571392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:21:53.571392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:53.571392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:21:53.571392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:53.571392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:21:53.571392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:53.571392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:21:53.571392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:53.571392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:21:53.571392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:53.571392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:21:53.571392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:21:53.571392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:53.571392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:21:53.571910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:53.571910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:21:53.571910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:21:53.571910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:53.571910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:21:53.571910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:53.571910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:21:53.575537 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:21:53.575857 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:21:53.575965 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:21:53.576045 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:21:53.578353 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:21:53.579019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:53.579718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:21:53.579718 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:53.579718 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:21:53.579718 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:21:53.579718 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:53.579718 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:21:53.579718 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:53.579718 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:21:53.580336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:53.580336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:21:53.580336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:21:53.580336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:21:53.580336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:53.580336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:21:53.580336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:53.580336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:21:53.580336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:53.580336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:21:53.580336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:53.580336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:21:53.580336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:53.580336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:21:53.580336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:21:53.580938 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:53.580938 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:21:53.580938 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:53.580938 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:21:53.580938 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:21:53.580938 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:53.580938 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:21:53.580938 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:53.580938 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:21:53.581302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:53.581302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:21:53.581302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:21:53.581302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:21:53.581302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:21:53.581302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:21:53.581302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:21:53.581302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:21:53.581302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:21:53.581302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:53.581302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:21:53.581860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:53.581860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:21:53.581860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:53.581860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:21:53.581860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:53.581860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:21:53.581860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:53.581860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:21:53.581860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:21:53.581860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:53.581860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:21:53.581860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:53.581860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:21:53.581860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:21:53.582945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:53.582945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:21:53.582945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:53.582945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:21:53.582945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:53.582945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:21:53.582945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:21:53.582945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:21:53.582945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:21:53.582945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:21:53.582945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:21:53.582945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:21:53.582945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:21:53.582945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:21:53.583539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:21:53.583539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:21:53.583539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:21:53.583539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:21:53.583539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:21:53.583539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:21:53.583539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:21:53.583539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:21:53.583539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:21:53.583539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:21:53.583539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:53.583539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:21:53.583539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:53.583539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:21:53.583539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:53.583539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:21:53.583539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:53.583539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:21:53.583539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:53.584326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:21:53.584326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:21:53.584326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:53.584326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:21:53.584326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:53.584326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:21:53.584326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:21:53.584326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:53.584326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:21:53.584326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:53.584326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:21:53.584326 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:21:53.584326 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:21:53.584759 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:21:53.584759 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:21:53.584759 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:21:53.585002 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:21:53.585245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:53.586045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:21:53.586045 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:53.586045 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:21:53.586045 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:21:53.586045 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:53.586045 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:21:53.586045 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:53.586045 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:21:53.587037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:53.587037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:21:53.587037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:21:53.587037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:21:53.587037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:21:53.587037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:21:53.587037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:21:53.587037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:53.587037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:21:53.587037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:53.587037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:21:53.587037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:53.587338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:21:53.587338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:53.587338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:21:53.587338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:53.587338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:21:53.587338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:21:53.587338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:53.587338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:21:53.587338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:53.587338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:21:53.587338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:21:53.587338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:53.587338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:21:53.587561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:53.587561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:21:53.588233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:53.588233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:21:53.588233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:21:53.588233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:21:53.588233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:21:53.588233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:21:53.588233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:21:53.588233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:21:53.588233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:21:53.588233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:21:53.588233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:21:53.588233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:21:53.588233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:21:53.588233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:21:53.588233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:21:53.588233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:21:53.589181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:21:53.589181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:21:53.589181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:53.589181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:21:53.589181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:53.589181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:21:53.589181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:53.589181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:21:53.589181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:53.589181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:21:53.589181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:53.589181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:21:53.589181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:21:53.589181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:53.589181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:21:53.589181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:53.589181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:21:53.589181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:21:53.589181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:53.589799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:21:53.589799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:53.589799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:21:58.394317 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:21:58.395092 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:21:58.395092 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:21:58.395801 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:21:58.402084 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG dbcounter [-] [106314] Writing DB stats neutron:SELECT=241 {{(pid=106314) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 09 09:21:58.440192 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:21:58.440590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:58.441152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:21:58.441152 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:58.441152 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:21:58.441152 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:21:58.441152 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:58.441152 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:21:58.441152 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:58.441152 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:21:58.442072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.046 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:58.442072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:21:58.442072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:21:58.442072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:21:58.442072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:21:58.442072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:21:58.442072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:21:58.442072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:58.442072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:21:58.442072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:58.442072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:21:58.442072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:58.442438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:21:58.442438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:58.442438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:21:58.442438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:58.442438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:21:58.442438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:21:58.442438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:58.442438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:21:58.442438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:58.442438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:21:58.442438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:21:58.442438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:58.442438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:21:58.442725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:58.442725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:21:58.443085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:58.443085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:21:58.443085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:21:58.443085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:21:58.443085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:21:58.443085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:21:58.443085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:21:58.443085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:21:58.443085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:21:58.443085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:21:58.443085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:21:58.443085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:21:58.443085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:21:58.443085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:21:58.443085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:21:58.443085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:21:58.444295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:21:58.444295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:21:58.444295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:58.444295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:21:58.444295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:58.444295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:21:58.444295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:58.444295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:21:58.444295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:58.444295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:21:58.444295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:58.444295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:21:58.444295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:21:58.444295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:58.444295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:21:58.444295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:58.444295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:21:58.444295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:21:58.444295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:58.444792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:21:58.444792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:58.444792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:21:58.558133 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:21:58.558785 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:21:58.559108 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:21:58.561398 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:21:58.561398 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:21:58.561398 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:58.561709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:21:58.561709 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:58.561709 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:21:58.561709 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:21:58.561709 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:58.561709 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:21:58.561709 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:58.561709 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:21:58.563663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:58.563663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:21:58.563663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:21:58.563663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:21:58.563663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:21:58.563663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:21:58.563663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:21:58.563663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:58.563663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:21:58.563663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:58.563663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:21:58.563663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:58.565271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:21:58.565271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:58.565271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:21:58.565271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:58.565271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:21:58.565271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:21:58.565271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:58.565271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:21:58.565271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:58.565271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:21:58.565271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:21:58.565271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:58.565271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:21:58.566072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:58.566072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:21:58.566700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:58.566700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:21:58.566700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:21:58.566700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:21:58.566700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:21:58.566700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:21:58.566700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:21:58.566700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:21:58.566700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:21:58.566700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:21:58.566700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:21:58.566700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:21:58.566700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:21:58.566700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:21:58.566700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:21:58.566700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:21:58.569473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:21:58.569473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:21:58.569473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:58.569473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:21:58.569473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:58.569473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:21:58.569473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:58.569473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:21:58.569473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:58.569473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:21:58.569473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:58.569473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:21:58.569473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:21:58.569473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:58.569473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:21:58.569473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:58.569473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:21:58.569473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:21:58.569473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:58.570525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:21:58.570525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:58.570525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:21:58.570525 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:21:58.570525 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:21:58.570525 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:21:58.570525 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:21:58.570525 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:21:58.571013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:58.571013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:21:58.571013 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:58.571013 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:21:58.571013 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:21:58.571013 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:58.571013 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:21:58.571013 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:58.571013 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:21:58.571013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:58.571013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:21:58.571013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:21:58.571013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:21:58.571013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:58.571013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:21:58.571013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:58.571013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:21:58.571512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:58.571512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:21:58.571512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:58.571512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:21:58.571512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:58.571512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:21:58.571512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:21:58.571512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:58.571512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:21:58.571512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:58.571512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:21:58.571512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:21:58.571512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:58.571512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:21:58.571512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:58.571512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:21:58.572389 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:58.572389 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:21:58.572389 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:21:58.572389 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:21:58.572389 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:21:58.572389 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:21:58.572389 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:21:58.572389 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:21:58.572389 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:21:58.572389 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:21:58.572389 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:58.572389 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:21:58.573007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:58.573007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:21:58.573007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:58.573007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:21:58.573007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:58.573007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:21:58.573007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:58.573007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:21:58.573007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:21:58.573007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:58.573007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:21:58.573007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:58.573007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:21:58.573007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:21:58.573538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:58.573538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:21:58.573538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:58.573538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:21:58.573538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:58.573538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:21:58.573538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:21:58.573538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:21:58.573538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:21:58.573538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:21:58.573538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:21:58.573538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:21:58.573538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:21:58.573538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:21:58.573842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:21:58.573842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:21:58.573842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:21:58.573842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:21:58.573842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:21:58.573842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:21:58.573842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:21:58.573842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:21:58.573842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:21:58.573842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:21:58.573842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:21:58.573842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:58.573842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:21:58.573842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:58.573842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:21:58.573842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:58.573842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:21:58.573842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:58.573842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:21:58.574175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:58.574175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:21:58.574175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:21:58.574175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:58.574175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:21:58.574175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:58.574175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:21:58.574175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:21:58.574175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:58.574175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:21:58.574175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:58.574175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:21:58.575993 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:21:58.576283 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:21:58.576432 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:21:58.576546 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:21:58.576966 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:21:58.577258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:58.577729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:21:58.577729 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:58.577729 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:21:58.577729 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:21:58.577729 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:58.577729 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:21:58.577729 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:58.577729 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:21:58.578448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:58.578448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:21:58.578448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:21:58.578448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:21:58.578448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:58.578448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:21:58.578448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:58.578448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:21:58.578448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:58.578448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:21:58.578448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:58.578448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:21:58.578448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:58.578448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:21:58.578448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:21:58.579523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:58.579523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:21:58.579523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:58.579523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:21:58.579523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:21:58.579523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:58.579523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:21:58.579523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:58.579523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:21:58.579523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:58.579523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:21:58.579523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:21:58.579523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:21:58.579523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:21:58.579903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:21:58.579903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:21:58.579903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:21:58.579903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:21:58.579903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:58.579903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:21:58.579903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:58.579903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:21:58.579903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:58.579903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:21:58.579903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:58.579903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:21:58.579903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:58.580250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:21:58.580250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:21:58.580250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:58.580250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:21:58.580250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:58.580250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:21:58.580250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:21:58.580250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:58.580250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:21:58.580250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:58.580250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:21:58.580531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:58.580531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:21:58.580531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:21:58.580531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:21:58.580531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:21:58.580531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:21:58.580531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:21:58.580531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:21:58.580531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:21:58.580531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:21:58.580531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:21:58.580531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:21:58.580531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:21:58.580531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:21:58.580531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:21:58.580531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:21:58.580531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:21:58.580862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:21:58.580862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:21:58.580862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:21:58.580862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:58.580862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:21:58.580862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:58.580862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:21:58.580862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:58.580862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:21:58.580862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:58.580862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:21:58.580862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:58.580862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:21:58.580862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:21:58.580862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:58.580862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:21:58.580862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:58.580862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:21:58.580862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:21:58.581197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:58.581197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:21:58.581197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:58.581197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:21:58.584297 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:21:58.584621 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:21:58.584813 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:21:58.585005 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:21:58.586070 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:21:58.586070 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:21:58.586070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:58.586385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:21:58.586385 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:58.586385 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:21:58.586385 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:21:58.586385 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:58.586385 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:21:58.586385 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:58.586385 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:21:58.587369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:58.587369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:21:58.587369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:21:58.587369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:21:58.587369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:21:58.587369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:21:58.587369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:21:58.587369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:58.587369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:21:58.587369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:58.587369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:21:58.587369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:58.587868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:21:58.587868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:58.587868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:21:58.587868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:58.587868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:21:58.587868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:21:58.587868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:58.587868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:21:58.587868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:58.587868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:21:58.587868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:21:58.587868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:58.587868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:21:58.588309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:58.588309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:21:58.588890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:58.588890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:21:58.588890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:21:58.588890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:21:58.588890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:21:58.588890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:21:58.588890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:21:58.588890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:21:58.588890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:21:58.588890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:21:58.588890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:21:58.588890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:21:58.588890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:21:58.588890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:21:58.588890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:21:58.588890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:21:58.589681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:21:58.589681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:21:58.589681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:58.589681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:21:58.589681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:21:58.589681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:21:58.589681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:21:58.589681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:21:58.589681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:21:58.589681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:21:58.589681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:21:58.589681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:21:58.589681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:21:58.589681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:21:58.589681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:21:58.589681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:21:58.589681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:21:58.589681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:21:58.589681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:21:58.590150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:21:58.590150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:21:58.590150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:22:03.394776 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:22:03.395563 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:22:03.395563 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:22:03.396617 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:22:03.451974 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:22:03.452212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:03.453885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:22:03.453885 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:03.453885 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:22:03.453885 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:22:03.453885 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:03.453885 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:22:03.453885 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:03.453885 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:22:03.454388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.058 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:03.454388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:22:03.454388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:22:03.454388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:22:03.454388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:22:03.454388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:22:03.454388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:22:03.454388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:03.454388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:22:03.454388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:03.454388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:22:03.454388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:03.454701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:22:03.454701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:03.454701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:22:03.454701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:03.454701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:22:03.454701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:22:03.454701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:03.454701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:22:03.454701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:03.454701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:22:03.454701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:22:03.454701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:03.454701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:22:03.455027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:03.455027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:22:03.455473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:03.455473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:22:03.455473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:22:03.455473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:22:03.455473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:22:03.455473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:22:03.455473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:22:03.455473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:22:03.455473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:22:03.455473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:22:03.455473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:22:03.455473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:22:03.455473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:22:03.455473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:22:03.455473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:22:03.455473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:22:03.455834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:22:03.455834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:22:03.455834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:03.455834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:22:03.455834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:03.455834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:22:03.455834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:03.455834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:22:03.455834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:03.455834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:22:03.455834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:03.455834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:22:03.455834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:22:03.455834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:03.455834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:22:03.455834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:03.455834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:22:03.455834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:22:03.455834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:03.456243 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:22:03.456243 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:03.456243 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:22:03.559024 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:22:03.559498 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:22:03.559698 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:22:03.561317 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:22:03.562587 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:22:03.563008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:03.563968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:22:03.563968 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:03.563968 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:22:03.563968 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:22:03.563968 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:03.563968 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:22:03.563968 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:03.563968 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:22:03.565011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:03.565011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:22:03.565011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:22:03.565011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:22:03.565011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:22:03.565011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:22:03.565011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:22:03.565011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:03.565011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:22:03.565011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:03.565011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:22:03.565011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:03.565726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:22:03.565726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:03.565726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:22:03.565726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:03.565726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:22:03.565726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:22:03.565726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:03.565726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:22:03.565726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:03.565726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:22:03.565726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:22:03.565726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:03.565726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:22:03.566346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:03.566346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:22:03.566346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:03.566346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:22:03.566346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:22:03.566346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:22:03.566346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:22:03.566346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:22:03.566346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:22:03.566346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:22:03.566346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:22:03.566346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:22:03.566346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:22:03.566346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:22:03.566346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:22:03.567095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:22:03.567095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:22:03.567095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:22:03.567095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:22:03.567095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:22:03.567095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:03.567095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:22:03.567095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:03.567095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:22:03.567095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:03.567095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:22:03.567095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:03.567095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:22:03.567095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:03.567095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:22:03.567095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:22:03.567095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:03.567095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:22:03.567095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:03.567982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:22:03.567982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:22:03.567982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:03.567982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:22:03.567982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:03.567982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:22:03.567982 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:22:03.567982 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:22:03.567982 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:22:03.567982 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:22:03.568951 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:22:03.569706 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:03.570407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:22:03.570407 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:03.570407 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:22:03.570407 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:22:03.570407 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:03.570407 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:22:03.570407 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:03.570407 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:22:03.571228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:03.571228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:22:03.571228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:22:03.571228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:22:03.571228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:03.571228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:22:03.571228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:03.571228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:22:03.571228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:03.571228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:22:03.571228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:03.571228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:22:03.571228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:03.571228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:22:03.571228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:22:03.571719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:03.571719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:22:03.571719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:03.571719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:22:03.571719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:22:03.571719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:03.571719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:22:03.571719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:03.571719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:22:03.572108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:03.572108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:22:03.572108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:22:03.572108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:22:03.572108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:22:03.572108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:22:03.572108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:22:03.572108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:22:03.572108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:22:03.572108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:22:03.572108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:03.572108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:22:03.572468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:03.572468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:22:03.572468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:03.572468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:22:03.572468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:03.572468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:22:03.572468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:03.572468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:22:03.572468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:22:03.572468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:03.572468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:22:03.572468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:03.572468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:22:03.572468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:22:03.572863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:03.572863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:22:03.572863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:03.572863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:22:03.573157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:03.573157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:22:03.573157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:22:03.573157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:22:03.573157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:22:03.573157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:22:03.573157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:22:03.573157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:22:03.573157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:22:03.573157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:22:03.573157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:22:03.573157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:22:03.573157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:22:03.573157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:22:03.573157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:22:03.573157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:22:03.574653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:22:03.574653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:22:03.574653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:22:03.574653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:22:03.574653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:22:03.574653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:03.574653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:22:03.574653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:03.574653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:22:03.574653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:03.574653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:22:03.574653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:03.574653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:22:03.574653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:03.574653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:22:03.574653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:22:03.574653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:03.574653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:22:03.575217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:03.575217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:22:03.575217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:22:03.575217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:03.575217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:22:03.575217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:03.575217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:22:03.576308 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:22:03.576525 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:22:03.576646 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:22:03.576806 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:22:03.577309 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:22:03.577529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:03.578063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:22:03.578063 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:03.578063 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:22:03.578063 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:22:03.578063 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:03.578063 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:22:03.578063 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:03.578063 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:22:03.578800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:03.578800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:22:03.578800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:22:03.578800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:22:03.578800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:03.578800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:22:03.578800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:03.578800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:22:03.578800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:03.578800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:22:03.578800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:03.578800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:22:03.578800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:03.578800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:22:03.578800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:22:03.579231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:03.579231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:22:03.579231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:03.579231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:22:03.579231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:22:03.579231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:03.579231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:22:03.579231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:03.579231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:22:03.579697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:03.579697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:22:03.579697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:22:03.579697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:22:03.579697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:22:03.579697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:22:03.579697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:22:03.579697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:22:03.579697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:22:03.579697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:03.579697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:22:03.580107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:03.580107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:22:03.580107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:03.580107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:22:03.580107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:03.580107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:22:03.580107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:03.580107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:22:03.580107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:22:03.580107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:03.580107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:22:03.580107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:03.580107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:22:03.580107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:22:03.580621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:03.580621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:22:03.580621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:03.580621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:22:03.580856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:03.580856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:22:03.580856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:22:03.580856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:22:03.580856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:22:03.580856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:22:03.580856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:22:03.580856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:22:03.580856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:22:03.580856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:22:03.580856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:22:03.580856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:22:03.580856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:22:03.580856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:22:03.580856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:22:03.580856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:22:03.580856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:22:03.581506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:22:03.581506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:22:03.581506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:22:03.581506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:03.581506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:22:03.581506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:03.581506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:22:03.581506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:03.581506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:22:03.581506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:03.581506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:22:03.581506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:03.581506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:22:03.581506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:22:03.581506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:03.581506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:22:03.581506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:03.581506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:22:03.581506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:22:03.582083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:03.582083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:22:03.582083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:03.582083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:22:03.584651 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:22:03.584903 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:22:03.585021 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:22:03.585313 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:22:03.585794 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:22:03.586619 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:22:03.586837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:03.588880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:22:03.588880 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:03.588880 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:22:03.588880 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:22:03.588880 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:03.588880 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:22:03.588880 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:03.588880 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:22:03.590871 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:03.590871 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:22:03.590871 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:22:03.590871 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:22:03.590871 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:22:03.590871 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:22:03.590871 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:22:03.590871 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:03.590871 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:22:03.590871 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:03.590871 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:22:03.590871 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:03.591282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:22:03.591282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:03.591282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:22:03.591282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:03.591282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:22:03.591282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:22:03.591282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:03.591282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:22:03.591282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:03.591282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:22:03.591282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:22:03.591282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:03.591282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:22:03.591672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:03.591672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:22:03.592939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:03.592939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:22:03.592939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:22:03.592939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:22:03.592939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:22:03.592939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:22:03.592939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:22:03.592939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:22:03.592939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:22:03.592939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:22:03.592939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:22:03.592939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:22:03.592939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:22:03.592939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:22:03.592939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:22:03.592939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:22:03.593446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:22:03.593446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:22:03.593446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:03.593446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:22:03.593446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:03.593446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:22:03.593446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:03.593446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:22:03.593446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:03.593446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:22:03.593446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:03.593446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:22:03.593446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:22:03.593446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:03.593446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:22:03.593446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:03.593446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:22:03.593446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:22:03.593446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:03.594069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:22:03.594069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:03.594069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:22:08.395648 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:22:08.396489 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:22:08.396489 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:22:08.397274 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:22:08.437704 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:22:08.437917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:08.438626 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:22:08.438626 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:08.438626 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:22:08.438626 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:22:08.438626 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:08.438626 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:22:08.438626 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:08.438626 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:22:08.439725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:08.439725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:22:08.439725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:22:08.439725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:22:08.439725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:22:08.439725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:22:08.439725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:22:08.439725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:08.439725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:22:08.439725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:08.439725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:22:08.439725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:08.440318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:22:08.440318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:08.440318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:22:08.440318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:08.440318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:22:08.440318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:22:08.440318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:08.440318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:22:08.440318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:08.440318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:22:08.440318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:22:08.440318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:08.440318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:22:08.440883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:08.440883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:22:08.440883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:08.440883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:22:08.440883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:22:08.440883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:22:08.440883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:22:08.440883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:22:08.440883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:22:08.440883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:22:08.440883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:22:08.440883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:22:08.440883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:22:08.440883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:22:08.440883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:22:08.441428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:22:08.441428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:22:08.441428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:22:08.441428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:22:08.441428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:22:08.441428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:08.441428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:22:08.441428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:08.441428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:22:08.441428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:08.441428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:22:08.441428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:08.441428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:22:08.441428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:08.441428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:22:08.441428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:22:08.441428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:08.441428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:22:08.441428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:08.442312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:22:08.442312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:22:08.442312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:08.442312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:22:08.442312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:08.442312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:22:08.559713 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:22:08.560177 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:22:08.560367 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:22:08.561252 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:22:08.561483 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:22:08.561646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:08.562284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:22:08.562284 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:08.562284 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:22:08.562284 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:22:08.562284 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:08.562284 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:22:08.562284 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:08.562284 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:22:08.563492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:08.563492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:22:08.563492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:22:08.563492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:22:08.563492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:22:08.563492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:22:08.563492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:22:08.563492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:08.563492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:22:08.563492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:08.563492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:22:08.563492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:08.564038 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:22:08.564038 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:08.564038 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:22:08.564038 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:08.564038 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:22:08.564038 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:22:08.564038 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:08.564038 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:22:08.564038 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:08.564038 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:22:08.564038 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:22:08.564038 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:08.564038 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:22:08.564419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:08.564419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:22:08.564582 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:08.564582 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:22:08.564582 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:22:08.564582 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:22:08.564582 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:22:08.564582 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:22:08.564582 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:22:08.564582 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:22:08.564582 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:22:08.564582 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:22:08.564582 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:22:08.564582 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:22:08.564582 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:22:08.564582 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:22:08.564582 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:22:08.564582 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:22:08.565094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:22:08.565094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:22:08.565094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:08.565094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:22:08.565094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:08.565094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:22:08.565094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:08.565094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:22:08.565094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:08.565094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:22:08.565094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:08.565094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:22:08.565094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:22:08.565094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:08.565094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:22:08.565094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:08.565094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:22:08.565094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:22:08.565094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:08.565640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:22:08.565640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:08.565640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:22:08.567940 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:22:08.568414 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:22:08.568609 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:22:08.568891 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:22:08.570167 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:22:08.570396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:08.571026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:22:08.571026 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:08.571026 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:22:08.571026 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:22:08.571026 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:08.571026 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:22:08.571026 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:08.571026 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:22:08.571814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:08.571814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:22:08.571814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:22:08.571814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:22:08.571814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:08.571814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:22:08.571814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:08.571814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:22:08.571814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:08.571814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:22:08.571814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:08.571814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:22:08.571814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:08.571814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:22:08.571814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:22:08.572265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:08.572265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:22:08.572265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:08.572265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:22:08.572265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:22:08.572265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:08.572265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:22:08.572265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:08.572265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:22:08.572631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:08.572631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:22:08.572631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:22:08.572631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:22:08.572631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:22:08.572631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:22:08.572631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:22:08.572631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:22:08.572631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:22:08.572631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:22:08.572631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:08.572631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:22:08.572999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:08.572999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:22:08.572999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:08.572999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:22:08.572999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:08.572999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:22:08.572999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:08.572999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:22:08.572999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:22:08.572999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:08.572999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:22:08.572999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:08.572999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:22:08.572999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:22:08.573416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:08.573416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:22:08.573416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:08.573416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:22:08.573879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:08.573879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:22:08.573879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:22:08.573879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:22:08.573879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:22:08.573879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:22:08.573879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:22:08.573879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:22:08.573879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:22:08.573879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:22:08.573879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:22:08.573879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:22:08.573879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:22:08.573879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:22:08.573879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:22:08.573879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:22:08.574359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:22:08.574359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:22:08.574359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:22:08.574359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:22:08.574359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:22:08.574359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:08.574359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:22:08.574359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:08.574359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:22:08.574359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:08.574359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:22:08.574359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:08.574359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:22:08.574359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:08.574359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:22:08.574359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:22:08.574359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:08.574359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:22:08.574913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:08.574913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:22:08.574913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:22:08.574913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:08.574913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:22:08.574913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:08.574913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:22:08.576625 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:22:08.576954 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:22:08.577063 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:22:08.577196 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:22:08.577627 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:22:08.577923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:08.578481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:22:08.578481 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:08.578481 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:22:08.578481 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:22:08.578481 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:08.578481 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:22:08.578481 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:08.578481 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:22:08.579304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:08.579304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:22:08.579304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:22:08.579304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:22:08.579304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:08.579304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:22:08.579304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:08.579304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:22:08.579304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:08.579304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:22:08.579304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:08.579304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:22:08.579304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:08.579304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:22:08.579304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:22:08.579963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:08.579963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:22:08.579963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:08.579963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:22:08.579963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:22:08.579963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:08.579963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:22:08.579963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:08.579963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:22:08.580399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:08.580399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:22:08.580399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:22:08.580399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:22:08.580399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:22:08.580399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:22:08.580399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:22:08.580399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:22:08.580399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:22:08.580399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:08.580399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:22:08.580759 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:08.580759 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:22:08.580759 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:08.580759 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:22:08.580759 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:08.580759 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:22:08.580759 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:08.580759 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:22:08.580759 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:22:08.580759 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:08.580759 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:22:08.580759 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:08.580759 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:22:08.580759 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:22:08.581219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:08.581219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:22:08.581219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:08.581219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:22:08.581780 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:08.581780 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:22:08.581780 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:22:08.581780 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:22:08.581780 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:22:08.581780 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:22:08.581780 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:22:08.581780 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:22:08.581780 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:22:08.581780 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:22:08.581780 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:22:08.581780 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:22:08.581780 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:22:08.581780 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:22:08.581780 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:22:08.581780 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:22:08.581780 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:22:08.582326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:22:08.582326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:22:08.582326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:22:08.582326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:08.582326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:22:08.582326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:08.582326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:22:08.582326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:08.582326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:22:08.582326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:08.582326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:22:08.582326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:08.582326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:22:08.582326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:22:08.582326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:08.582326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:22:08.582326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:08.582326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:22:08.582326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:22:08.582940 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:08.582940 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:22:08.582940 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:08.582940 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:22:08.584942 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:22:08.585336 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:22:08.585544 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:22:08.585822 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:22:08.586147 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:22:08.586314 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:22:08.586514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:08.586947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:22:08.586947 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:08.586947 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:22:08.586947 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:22:08.586947 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:08.586947 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:22:08.586947 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:08.586947 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:22:08.589078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:08.589078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:22:08.589078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:22:08.589078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:22:08.589078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:22:08.589078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:22:08.589078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:22:08.589078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:08.589078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:22:08.589078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:08.589078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:22:08.589078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:08.589490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:22:08.589490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:08.589490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:22:08.589490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:08.589490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:22:08.589490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:22:08.589490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:08.589490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:22:08.589490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:08.589490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:22:08.589490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:22:08.589490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:08.589490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:22:08.589910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:08.589910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:22:08.590676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:08.590676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:22:08.590676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:22:08.590676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:22:08.590676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:22:08.590676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:22:08.590676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:22:08.590676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:22:08.590676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:22:08.590676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:22:08.590676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:22:08.590676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:22:08.590676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:22:08.590676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:22:08.590676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:22:08.590676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:22:08.591295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:22:08.591295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:22:08.591295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:08.591295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:22:08.591295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:08.591295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:22:08.591295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:08.591295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:22:08.591295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:08.591295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:22:08.591295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:08.591295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:22:08.591295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:22:08.591295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:08.591295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:22:08.591295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:08.591295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:22:08.591295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:22:08.591295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:08.591887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:22:08.591887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:08.591887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:22:13.396480 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:22:13.397498 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:22:13.397498 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:22:13.399252 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:22:13.431850 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:22:13.432169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:13.433419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:22:13.433419 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:13.433419 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:22:13.433419 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:22:13.433419 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:13.433419 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:22:13.433419 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:13.433419 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:22:13.434874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:13.434874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:22:13.434874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:22:13.434874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:22:13.434874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:22:13.434874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:22:13.434874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:22:13.434874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:13.434874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:22:13.434874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:13.434874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:22:13.434874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:13.436150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:22:13.436150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:13.436150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:22:13.436150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:13.436150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:22:13.436150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:22:13.436150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:13.436150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:22:13.436150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:13.436150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:22:13.436150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:22:13.436150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:13.436150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:22:13.436534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:13.436534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:22:13.436534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:13.436534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:22:13.436534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:22:13.436534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:22:13.436534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:22:13.436534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:22:13.436534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:22:13.436534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:22:13.436534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:22:13.436534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:22:13.436534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:22:13.436534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:22:13.436534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:22:13.437064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:22:13.437064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:22:13.437064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:22:13.437064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:22:13.437064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:22:13.437064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:13.437064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:22:13.437064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:13.437064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:22:13.437064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:13.437064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:22:13.437064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:13.437064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:22:13.437064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:13.437064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:22:13.437064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:22:13.437064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:13.437064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:22:13.437064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:13.437925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:22:13.437925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:22:13.437925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:13.437925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:22:13.437925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:13.437925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:22:13.560713 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:22:13.561374 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:22:13.562503 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:22:13.564636 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:22:13.564636 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:22:13.564636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:13.564636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:22:13.564636 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:13.564636 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:22:13.564636 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:22:13.564636 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:13.564636 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:22:13.564636 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:13.564636 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:22:13.566041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:13.566041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:22:13.566041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:22:13.566041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:22:13.566041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:22:13.566041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:22:13.566041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:22:13.566041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:13.566041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:22:13.566041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:13.566041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:22:13.566041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:13.566722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:22:13.566722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:13.566722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:22:13.566722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:13.566722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:22:13.566722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:22:13.566722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:13.566722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:22:13.566722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:13.566722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:22:13.566722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:22:13.566722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:13.566722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:22:13.567069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:13.567069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:22:13.567149 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:13.567149 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:22:13.567149 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:22:13.567149 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:22:13.567149 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:22:13.567149 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:22:13.567149 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:22:13.567149 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:22:13.567149 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:22:13.567149 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:22:13.567149 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:22:13.567149 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:22:13.567149 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:22:13.567149 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:22:13.567149 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:22:13.567149 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:22:13.567569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:22:13.567569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:22:13.567569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:13.567569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:22:13.567569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:13.567569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:22:13.567569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:13.567569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:22:13.567569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:13.567569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:22:13.567569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:13.567569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:22:13.567569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:22:13.567569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:13.567569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:22:13.567569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:13.567569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:22:13.567569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:22:13.567569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:13.569573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:22:13.569573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:13.569573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:22:13.569573 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:22:13.569573 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:22:13.569573 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:22:13.569573 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:22:13.569573 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:22:13.569988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:13.570411 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:22:13.570411 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:13.570411 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:22:13.570411 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:22:13.570411 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:13.570411 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:22:13.570411 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:13.570411 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:22:13.571172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:13.571172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:22:13.571172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:22:13.571172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:22:13.571172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:13.571172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:22:13.571172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:13.571172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:22:13.571172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:13.571172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:22:13.571172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:13.571172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:22:13.571172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:13.571172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:22:13.571172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:22:13.571656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:13.571656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:22:13.571656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:13.571656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:22:13.571656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:22:13.571656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:13.571656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:22:13.571656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:13.571656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:22:13.572382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:13.572382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:22:13.572382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:22:13.572382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:22:13.572382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:22:13.572382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:22:13.572382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:22:13.572382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:22:13.572382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:22:13.572382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:22:13.572382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:13.572382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:22:13.573236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:13.573236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:22:13.573236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:13.573236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:22:13.573236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:13.573236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:22:13.573236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:13.573236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:22:13.573236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:22:13.573236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:13.573236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:22:13.573236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:13.573236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:22:13.573236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:22:13.573618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:13.573618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:22:13.573618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:13.573618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:22:13.573618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:13.573618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:22:13.573618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:22:13.573618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:22:13.573618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:22:13.573618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:22:13.573618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:22:13.573618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:22:13.573618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:22:13.573618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:22:13.574104 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:22:13.574104 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:22:13.574104 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:22:13.574104 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:22:13.574104 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:22:13.574104 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:22:13.574104 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:22:13.574104 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:22:13.574104 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:22:13.574104 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:22:13.574104 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:22:13.574104 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:13.574104 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:22:13.574104 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:13.574104 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:22:13.574104 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:13.574104 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:22:13.574104 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:13.574104 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:22:13.574619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:13.574619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:22:13.574619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:22:13.574619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:13.574619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:22:13.574619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:13.574619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:22:13.574619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:22:13.574619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:13.574619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:22:13.574619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:13.574619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:22:13.576999 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:22:13.577319 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:22:13.577592 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:22:13.577730 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:22:13.578297 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:22:13.578646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:13.579319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:22:13.579319 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:13.579319 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:22:13.579319 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:22:13.579319 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:13.579319 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:22:13.579319 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:13.579319 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:22:13.580120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:13.580120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:22:13.580120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:22:13.580120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:22:13.580120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:13.580120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:22:13.580120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:13.580120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:22:13.580120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:13.580120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:22:13.580120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:13.580120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:22:13.580120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:13.580120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:22:13.580120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:22:13.580466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:13.580466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:22:13.580466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:13.580466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:22:13.580466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:22:13.580466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:13.580466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:22:13.580466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:13.580466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:22:13.581035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:13.581035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:22:13.581035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:22:13.581035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:22:13.581035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:22:13.581035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:22:13.581035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:22:13.581035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:22:13.581035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:22:13.581035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:13.581035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:22:13.581310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:13.581310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:22:13.581310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:13.581310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:22:13.581310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:13.581310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:22:13.581310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:13.581310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:22:13.581310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:22:13.581310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:13.581310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:22:13.581310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:13.581310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:22:13.581310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:22:13.581562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:13.581562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:22:13.581562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:13.581562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:22:13.582188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:13.582188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:22:13.582188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:22:13.582188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:22:13.582188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:22:13.582188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:22:13.582188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:22:13.582188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:22:13.582188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:22:13.582188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:22:13.582188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:22:13.582188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:22:13.582188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:22:13.582188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:22:13.582188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:22:13.582188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:22:13.582188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:22:13.582593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:22:13.582593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:22:13.582593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:22:13.582593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:13.582593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:22:13.582593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:13.582593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:22:13.582593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:13.582593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:22:13.582593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:13.582593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:22:13.582593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:13.582593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:22:13.582593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:22:13.582593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:13.582593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:22:13.582593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:13.582593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:22:13.582593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:22:13.582940 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:13.582940 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:22:13.582940 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:13.582940 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:22:13.585385 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:22:13.585694 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:22:13.585829 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:22:13.585985 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:22:13.586476 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:22:13.586909 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:22:13.587211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:13.587937 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:22:13.587937 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:13.587937 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:22:13.587937 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:22:13.587937 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:13.587937 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:22:13.587937 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:13.587937 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:22:13.588872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:13.588872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:22:13.588872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:22:13.588872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:22:13.588872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:22:13.588872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:22:13.588872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:22:13.588872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:13.588872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:22:13.588872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:13.588872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:22:13.588872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:13.589322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:22:13.589322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:13.589322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:22:13.589322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:13.589322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:22:13.589322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:22:13.589322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:13.589322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:22:13.589322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:13.589322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:22:13.589322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:22:13.589322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:13.589322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:22:13.589734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:13.589734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:22:13.590025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:13.590025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:22:13.590025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:22:13.590025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:22:13.590025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:22:13.590025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:22:13.590025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:22:13.590025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:22:13.590025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:22:13.590025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:22:13.590025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:22:13.590025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:22:13.590025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:22:13.590025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:22:13.590025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:22:13.590025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:22:13.590460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:22:13.590460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:22:13.590460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:13.590460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:22:13.590460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:13.590460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:22:13.590460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:13.590460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:22:13.590460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:13.590460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:22:13.590460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:13.590460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:22:13.590460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:22:13.590460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:13.590460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:22:13.590460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:13.590460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:22:13.590460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:22:13.590460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:13.591052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:22:13.591052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:13.591052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:22:18.397245 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:22:18.398419 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:22:18.398419 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:22:18.398969 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:22:18.436855 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:22:18.436855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:18.436855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:22:18.436855 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:18.436855 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:22:18.436855 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:22:18.436855 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:18.436855 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:22:18.436855 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:18.436855 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:22:18.437731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:18.437731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:22:18.437731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:22:18.437731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:22:18.437731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:22:18.437731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:22:18.437731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:22:18.437731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:18.437731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:22:18.437731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:18.437731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:22:18.437731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:18.438656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:22:18.438656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:18.438656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:22:18.438656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:18.438656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:22:18.438656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:22:18.438656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:18.438656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:22:18.438656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:18.438656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:22:18.438656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:22:18.438656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:18.438656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:22:18.439078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:18.439078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:22:18.439078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:18.439078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:22:18.439078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:22:18.439078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:22:18.439078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:22:18.439078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:22:18.439078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:22:18.439078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:22:18.439078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:22:18.439078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:22:18.439078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:22:18.439078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:22:18.439078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:22:18.439689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:22:18.439689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:22:18.439689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:22:18.439689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:22:18.439689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:22:18.439689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:18.439689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:22:18.439689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:18.439689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:22:18.439689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:18.439689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:22:18.439689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:18.439689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:22:18.439689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:18.439689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:22:18.439689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:22:18.439689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:18.439689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:22:18.439689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:18.440237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:22:18.440237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:22:18.440237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:18.440237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:22:18.440237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:18.440237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:22:18.561324 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:22:18.561989 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:22:18.562087 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:22:18.564126 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:22:18.564126 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:22:18.564126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:18.564562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:22:18.564562 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:18.564562 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:22:18.564562 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:22:18.564562 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:18.564562 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:22:18.564562 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:18.564562 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:22:18.565430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:18.565430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:22:18.565430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:22:18.565430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:22:18.565430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:22:18.565430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:22:18.565430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:22:18.565430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:18.565430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:22:18.565430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:18.565430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:22:18.565430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:18.566124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:22:18.566124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:18.566124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:22:18.566124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:18.566124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:22:18.566124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:22:18.566124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:18.566124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:22:18.566124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:18.566124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:22:18.566124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:22:18.566124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:18.566124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:22:18.567476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:18.567476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:22:18.567476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:18.567476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:22:18.567476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:22:18.567476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:22:18.567476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:22:18.567476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:22:18.567476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:22:18.567476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:22:18.567476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:22:18.567476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:22:18.567476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:22:18.567476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:22:18.567476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:22:18.568357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:22:18.568357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:22:18.568357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:22:18.568357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:22:18.568357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:22:18.568357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:18.568357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:22:18.568357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:18.568357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:22:18.568357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:18.568357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:22:18.568357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:18.568357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:22:18.568357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:18.568357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:22:18.568357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:22:18.568357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:18.568357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:22:18.568357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:18.569179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:22:18.569179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:22:18.569179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:18.569179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:22:18.569179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:18.569179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:22:18.569179 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:22:18.569375 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:22:18.569375 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:22:18.569855 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:22:18.570294 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:22:18.571300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:18.571467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:22:18.571467 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:18.571467 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:22:18.571467 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:22:18.571467 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:18.571467 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:22:18.571467 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:18.571467 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:22:18.572270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:18.572270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:22:18.572270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:22:18.572270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:22:18.572270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:18.572270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:22:18.572270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:18.572270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:22:18.572270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:18.572270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:22:18.572270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:18.572270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:22:18.572270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:18.572270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:22:18.572270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:22:18.572917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:18.572917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:22:18.572917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:18.572917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:22:18.572917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:22:18.572917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:18.572917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:22:18.572917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:18.572917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:22:18.573432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:18.573432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:22:18.573432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:22:18.573432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:22:18.573432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:22:18.573432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:22:18.573432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:22:18.573432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:22:18.573432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:22:18.573432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:22:18.573432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:18.573432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:22:18.574304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:18.574304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:22:18.574304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:18.574304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:22:18.574304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:18.574304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:22:18.574304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:18.574304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:22:18.574304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:22:18.574304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:18.574304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:22:18.574304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:18.574304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:22:18.574304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:22:18.575102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:18.575102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:22:18.575102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:18.575102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:22:18.575102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:18.575102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:22:18.575102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:22:18.575102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:22:18.575102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:22:18.575102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:22:18.575102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:22:18.575102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:22:18.575102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:22:18.575102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:22:18.575551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:22:18.575551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:22:18.575551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:22:18.575551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:22:18.575551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:22:18.575551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:22:18.575551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:22:18.575551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:22:18.575551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:22:18.575551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:22:18.575551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:22:18.575551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:18.575551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:22:18.575551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:18.575551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:22:18.575551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:18.575551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:22:18.575551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:18.575551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:22:18.575926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:18.575926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:22:18.575926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:22:18.575926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:18.575926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:22:18.575926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:18.575926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:22:18.575926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:22:18.575926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:18.575926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:22:18.575926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:18.575926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:22:18.577404 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:22:18.577669 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:22:18.577800 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:22:18.577927 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:22:18.578225 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:22:18.578791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:18.578999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:22:18.578999 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:18.578999 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:22:18.578999 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:22:18.578999 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:18.578999 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:22:18.578999 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:18.578999 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:22:18.579682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:18.579682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:22:18.579682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:22:18.579682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:22:18.579682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:18.579682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:22:18.579682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:18.579682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:22:18.579682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:18.579682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:22:18.579682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:18.579682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:22:18.579682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:18.579682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:22:18.579682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:22:18.580111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:18.580111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:22:18.580111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:18.580111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:22:18.580111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:22:18.580111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:18.580111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:22:18.580111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:18.580111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:22:18.580603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:18.580603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:22:18.580603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:22:18.580603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:22:18.580603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:22:18.580603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:22:18.580603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:22:18.580603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:22:18.580603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:22:18.580603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:18.580603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:22:18.581261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:18.581261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:22:18.581261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:18.581261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:22:18.581261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:18.581261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:22:18.581261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:18.581261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:22:18.581261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:22:18.581261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:18.581261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:22:18.581261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:18.581261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:22:18.581261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:22:18.581643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:18.581643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:22:18.581643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:18.581643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:22:18.581643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:18.581643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:22:18.581643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:22:18.581643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:22:18.581643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:22:18.581643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:22:18.581643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:22:18.581643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:22:18.581643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:22:18.581643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:22:18.582014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:22:18.582014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:22:18.582014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:22:18.582014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:22:18.582014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:22:18.582014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:22:18.582014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:22:18.582014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:22:18.582014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:22:18.582014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:22:18.582014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:18.582014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:22:18.582014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:18.582014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:22:18.582014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:18.582014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:22:18.582014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:18.582014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:22:18.582014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:18.582531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:22:18.582531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:22:18.582531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:18.582531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:22:18.582531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:18.582531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:22:18.582531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:22:18.582531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:18.582531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:22:18.582531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:18.582531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:22:18.585963 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:22:18.586666 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:22:18.586725 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:22:18.586877 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:22:18.588022 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:22:18.588022 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:22:18.588022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:18.588140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:22:18.588140 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:18.588140 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:22:18.588140 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:22:18.588140 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:18.588140 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:22:18.588140 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:18.588140 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:22:18.588818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:18.588818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:22:18.588818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:22:18.588818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:22:18.588818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:22:18.588818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:22:18.588818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:22:18.588818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:18.588818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:22:18.588818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:18.588818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:22:18.588818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:18.589165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:22:18.589165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:18.589165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:22:18.589165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:18.589165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:22:18.589165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:22:18.589165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:18.589165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:22:18.589165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:18.589165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:22:18.589165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:22:18.589165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:18.589165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:22:18.589493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:18.589493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:22:18.589756 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:18.589756 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:22:18.589756 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:22:18.589756 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:22:18.589756 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:22:18.589756 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:22:18.589756 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:22:18.589756 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:22:18.589756 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:22:18.589756 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:22:18.589756 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:22:18.589756 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:22:18.589756 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:22:18.589756 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:22:18.589756 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:22:18.589756 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:22:18.590200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:22:18.590200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:22:18.590200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:18.590200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:22:18.590200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:18.590200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:22:18.590200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:18.590200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:22:18.590200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:18.590200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:22:18.590200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:18.590200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:22:18.590200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:22:18.590200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:18.590200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:22:18.590200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:18.590200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:22:18.590200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:22:18.590200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:18.590675 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:22:18.590675 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:18.590675 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:22:23.401021 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:22:23.401972 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:22:23.401972 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:22:23.402322 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:22:23.437374 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:22:23.437781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:23.438295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:22:23.438295 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:23.438295 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:22:23.438295 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:22:23.438295 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:23.438295 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:22:23.438295 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:23.438295 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:22:23.439115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:23.439115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:22:23.439115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:22:23.439115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:22:23.439115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:22:23.439115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:22:23.439115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:22:23.439115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:23.439115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:22:23.439115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:23.439115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:22:23.439115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:23.439807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:22:23.439807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:23.439807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:22:23.439807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:23.439807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:22:23.439807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:22:23.439807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:23.439807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:22:23.439807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:23.439807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:22:23.439807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:22:23.439807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:23.439807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:22:23.440428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:23.440428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:22:23.440428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:23.440428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:22:23.440428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:22:23.440428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:22:23.440428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:22:23.440428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:22:23.440428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:22:23.440428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:22:23.440428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:22:23.440428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:22:23.440428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:22:23.440428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:22:23.440428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:22:23.441302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:22:23.441302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:22:23.441302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:22:23.441302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:22:23.441302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:22:23.441302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:23.441302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:22:23.441302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:23.441302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:22:23.441302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:23.441302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:22:23.441302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:23.441302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:22:23.441302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:23.441302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:22:23.441302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:22:23.441302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:23.441302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:22:23.441302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:23.441999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:22:23.441999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:22:23.441999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:23.441999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:22:23.441999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:23.441999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:22:23.562945 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:22:23.562945 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:22:23.562945 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:22:23.563615 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:22:23.563866 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:22:23.564035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:23.564700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:22:23.564700 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:23.564700 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:22:23.564700 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:22:23.564700 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:23.564700 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:22:23.564700 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:23.564700 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:22:23.566338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:23.566338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:22:23.566338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:22:23.566338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:22:23.566338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:22:23.566338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:22:23.566338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:22:23.566338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:23.566338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:22:23.566338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:23.566338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:22:23.566338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:23.566729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:22:23.566729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:23.566729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:22:23.566729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:23.566729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:22:23.566729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:22:23.566729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:23.566729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:22:23.566729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:23.566729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:22:23.566729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:22:23.566729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:23.566729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:22:23.567204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:23.567204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:22:23.568150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:23.568150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:22:23.568150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:22:23.568150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:22:23.568150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:22:23.568150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:22:23.568150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:22:23.568150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:22:23.568150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:22:23.568150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:22:23.568150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:22:23.568150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:22:23.568150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:22:23.568150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:22:23.568150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:22:23.568150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:22:23.568565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:22:23.568565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:22:23.568565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:23.568565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:22:23.568565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:23.568565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:22:23.568565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:23.568565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:22:23.568565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:23.568565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:22:23.568565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:23.568565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:22:23.568565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:22:23.568565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:23.568565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:22:23.568565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:23.568565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:22:23.568565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:22:23.568565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:23.569035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:22:23.569035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:23.569035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:22:23.569417 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:22:23.569862 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:22:23.570053 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:22:23.570250 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:22:23.570985 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:22:23.571305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:23.572078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:22:23.572078 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:23.572078 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:22:23.572078 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:22:23.572078 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:23.572078 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:22:23.572078 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:23.572078 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:22:23.572981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:23.572981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:22:23.572981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:22:23.572981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:22:23.572981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:23.572981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:22:23.572981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:23.572981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:22:23.572981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:23.572981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:22:23.572981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:23.572981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:22:23.572981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:23.572981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:22:23.572981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:22:23.574148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:23.574148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:22:23.574148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:23.574148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:22:23.574148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:22:23.574148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:23.574148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:22:23.574148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:23.574148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:22:23.574148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:23.574148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:22:23.574148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:22:23.574148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:22:23.574148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:22:23.574904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:22:23.574904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:22:23.574904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:22:23.574904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:22:23.574904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:22:23.574904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:23.574904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:22:23.574904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:23.574904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:22:23.574904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:23.574904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:22:23.574904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:23.574904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:22:23.575708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:23.575708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:22:23.575708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:22:23.575708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:23.575708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:22:23.575708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:23.575708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:22:23.575708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:22:23.575708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:23.575708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:22:23.575708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:23.575708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:22:23.576438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:23.576438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:22:23.576438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:22:23.576438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:22:23.576438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:22:23.576438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:22:23.576438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:22:23.576438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:22:23.576438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:22:23.576438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:22:23.576438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:22:23.576438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:22:23.576438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:22:23.576438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:22:23.576438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:22:23.576438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:22:23.577491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:22:23.577491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:22:23.577491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:22:23.577491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:22:23.577491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:22:23.577491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:23.577491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:22:23.577491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:23.577491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:22:23.577491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:23.577491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:22:23.577491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:23.577491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:22:23.577491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:23.577491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:22:23.577491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:22:23.577491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:23.577491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:22:23.578396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:23.578396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:22:23.578396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:22:23.578396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:23.578396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:22:23.578396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:23.578396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:22:23.578396 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:22:23.578396 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:22:23.578396 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:22:23.578908 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:22:23.578969 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:22:23.579346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:23.580100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:22:23.580100 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:23.580100 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:22:23.580100 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:22:23.580100 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:23.580100 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:22:23.580100 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:23.580100 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:22:23.581017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:23.581017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:22:23.581017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:22:23.581017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:22:23.581017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:23.581017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:22:23.581017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:23.581017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:22:23.581017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:23.581017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:22:23.581017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:23.581017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:22:23.581017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:23.581017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:22:23.581017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:22:23.581738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:23.581738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:22:23.581738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:23.581738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:22:23.581738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:22:23.581738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:23.581738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:22:23.581738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:23.581738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:22:23.582165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:23.582165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:22:23.582165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:22:23.582165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:22:23.582165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:22:23.582165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:22:23.582165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:22:23.582165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:22:23.582165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:22:23.582165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:23.582165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:22:23.582712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:23.582712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:22:23.582712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:23.582712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:22:23.582712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:23.582712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:22:23.582712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:23.582712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:22:23.582712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:22:23.582712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:23.582712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:22:23.582712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:23.582712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:22:23.582712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:22:23.583347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:23.583347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:22:23.583347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:23.583347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:22:23.583347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:23.583347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:22:23.583347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:22:23.583347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:22:23.583347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:22:23.583347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:22:23.583347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:22:23.583347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:22:23.583347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:22:23.583347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:22:23.584668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:22:23.584668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:22:23.584668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:22:23.584668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:22:23.584668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:22:23.584668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:22:23.584668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:22:23.584668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:22:23.584668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:22:23.584668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:22:23.584668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:23.584668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:22:23.584668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:23.584668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:22:23.584668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:23.584668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:22:23.584668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:23.584668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:22:23.584668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:23.585653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:22:23.585653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:22:23.585653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:23.585653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:22:23.585653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:23.585653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:22:23.585653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:22:23.585653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:23.585653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:22:23.585653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:23.585653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:22:23.586556 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:22:23.586855 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:22:23.586975 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:22:23.587146 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:22:23.587633 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:22:23.588054 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:22:23.588298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:23.589188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:22:23.589188 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:23.589188 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:22:23.589188 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:22:23.589188 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:23.589188 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:22:23.589188 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:23.589188 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:22:23.590134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:23.590134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:22:23.590134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:22:23.590134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:22:23.590134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:22:23.590134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:22:23.590134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:22:23.590134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:23.590134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:22:23.590134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:23.590134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:22:23.590134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:23.590361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:22:23.590361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:23.590361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:22:23.590361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:23.590361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:22:23.590361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:22:23.590361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:23.590361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:22:23.590361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:23.590361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:22:23.590361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:22:23.590361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:23.590361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:22:23.590588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:23.590588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:22:23.591527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:23.591527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:22:23.591527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:22:23.591527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:22:23.591527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:22:23.591527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:22:23.591527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:22:23.591527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:22:23.591527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:22:23.591527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:22:23.591527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:22:23.591527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:22:23.591527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:22:23.591527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:22:23.591527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:22:23.591527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:22:23.591834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:22:23.591834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:22:23.591834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:23.591834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:22:23.591834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:23.591834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:22:23.591834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:23.591834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:22:23.591834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:23.591834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:22:23.591834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:23.591834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:22:23.591834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:22:23.591834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:23.591834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:22:23.591834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:23.591834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:22:23.591834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:22:23.591834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:23.592331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:22:23.592331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:23.592331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:22:28.401693 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:22:28.402465 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:22:28.402465 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:22:28.403329 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:22:28.448189 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:22:28.448490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:28.450899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:22:28.450899 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:28.450899 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:22:28.450899 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:22:28.450899 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:28.450899 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:22:28.450899 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:28.450899 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:22:28.450899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.047 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:28.450899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:22:28.450899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:22:28.450899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:22:28.450899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:22:28.450899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:22:28.450899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:22:28.451346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:28.451346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:22:28.451346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:28.451346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:22:28.451346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:28.451346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:22:28.451346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:28.451346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:22:28.451346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:28.451346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:22:28.451346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:22:28.451346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:28.451346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:22:28.451346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:28.451775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:22:28.451775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:22:28.451775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:28.451775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:22:28.451775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:28.451775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:22:28.451775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:28.451775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:22:28.451775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:22:28.451775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:22:28.451775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:22:28.451775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:22:28.451775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:22:28.451775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:22:28.452119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:22:28.452119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:22:28.452119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:22:28.452119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:22:28.452119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:22:28.452119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:22:28.452119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:22:28.452119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:22:28.452119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:22:28.452119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:22:28.452119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:28.452119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:22:28.452119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:28.452119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:22:28.452119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:28.452119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:22:28.452119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:28.452119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:22:28.452119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:28.452119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:22:28.452119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:22:28.452567 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:28.452567 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:22:28.452567 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:28.452567 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:22:28.452567 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:22:28.452567 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:28.452567 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:22:28.452567 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:28.452567 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:22:28.562618 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:22:28.563474 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:22:28.563712 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:22:28.564868 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:22:28.564868 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:22:28.564868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:28.565224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:22:28.565224 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:28.565224 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:22:28.565224 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:22:28.565224 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:28.565224 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:22:28.565224 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:28.565224 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:22:28.566250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:28.566250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:22:28.566250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:22:28.566250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:22:28.566250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:22:28.566250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:22:28.566250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:22:28.566250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:28.566250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:22:28.566250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:28.566250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:22:28.566250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:28.566728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:22:28.566728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:28.566728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:22:28.566728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:28.566728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:22:28.566728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:22:28.566728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:28.566728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:22:28.566728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:28.566728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:22:28.566728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:22:28.566728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:28.566728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:22:28.567186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:28.567186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:22:28.567534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:28.567534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:22:28.567534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:22:28.567534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:22:28.567534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:22:28.567534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:22:28.567534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:22:28.567534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:22:28.567534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:22:28.567534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:22:28.567534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:22:28.567534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:22:28.567534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:22:28.567534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:22:28.567534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:22:28.567534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:22:28.568063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:22:28.568063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:22:28.568063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:28.568063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:22:28.568063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:28.568063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:22:28.568063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:28.568063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:22:28.568063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:28.568063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:22:28.568063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:28.568063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:22:28.568063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:22:28.568063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:28.568063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:22:28.568063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:28.568063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:22:28.568063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:22:28.568063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:28.568580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:22:28.568580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:28.568580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:22:28.569841 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:22:28.571817 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:22:28.572014 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:22:28.572091 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:22:28.572428 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:22:28.572882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:28.573049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:22:28.573049 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:28.573049 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:22:28.573049 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:22:28.573049 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:28.573049 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:22:28.573049 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:28.573049 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:22:28.573854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:28.573854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:22:28.573854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:22:28.573854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:22:28.573854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:28.573854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:22:28.573854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:28.573854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:22:28.573854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:28.573854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:22:28.573854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:28.573854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:22:28.573854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:28.573854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:22:28.573854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:22:28.574352 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:28.574352 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:22:28.574352 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:28.574352 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:22:28.574352 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:22:28.574352 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:28.574352 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:22:28.574352 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:28.574352 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:22:28.575020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:28.575020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:22:28.575020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:22:28.575020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:22:28.575020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:22:28.575020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:22:28.575020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:22:28.575020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:22:28.575020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:22:28.575020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:22:28.575020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:28.575020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:22:28.575498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:28.575498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:22:28.575498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:28.575498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:22:28.575498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:28.575498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:22:28.575498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:28.575498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:22:28.575498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:22:28.575498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:28.575498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:22:28.575498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:28.575498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:22:28.575498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:22:28.575912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:28.575912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:22:28.575912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:28.575912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:22:28.576027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:28.576027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:22:28.576027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:22:28.576027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:22:28.576027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:22:28.576027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:22:28.576027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:22:28.576027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:22:28.576027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:22:28.576027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:22:28.576027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:22:28.576027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:22:28.576027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:22:28.576027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:22:28.576027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:22:28.576027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:22:28.576490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:22:28.576490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:22:28.576490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:22:28.576490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:22:28.576490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:22:28.576490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:28.576490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:22:28.576490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:28.576490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:22:28.576490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:28.576490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:22:28.576490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:28.576490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:22:28.576490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:28.576490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:22:28.576490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:22:28.576490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:28.576490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:22:28.577024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:28.577024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:22:28.577024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:22:28.577024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:28.577024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:22:28.577024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:28.577024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:22:28.578897 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:22:28.578897 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:22:28.578897 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:22:28.579714 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:22:28.579714 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:22:28.579714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:28.580203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:22:28.580203 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:28.580203 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:22:28.580203 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:22:28.580203 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:28.580203 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:22:28.580203 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:28.580203 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:22:28.582082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:28.582082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:22:28.582082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:22:28.582082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:22:28.582082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:28.582082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:22:28.582082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:28.582082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:22:28.582082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:28.582082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:22:28.582082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:28.582082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:22:28.582082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:28.582082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:22:28.582082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:22:28.582403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:28.582403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:22:28.582403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:28.582403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:22:28.582403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:22:28.582403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:28.582403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:22:28.582403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:28.582403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:22:28.582403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:28.582403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:22:28.582403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:22:28.582403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:22:28.582403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:22:28.582710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:22:28.582710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:22:28.582710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:22:28.582710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:22:28.582710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:28.582710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:22:28.582710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:28.582710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:22:28.582710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:28.582710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:22:28.582710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:28.582710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:22:28.582710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:28.583783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:22:28.583783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:22:28.583783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:28.583783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:22:28.583783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:28.583783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:22:28.583783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:22:28.583783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:28.583783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:22:28.583783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:28.583783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:22:28.584312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:28.584312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:22:28.584312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:22:28.584312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:22:28.584312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:22:28.584312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:22:28.584312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:22:28.584312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:22:28.584312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:22:28.584312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:22:28.584312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:22:28.584312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:22:28.584312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:22:28.584312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:22:28.584312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:22:28.584312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:22:28.584312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:22:28.585174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:22:28.585174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:22:28.585174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:22:28.585174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:28.585174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:22:28.585174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:28.585174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:22:28.585174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:28.585174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:22:28.585174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:28.585174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:22:28.585174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:28.585174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:22:28.585174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:22:28.585174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:28.585174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:22:28.585174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:28.585174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:22:28.585174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:22:28.586013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:28.586013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:22:28.586013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:28.586013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:22:28.587140 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:22:28.587487 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:22:28.587615 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:22:28.587847 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:22:28.588340 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:22:28.588500 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:22:28.588629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:28.589196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:22:28.589196 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:28.589196 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:22:28.589196 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:22:28.589196 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:28.589196 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:22:28.589196 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:28.589196 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:22:28.590338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:28.590338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:22:28.590338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:22:28.590338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:22:28.590338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:22:28.590338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:22:28.590338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:22:28.590338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:28.590338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:22:28.590338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:28.590338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:22:28.590338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:28.590789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:22:28.590789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:28.590789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:22:28.590789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:28.590789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:22:28.590789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:22:28.590789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:28.590789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:22:28.590789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:28.590789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:22:28.590789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:22:28.590789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:28.590789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:22:28.591219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:28.591219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:22:28.592173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:28.592173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:22:28.592173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:22:28.592173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:22:28.592173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:22:28.592173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:22:28.592173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:22:28.592173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:22:28.592173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:22:28.592173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:22:28.592173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:22:28.592173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:22:28.592173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:22:28.592173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:22:28.592173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:22:28.592173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:22:28.593931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:22:28.593931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:22:28.593931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:28.593931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:22:28.593931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:28.593931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:22:28.593931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:28.593931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:22:28.593931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:28.593931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:22:28.593931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:28.593931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:22:28.593931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:22:28.593931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:28.593931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:22:28.593931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:28.593931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:22:28.593931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:22:28.593931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:28.595041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:22:28.595041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:28.595041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:22:33.402824 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:22:33.403421 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:22:33.403421 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:22:33.404897 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:22:33.445308 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:22:33.445785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:33.446503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:22:33.446503 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:33.446503 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:22:33.446503 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:22:33.446503 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:33.446503 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:22:33.446503 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:33.446503 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:22:33.447537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.043 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:33.447537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:22:33.447537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:22:33.447537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:22:33.447537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:22:33.447537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:22:33.447537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:22:33.447537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:33.447537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:22:33.447537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:33.447537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:22:33.447537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:33.448839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:22:33.448839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:33.448839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:22:33.448839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:33.448839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:22:33.448839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:22:33.448839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:33.448839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:22:33.448839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:33.448839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:22:33.448839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:22:33.448839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:33.448839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:22:33.449244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:33.449244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:22:33.449244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:33.449244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:22:33.449244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:22:33.449244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:22:33.449244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:22:33.449244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:22:33.449244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:22:33.449244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:22:33.449244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:22:33.449244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:22:33.449244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:22:33.449244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:22:33.449244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:22:33.449733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:22:33.449733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:22:33.449733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:22:33.449733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:22:33.449733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:22:33.449733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:33.449733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:22:33.449733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:33.449733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:22:33.449733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:33.449733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:22:33.449733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:33.449733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:22:33.449733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:33.449733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:22:33.449733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:22:33.449733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:33.449733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:22:33.449733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:33.450499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:22:33.450499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:22:33.450499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:33.450499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:22:33.450499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:33.450499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:22:33.563294 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:22:33.565417 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:22:33.565417 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:22:33.565417 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:22:33.565417 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:22:33.565417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:33.565587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:22:33.565587 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:33.565587 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:22:33.565587 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:22:33.565587 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:33.565587 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:22:33.565587 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:33.565587 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:22:33.566662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:33.566662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:22:33.566662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:22:33.566662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:22:33.566662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:22:33.566662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:22:33.566662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:22:33.566662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:33.566662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:22:33.566662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:33.566662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:22:33.566662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:33.566955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:22:33.566955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:33.566955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:22:33.566955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:33.566955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:22:33.566955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:22:33.566955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:33.566955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:22:33.566955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:33.566955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:22:33.566955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:22:33.566955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:33.566955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:22:33.567179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:33.567179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:22:33.568143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:33.568143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:22:33.568143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:22:33.568143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:22:33.568143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:22:33.568143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:22:33.568143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:22:33.568143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:22:33.568143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:22:33.568143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:22:33.568143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:22:33.568143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:22:33.568143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:22:33.568143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:22:33.568143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:22:33.568143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:22:33.568477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:22:33.568477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:22:33.568477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:33.568477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:22:33.568477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:33.568477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:22:33.568477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:33.568477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:22:33.568477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:33.568477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:22:33.568477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:33.568477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:22:33.568477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:22:33.568477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:33.568477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:22:33.568477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:33.568477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:22:33.568477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:22:33.568477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:33.568826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:22:33.568826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:33.568826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:22:33.571814 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:22:33.572200 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:22:33.572324 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:22:33.572495 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:22:33.576789 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:22:33.576789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:33.576789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:22:33.576789 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:33.576789 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:22:33.576789 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:22:33.576789 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:33.576789 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:22:33.576789 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:33.576789 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:22:33.577453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:33.577453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:22:33.577453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:22:33.577453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:22:33.577453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:33.577453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:22:33.577453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:33.577453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:22:33.577453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:33.577453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:22:33.577453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:33.577453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:22:33.577453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:33.577453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:22:33.577453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:22:33.578024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:33.578024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:22:33.578024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:33.578024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:22:33.578024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:22:33.578024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:33.578024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:22:33.578024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:33.578024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:22:33.578405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:33.578405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:22:33.578405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:22:33.578405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:22:33.578405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:22:33.578405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:22:33.578405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:22:33.578405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:22:33.578405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:22:33.578405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:22:33.578405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:33.578405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:22:33.578802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:33.578802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:22:33.578802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:33.578802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:22:33.578802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:33.578802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:22:33.578802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:33.578802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:22:33.578802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:22:33.578802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:33.578802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:22:33.578802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:33.578802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:22:33.578802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:22:33.579136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:33.579136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:22:33.579136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:33.579136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:22:33.579625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:33.579625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:22:33.579625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:22:33.579625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:22:33.579625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:22:33.579625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:22:33.579625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:22:33.579625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:22:33.579625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:22:33.579625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:22:33.579625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:22:33.579625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:22:33.579625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:22:33.579625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:22:33.579625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:22:33.579625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:22:33.580064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:22:33.580064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:22:33.580064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:22:33.580064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:22:33.580064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:22:33.580064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:33.580064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:22:33.580064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:33.580064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:22:33.580064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:33.580064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:22:33.580064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:33.580064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:22:33.580064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:33.580064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:22:33.580064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:22:33.580064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:33.580064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:22:33.580538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:33.580538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:22:33.580538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:22:33.580538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:33.580538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:22:33.580538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:33.580538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:22:33.580538 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:22:33.580538 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:22:33.580538 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:22:33.580737 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:22:33.580850 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:22:33.581147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:33.581846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:22:33.581846 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:33.581846 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:22:33.581846 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:22:33.581846 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:33.581846 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:22:33.581846 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:33.581846 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:22:33.582637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:33.582637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:22:33.582637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:22:33.582637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:22:33.582637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:33.582637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:22:33.582637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:33.582637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:22:33.582637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:33.582637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:22:33.582637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:33.582637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:22:33.582637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:33.582637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:22:33.582637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:22:33.583085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:33.583085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:22:33.583085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:33.583085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:22:33.583085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:22:33.583085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:33.583085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:22:33.583085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:33.583085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:22:33.583559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:33.583559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:22:33.583559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:22:33.583559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:22:33.583559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:22:33.583559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:22:33.583559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:22:33.583559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:22:33.583559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:22:33.583559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:33.583559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:22:33.583926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:33.583926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:22:33.583926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:33.583926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:22:33.583926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:33.583926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:22:33.583926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:33.583926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:22:33.583926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:22:33.583926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:33.583926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:22:33.583926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:33.583926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:22:33.583926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:22:33.584268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:33.584268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:22:33.584268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:33.584268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:22:33.584670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:33.584670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:22:33.584670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:22:33.584670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:22:33.584670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:22:33.584670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:22:33.584670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:22:33.584670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:22:33.584670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:22:33.584670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:22:33.584670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:22:33.584670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:22:33.584670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:22:33.584670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:22:33.584670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:22:33.584670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:22:33.584670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:22:33.585065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:22:33.585065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:22:33.585065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:22:33.585065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:33.585065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:22:33.585065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:33.585065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:22:33.585065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:33.585065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:22:33.585065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:33.585065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:22:33.585065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:33.585065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:22:33.585065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:22:33.585065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:33.585065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:22:33.585065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:33.585065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:22:33.585065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:22:33.585416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:33.585416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:22:33.585416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:33.585416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:22:33.588775 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:22:33.588775 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:22:33.588775 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:22:33.588775 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:22:33.588775 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:22:33.588775 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:22:33.588969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:33.589429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:22:33.589429 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:33.589429 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:22:33.589429 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:22:33.589429 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:33.589429 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:22:33.589429 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:33.589429 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:22:33.590344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:33.590344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:22:33.590344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:22:33.590344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:22:33.590344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:22:33.590344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:22:33.590344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:22:33.590344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:33.590344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:22:33.590344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:33.590344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:22:33.590344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:33.590725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:22:33.590725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:33.590725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:22:33.590725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:33.590725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:22:33.590725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:22:33.590725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:33.590725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:22:33.590725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:33.590725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:22:33.590725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:22:33.590725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:33.590725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:22:33.591117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:33.591117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:22:33.591372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:33.591372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:22:33.591372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:22:33.591372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:22:33.591372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:22:33.591372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:22:33.591372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:22:33.591372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:22:33.591372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:22:33.591372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:22:33.591372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:22:33.591372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:22:33.591372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:22:33.591372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:22:33.591372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:22:33.591372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:22:33.591709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:22:33.591709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:22:33.591709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:33.591709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:22:33.591709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:33.591709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:22:33.591709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:33.591709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:22:33.591709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:33.591709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:22:33.591709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:33.591709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:22:33.591709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:22:33.591709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:33.591709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:22:33.591709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:33.591709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:22:33.591709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:22:33.591709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:33.592068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:22:33.592068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:33.592068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:22:38.402916 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:22:38.405068 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:22:38.405182 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:22:38.406022 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:22:38.449568 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:22:38.449894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:38.450628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:22:38.450628 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:38.450628 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:22:38.450628 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:22:38.450628 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:38.450628 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:22:38.450628 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:38.450628 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:22:38.451698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.045 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:38.451698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:22:38.451698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:22:38.451698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:22:38.451698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:22:38.451698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:22:38.451698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:22:38.451698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:38.451698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:22:38.451698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:38.451698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:22:38.451698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:38.452177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:22:38.452177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:38.452177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:22:38.452177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:38.452177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:22:38.452177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:22:38.452177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:38.452177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:22:38.452177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:38.452177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:22:38.452177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:22:38.452177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:38.452177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:22:38.452560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:38.452560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:22:38.452797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:38.452797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:22:38.452797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:22:38.452797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:22:38.452797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:22:38.452797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:22:38.452797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:22:38.452797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:22:38.452797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:22:38.452797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:22:38.452797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:22:38.452797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:22:38.452797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:22:38.452797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:22:38.452797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:22:38.452797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:22:38.453318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:22:38.453318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:22:38.453318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:38.453318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:22:38.453318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:38.453318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:22:38.453318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:38.453318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:22:38.453318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:38.453318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:22:38.453318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:38.453318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:22:38.453318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:22:38.453318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:38.453318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:22:38.453318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:38.453318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:22:38.453318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:22:38.453318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:38.454168 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:22:38.454168 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:38.454168 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:22:38.564839 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:22:38.565181 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:22:38.565181 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:22:38.566076 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:22:38.566310 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:22:38.566554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:38.567599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:22:38.567599 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:38.567599 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:22:38.567599 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:22:38.567599 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:38.567599 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:22:38.567599 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:38.567599 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:22:38.569249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:38.569249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:22:38.569249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:22:38.569249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:22:38.569249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:22:38.569249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:22:38.569249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:22:38.569249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:38.569249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:22:38.569249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:38.569249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:22:38.569249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:38.569697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:22:38.569697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:38.569697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:22:38.569697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:38.569697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:22:38.569697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:22:38.569697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:38.569697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:22:38.569697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:38.569697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:22:38.569697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:22:38.569697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:38.569697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:22:38.570040 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:38.570040 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:22:38.572231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:38.572231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:22:38.572231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:22:38.572231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:22:38.572231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:22:38.572231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:22:38.572231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:22:38.572231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:22:38.572231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:22:38.572231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:22:38.572231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:22:38.572231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:22:38.572231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:22:38.572231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:22:38.572231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:22:38.572231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:22:38.572787 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:22:38.572787 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:22:38.572787 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:38.572787 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:22:38.572787 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:38.572787 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:22:38.572787 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:38.572787 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:22:38.572787 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:38.572787 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:22:38.572787 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:38.572787 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:22:38.572787 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:22:38.572787 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:38.572787 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:22:38.572787 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:38.572787 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:22:38.572787 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:22:38.572787 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:38.573370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:22:38.573370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:38.573370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:22:38.573871 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:22:38.574797 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:22:38.575763 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:22:38.575763 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:22:38.576352 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:22:38.576352 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:38.576824 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:22:38.576824 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:38.576824 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:22:38.576824 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:22:38.576824 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:38.576824 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:22:38.576824 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:38.576824 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:22:38.577842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:38.577842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:22:38.577842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:22:38.577842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:22:38.577842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:38.577842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:22:38.577842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:38.577842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:22:38.577842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:38.577842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:22:38.577842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:38.577842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:22:38.577842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:38.577842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:22:38.577842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:22:38.578313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:38.578313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:22:38.578313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:38.578313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:22:38.578313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:22:38.578313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:38.578313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:22:38.578313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:38.578313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:22:38.578811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:38.578811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:22:38.578811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:22:38.578811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:22:38.578811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:22:38.578811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:22:38.578811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:22:38.578811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:22:38.578811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:22:38.578811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:22:38.578811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:38.578811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:22:38.579192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:38.579192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:22:38.579192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:38.579192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:22:38.579192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:38.579192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:22:38.579192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:38.579192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:22:38.579192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:22:38.579192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:38.579192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:22:38.579192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:38.579192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:22:38.579192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:22:38.579537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:38.579537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:22:38.579537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:38.579537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:22:38.581524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:38.581524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:22:38.581524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:22:38.581524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:22:38.581524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:22:38.581524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:22:38.581524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:22:38.581524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:22:38.581524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:22:38.581524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:22:38.581524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:22:38.581524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:22:38.581524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:22:38.581524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:22:38.581524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:22:38.581524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:22:38.582180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:22:38.582180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:22:38.582180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:22:38.582180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:22:38.582180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:22:38.582180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:38.582180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:22:38.582180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:38.582180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:22:38.582180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:38.582180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:22:38.582180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:38.582180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:22:38.582180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:38.582180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:22:38.582180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:22:38.582180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:38.582180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:22:38.582660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:38.582660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:22:38.582660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:22:38.582660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:38.582660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:22:38.582660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:38.582660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:22:38.582660 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:22:38.582660 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:22:38.582660 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:22:38.582987 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:22:38.582987 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:22:38.583292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:38.583639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:22:38.583639 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:38.583639 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:22:38.583639 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:22:38.583639 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:38.583639 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:22:38.583639 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:38.583639 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:22:38.584254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:38.584254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:22:38.584254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:22:38.584254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:22:38.584254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:38.584254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:22:38.584254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:38.584254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:22:38.584254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:38.584254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:22:38.584254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:38.584254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:22:38.584254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:38.584254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:22:38.584254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:22:38.584722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:38.584722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:22:38.584722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:38.584722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:22:38.584722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:22:38.584722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:38.584722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:22:38.584722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:38.584722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:22:38.585007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:38.585007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:22:38.585007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:22:38.585007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:22:38.585007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:22:38.585007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:22:38.585007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:22:38.585007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:22:38.585007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:22:38.585007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:38.585007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:22:38.585366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:38.585366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:22:38.585366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:38.585366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:22:38.585366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:38.585366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:22:38.585366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:38.585366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:22:38.585366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:22:38.585366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:38.585366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:22:38.585366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:38.585366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:22:38.585366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:22:38.586081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:38.586081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:22:38.586081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:38.586081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:22:38.586081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:38.586081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:22:38.586081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:22:38.586081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:22:38.586081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:22:38.586081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:22:38.586081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:22:38.586081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:22:38.586081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:22:38.586081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:22:38.586465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:22:38.586465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:22:38.586465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:22:38.586465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:22:38.586465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:22:38.586465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:22:38.586465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:22:38.586465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:22:38.586465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:22:38.586465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:22:38.586465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:38.586465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:22:38.586465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:38.586465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:22:38.586465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:38.586465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:22:38.586465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:38.586465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:22:38.586465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:38.586960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:22:38.586960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:22:38.586960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:38.586960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:22:38.586960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:38.586960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:22:38.586960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:22:38.586960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:38.586960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:22:38.586960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:38.586960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:22:38.588003 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:22:38.588331 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:22:38.588331 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:22:38.588466 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:22:38.589569 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:22:38.589926 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:22:38.590120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:38.590891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:22:38.590891 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:38.590891 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:22:38.590891 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:22:38.590891 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:38.590891 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:22:38.590891 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:38.590891 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:22:38.591544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:38.591544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:22:38.591544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:22:38.591544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:22:38.591544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:22:38.591544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:22:38.591544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:22:38.591544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:38.591544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:22:38.591544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:38.591544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:22:38.591544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:38.591929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:22:38.591929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:38.591929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:22:38.591929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:38.591929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:22:38.591929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:22:38.591929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:38.591929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:22:38.591929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:38.591929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:22:38.591929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:22:38.591929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:38.591929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:22:38.592726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:38.592726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:22:38.592726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:38.592726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:22:38.592726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:22:38.592726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:22:38.592726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:22:38.592726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:22:38.592726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:22:38.592726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:22:38.592726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:22:38.592726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:22:38.592726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:22:38.592726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:22:38.592726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:22:38.593381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:22:38.593381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:22:38.593381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:22:38.593381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:22:38.593381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:22:38.593381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:38.593381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:22:38.593381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:38.593381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:22:38.593381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:38.593381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:22:38.593381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:38.593381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:22:38.593381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:38.593381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:22:38.593381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:22:38.593381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:38.593381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:22:38.593381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:38.594015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:22:38.594015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:22:38.594015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:38.594015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:22:38.594015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:38.594015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:22:43.405095 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:22:43.405729 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:22:43.405729 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:22:43.406605 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:22:43.443061 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:22:43.443331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:43.444080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:22:43.444080 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:43.444080 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:22:43.444080 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:22:43.444080 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:43.444080 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:22:43.444080 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:43.444080 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:22:43.444906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:43.444906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:22:43.444906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:22:43.444906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:22:43.444906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:22:43.444906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:22:43.444906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:22:43.444906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:43.444906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:22:43.444906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:43.444906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:22:43.444906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:43.445260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:22:43.445260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:43.445260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:22:43.445260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:43.445260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:22:43.445260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:22:43.445260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:43.445260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:22:43.445260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:43.445260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:22:43.445260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:22:43.445260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:43.445260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:22:43.445582 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:43.445582 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:22:43.445775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:43.445775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:22:43.445775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:22:43.445775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:22:43.445775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:22:43.445775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:22:43.445775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:22:43.445775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:22:43.445775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:22:43.445775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:22:43.445775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:22:43.445775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:22:43.445775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:22:43.445775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:22:43.445775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:22:43.445775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:22:43.446279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:22:43.446279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:22:43.446279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:43.446279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:22:43.446279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:43.446279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:22:43.446279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:43.446279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:22:43.446279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:43.446279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:22:43.446279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:43.446279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:22:43.446279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:22:43.446279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:43.446279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:22:43.446279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:43.446279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:22:43.446279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:22:43.446279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:43.446674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:22:43.446674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:43.446674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:22:43.565346 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:22:43.565644 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:22:43.565644 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:22:43.567033 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:22:43.567199 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:22:43.567334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:43.568077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:22:43.568077 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:43.568077 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:22:43.568077 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:22:43.568077 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:43.568077 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:22:43.568077 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:43.568077 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:22:43.569418 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:43.569418 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:22:43.569418 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:22:43.569418 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:22:43.569418 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:22:43.569418 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:22:43.569418 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:22:43.569418 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:43.569418 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:22:43.569418 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:43.569418 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:22:43.569418 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:43.569839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:22:43.569839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:43.569839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:22:43.569839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:43.569839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:22:43.569839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:22:43.569839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:43.569839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:22:43.569839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:43.569839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:22:43.569839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:22:43.569839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:43.569839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:22:43.570208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:43.570208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:22:43.571212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:43.571212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:22:43.571212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:22:43.571212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:22:43.571212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:22:43.571212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:22:43.571212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:22:43.571212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:22:43.571212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:22:43.571212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:22:43.571212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:22:43.571212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:22:43.571212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:22:43.571212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:22:43.571212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:22:43.571212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:22:43.571669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:22:43.571669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:22:43.571669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:43.571669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:22:43.571669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:43.571669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:22:43.571669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:43.571669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:22:43.571669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:43.571669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:22:43.571669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:43.571669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:22:43.571669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:22:43.571669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:43.571669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:22:43.571669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:43.571669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:22:43.571669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:22:43.571669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:43.572228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:22:43.572228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:43.572228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:22:43.574686 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:22:43.575036 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:22:43.575193 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:22:43.575360 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:22:43.575828 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:22:43.575985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:43.576708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:22:43.576708 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:43.576708 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:22:43.576708 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:22:43.576708 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:43.576708 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:22:43.576708 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:43.576708 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:22:43.578095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:43.578095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:22:43.578095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:22:43.578095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:22:43.578095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:43.578095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:22:43.578095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:43.578095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:22:43.578095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:43.578095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:22:43.578095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:43.578095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:22:43.578095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:43.578095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:22:43.578095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:22:43.578459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:43.578459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:22:43.578459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:43.578459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:22:43.578459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:22:43.578459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:43.578459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:22:43.578459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:43.578459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:22:43.579333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:43.579333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:22:43.579333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:22:43.579333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:22:43.579333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:22:43.579333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:22:43.579333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:22:43.579333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:22:43.579333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:22:43.579333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:22:43.579333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:43.579333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:22:43.579617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:43.579617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:22:43.579617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:43.579617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:22:43.579617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:43.579617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:22:43.579617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:43.579617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:22:43.579617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:22:43.579617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:43.579617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:22:43.579617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:43.579617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:22:43.579617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:22:43.579916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:43.579916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:22:43.579916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:43.579916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:22:43.582138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:43.582138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:22:43.582138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:22:43.582138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:22:43.582138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:22:43.582138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:22:43.582138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:22:43.582138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:22:43.582138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:22:43.582138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:22:43.582138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:22:43.582138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:22:43.582138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:22:43.582138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:22:43.582138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:22:43.582138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:22:43.582591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:22:43.582591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:22:43.582591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:22:43.582591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:22:43.582591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:22:43.582591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:43.582591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:22:43.582591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:43.582591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:22:43.582591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:43.582591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:22:43.582591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:43.582591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:22:43.582591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:43.582591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:22:43.582591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:22:43.582591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:43.582591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:22:43.583133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:43.583133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:22:43.583133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:22:43.583133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:43.583133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:22:43.583133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:43.583133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:22:43.583133 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:22:43.583504 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:22:43.583758 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:22:43.584073 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:22:43.584582 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:22:43.584860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:43.585495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:22:43.585495 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:43.585495 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:22:43.585495 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:22:43.585495 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:43.585495 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:22:43.585495 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:43.585495 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:22:43.586376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:43.586376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:22:43.586376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:22:43.586376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:22:43.586376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:43.586376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:22:43.586376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:43.586376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:22:43.586376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:43.586376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:22:43.586376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:43.586376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:22:43.586376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:43.586376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:22:43.586376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:22:43.586767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:43.586767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:22:43.586767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:43.586767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:22:43.586767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:22:43.586767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:43.586767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:22:43.586767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:43.586767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:22:43.587375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:43.587375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:22:43.587375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:22:43.587375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:22:43.587375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:22:43.587375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:22:43.587375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:22:43.587375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:22:43.587375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:22:43.587375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:43.587375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:22:43.587722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:43.587722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:22:43.587722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:43.587722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:22:43.587722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:43.587722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:22:43.587722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:43.587722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:22:43.587722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:22:43.587722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:43.587722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:22:43.587722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:43.587722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:22:43.587722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:22:43.588314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:43.588314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:22:43.588314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:43.588314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:22:43.588994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:43.588994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:22:43.588994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:22:43.588994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:22:43.588994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:22:43.588994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:22:43.588994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:22:43.588994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:22:43.588994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:22:43.588994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:22:43.588994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:22:43.588994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:22:43.588994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:22:43.588994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:22:43.588994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:22:43.588994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:22:43.588994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:22:43.589455 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:22:43.589455 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:22:43.589455 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:22:43.589455 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:43.589455 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:22:43.589455 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:43.589455 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:22:43.589455 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:43.589455 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:22:43.589455 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:43.589455 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:22:43.589455 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:43.589455 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:22:43.589455 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:22:43.589455 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:43.589455 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:22:43.589455 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:43.589455 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:22:43.589455 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:22:43.589993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:43.589993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:22:43.589993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:43.589993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:22:43.589993 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:22:43.589993 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:22:43.589993 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:22:43.590493 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:22:43.590846 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:22:43.591074 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:22:43.591256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:43.591805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:22:43.591805 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:43.591805 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:22:43.591805 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:22:43.591805 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:43.591805 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:22:43.591805 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:43.591805 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:22:43.592934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:43.592934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:22:43.592934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:22:43.592934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:22:43.592934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:22:43.592934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:22:43.592934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:22:43.592934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:43.592934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:22:43.592934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:43.592934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:22:43.592934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:43.593317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:22:43.593317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:43.593317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:22:43.593317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:43.593317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:22:43.593317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:22:43.593317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:43.593317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:22:43.593317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:43.593317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:22:43.593317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:22:43.593317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:43.593317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:22:43.593654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:43.593654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:22:43.594908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:43.594908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:22:43.594908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:22:43.594908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:22:43.594908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:22:43.594908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:22:43.594908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:22:43.594908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:22:43.594908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:22:43.594908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:22:43.594908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:22:43.594908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:22:43.594908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:22:43.594908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:22:43.594908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:22:43.594908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:22:43.595408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:22:43.595408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:22:43.595408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:43.595408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:22:43.595408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:43.595408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:22:43.595408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:43.595408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:22:43.595408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:43.595408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:22:43.595408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:43.595408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:22:43.595408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:22:43.595408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:43.595408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:22:43.595408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:43.595408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:22:43.595408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:22:43.595408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:43.595936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:22:43.595936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:43.595936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:22:48.405927 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:22:48.406542 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:22:48.406542 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:22:48.407223 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:22:48.436936 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:22:48.437258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:48.438143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:22:48.438143 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:48.438143 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:22:48.438143 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:22:48.438143 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:48.438143 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:22:48.438143 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:48.438143 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:22:48.439232 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.032 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:48.439232 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:22:48.439232 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:22:48.439232 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:22:48.439232 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:22:48.439232 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:22:48.439232 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:22:48.439232 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:48.439232 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:22:48.439232 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:48.439232 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:22:48.439232 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:48.439720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:22:48.439720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:48.439720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:22:48.439720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:48.439720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:22:48.439720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:22:48.439720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:48.439720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:22:48.439720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:48.439720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:22:48.439720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:22:48.439720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:48.439720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:22:48.440091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:48.440091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:22:48.440359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:48.440359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:22:48.440359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:22:48.440359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:22:48.440359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:22:48.440359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:22:48.440359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:22:48.440359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:22:48.440359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:22:48.440359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:22:48.440359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:22:48.440359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:22:48.440359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:22:48.440359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:22:48.440359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:22:48.440359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:22:48.440862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:22:48.440862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:22:48.440862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:48.440862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:22:48.440862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:48.440862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:22:48.440862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:48.440862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:22:48.440862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:48.440862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:22:48.440862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:48.440862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:22:48.440862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:22:48.440862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:48.440862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:22:48.440862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:48.440862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:22:48.440862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:22:48.440862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:48.441394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:22:48.441394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:48.441394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:22:48.565826 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:22:48.566182 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:22:48.566303 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:22:48.567575 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:22:48.567913 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:22:48.568125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:48.568996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:22:48.568996 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:48.568996 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:22:48.568996 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:22:48.568996 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:48.568996 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:22:48.568996 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:48.568996 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:22:48.570146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:48.570146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:22:48.570146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:22:48.570146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:22:48.570146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:22:48.570146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:22:48.570146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:22:48.570146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:48.570146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:22:48.570146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:48.570146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:22:48.570146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:48.570674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:22:48.570674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:48.570674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:22:48.570674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:48.570674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:22:48.570674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:22:48.570674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:48.570674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:22:48.570674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:48.570674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:22:48.570674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:22:48.570674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:48.570674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:22:48.571174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:48.571174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:22:48.571174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:48.571174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:22:48.571174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:22:48.571174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:22:48.571174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:22:48.571174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:22:48.571174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:22:48.571174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:22:48.571174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:22:48.571174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:22:48.571174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:22:48.571174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:22:48.571174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:22:48.571727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:22:48.571727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:22:48.571727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:22:48.571727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:22:48.571727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:22:48.571727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:48.571727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:22:48.571727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:48.571727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:22:48.571727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:48.571727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:22:48.571727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:48.571727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:22:48.571727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:48.571727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:22:48.571727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:22:48.571727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:48.571727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:22:48.571727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:48.572415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:22:48.572415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:22:48.572415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:48.572415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:22:48.572415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:48.572415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:22:48.575127 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:22:48.575384 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:22:48.575538 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:22:48.575649 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:22:48.576263 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:22:48.576507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:48.577165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:22:48.577165 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:48.577165 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:22:48.577165 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:22:48.577165 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:48.577165 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:22:48.577165 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:48.577165 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:22:48.578066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:48.578066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:22:48.578066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:22:48.578066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:22:48.578066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:48.578066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:22:48.578066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:48.578066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:22:48.578066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:48.578066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:22:48.578066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:48.578066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:22:48.578066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:48.578066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:22:48.578066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:22:48.578659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:48.578659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:22:48.578659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:48.578659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:22:48.578659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:22:48.578659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:48.578659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:22:48.578659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:48.578659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:22:48.579008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:48.579008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:22:48.579008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:22:48.579008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:22:48.579008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:22:48.579008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:22:48.579008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:22:48.579008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:22:48.579008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:22:48.579008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:22:48.579008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:48.579008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:22:48.579345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:48.579345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:22:48.579345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:48.579345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:22:48.579345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:48.579345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:22:48.579345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:48.579345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:22:48.579345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:22:48.579345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:48.579345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:22:48.579345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:48.579345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:22:48.579345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:22:48.580121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:48.580121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:22:48.580121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:48.580121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:22:48.580857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:48.580857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:22:48.580857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:22:48.580857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:22:48.580857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:22:48.580857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:22:48.580857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:22:48.580857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:22:48.580857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:22:48.580857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:22:48.580857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:22:48.580857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:22:48.580857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:22:48.580857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:22:48.580857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:22:48.580857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:22:48.581512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:22:48.581512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:22:48.581512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:22:48.581512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:22:48.581512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:22:48.581512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:48.581512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:22:48.581512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:48.581512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:22:48.581512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:48.581512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:22:48.581512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:48.581512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:22:48.581512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:48.581512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:22:48.581512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:22:48.581512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:48.581512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:22:48.582013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:48.582013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:22:48.582013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:22:48.582013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:48.582013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:22:48.582013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:48.582013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:22:48.583317 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:22:48.583583 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:22:48.583704 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:22:48.583867 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:22:48.584227 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:22:48.584412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:48.584870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:22:48.584870 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:48.584870 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:22:48.584870 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:22:48.584870 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:48.584870 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:22:48.584870 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:48.584870 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:22:48.585663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:48.585663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:22:48.585663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:22:48.585663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:22:48.585663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:48.585663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:22:48.585663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:48.585663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:22:48.585663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:48.585663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:22:48.585663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:48.585663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:22:48.585663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:48.585663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:22:48.585663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:22:48.586086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:48.586086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:22:48.586086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:48.586086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:22:48.586086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:22:48.586086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:48.586086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:22:48.586086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:48.586086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:22:48.586718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:48.586718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:22:48.586718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:22:48.586718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:22:48.586718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:22:48.586718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:22:48.586718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:22:48.586718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:22:48.586718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:22:48.586718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:48.586718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:22:48.587042 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:48.587042 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:22:48.587042 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:48.587042 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:22:48.587042 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:48.587042 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:22:48.587042 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:48.587042 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:22:48.587042 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:22:48.587042 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:48.587042 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:22:48.587042 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:48.587042 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:22:48.587042 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:22:48.587441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:48.587441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:22:48.587441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:48.587441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:22:48.588149 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:48.588149 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:22:48.588149 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:22:48.588149 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:22:48.588149 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:22:48.588149 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:22:48.588149 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:22:48.588149 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:22:48.588149 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:22:48.588149 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:22:48.588149 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:22:48.588149 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:22:48.588149 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:22:48.588149 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:22:48.588149 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:22:48.588149 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:22:48.588149 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:22:48.588607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:22:48.588607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:22:48.588607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:22:48.588607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:48.588607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:22:48.588607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:48.588607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:22:48.588607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:48.588607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:22:48.588607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:48.588607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:22:48.588607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:48.588607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:22:48.588607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:22:48.588607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:48.588607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:22:48.588607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:48.588607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:22:48.588607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:22:48.589112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:48.589112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:22:48.589112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:48.589112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:22:48.591591 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:22:48.591924 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:22:48.592052 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:22:48.592258 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:22:48.592902 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:22:48.593162 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:22:48.593317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:48.593985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:22:48.593985 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:48.593985 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:22:48.593985 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:22:48.593985 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:48.593985 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:22:48.593985 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:48.593985 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:22:48.594928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:48.594928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:22:48.594928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:22:48.594928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:22:48.594928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:22:48.594928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:22:48.594928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:22:48.594928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:48.594928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:22:48.594928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:48.594928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:22:48.594928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:48.595399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:22:48.595399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:48.595399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:22:48.595399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:48.595399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:22:48.595399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:22:48.595399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:48.595399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:22:48.595399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:48.595399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:22:48.595399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:22:48.595399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:48.595399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:22:48.595923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:48.595923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:22:48.595923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:48.595923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:22:48.595923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:22:48.595923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:22:48.595923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:22:48.595923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:22:48.595923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:22:48.595923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:22:48.595923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:22:48.595923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:22:48.595923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:22:48.595923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:22:48.595923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:22:48.596491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:22:48.596491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:22:48.596491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:22:48.596491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:22:48.596491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:22:48.596491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:48.596491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:22:48.596491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:48.596491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:22:48.596491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:48.596491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:22:48.596491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:48.596491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:22:48.596491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:48.596491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:22:48.596491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:22:48.596491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:48.596491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:22:48.596491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:48.597045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:22:48.597045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:22:48.597045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:48.597045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:22:48.597045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:48.597045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:22:53.406569 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:22:53.407333 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:22:53.407333 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:22:53.407904 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:22:53.450829 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:22:53.451224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:53.452155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:22:53.452155 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:53.452155 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:22:53.452155 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:22:53.452155 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:53.452155 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:22:53.452155 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:53.452155 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:22:53.453207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.045 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:53.453207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:22:53.453207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:22:53.453207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:22:53.453207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:22:53.453207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:22:53.453207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:22:53.453207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:53.453207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:22:53.453207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:53.453207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:22:53.453207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:53.453509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:22:53.453509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:53.453509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:22:53.453509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:53.453509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:22:53.453509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:22:53.453509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:53.453509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:22:53.453509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:53.453509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:22:53.453509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:22:53.453509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:53.453509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:22:53.453860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:53.453860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:22:53.454284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:53.454284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:22:53.454284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:22:53.454284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:22:53.454284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:22:53.454284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:22:53.454284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:22:53.454284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:22:53.454284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:22:53.454284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:22:53.454284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:22:53.454284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:22:53.454284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:22:53.454284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:22:53.454284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:22:53.454284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:22:53.454719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:22:53.454719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:22:53.454719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:53.454719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:22:53.454719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:53.454719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:22:53.454719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:53.454719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:22:53.454719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:53.454719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:22:53.454719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:53.454719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:22:53.454719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:22:53.454719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:53.454719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:22:53.454719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:53.454719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:22:53.454719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:22:53.454719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:53.455536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:22:53.455536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:53.455536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:22:53.566145 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:22:53.566660 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:22:53.566890 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:22:53.568010 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:22:53.568258 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:22:53.568447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:53.569056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:22:53.569056 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:53.569056 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:22:53.569056 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:22:53.569056 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:53.569056 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:22:53.569056 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:53.569056 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:22:53.569897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:53.569897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:22:53.569897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:22:53.569897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:22:53.569897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:22:53.569897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:22:53.569897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:22:53.569897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:53.569897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:22:53.569897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:53.569897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:22:53.569897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:53.570270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:22:53.570270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:53.570270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:22:53.570270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:53.570270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:22:53.570270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:22:53.570270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:53.570270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:22:53.570270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:53.570270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:22:53.570270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:22:53.570270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:53.570270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:22:53.570583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:53.570583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:22:53.571650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:53.571650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:22:53.571650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:22:53.571650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:22:53.571650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:22:53.571650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:22:53.571650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:22:53.571650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:22:53.571650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:22:53.571650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:22:53.571650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:22:53.571650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:22:53.571650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:22:53.571650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:22:53.571650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:22:53.571650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:22:53.572093 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:22:53.572093 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:22:53.572093 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:53.572093 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:22:53.572093 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:53.572093 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:22:53.572093 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:53.572093 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:22:53.572093 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:53.572093 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:22:53.572093 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:53.572093 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:22:53.572093 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:22:53.572093 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:53.572093 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:22:53.572093 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:53.572093 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:22:53.572093 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:22:53.572093 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:53.572461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:22:53.572461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:53.572461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:22:53.575639 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:22:53.575639 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:22:53.575837 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:22:53.575837 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:22:53.576158 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:22:53.576355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:53.576874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:22:53.576874 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:53.576874 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:22:53.576874 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:22:53.576874 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:53.576874 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:22:53.576874 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:53.576874 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:22:53.577443 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:53.577443 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:22:53.577443 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:22:53.577443 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:22:53.577443 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:53.577443 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:22:53.577443 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:53.577443 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:22:53.577443 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:53.577443 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:22:53.577443 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:53.577443 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:22:53.577443 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:53.577443 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:22:53.577443 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:22:53.578199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:53.578199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:22:53.578199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:53.578199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:22:53.578199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:22:53.578199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:53.578199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:22:53.578199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:53.578199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:22:53.578199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:53.578199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:22:53.578199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:22:53.578199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:22:53.578199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:22:53.579048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:22:53.579048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:22:53.579048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:22:53.579048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:22:53.579048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:22:53.579048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:53.579048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:22:53.579048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:53.579048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:22:53.579048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:53.579048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:22:53.579048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:53.579048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:22:53.579641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:53.579641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:22:53.579641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:22:53.579641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:53.579641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:22:53.579641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:53.579641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:22:53.579641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:22:53.579641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:53.579641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:22:53.579641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:53.579641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:22:53.580249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:53.580249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:22:53.580249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:22:53.580249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:22:53.580249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:22:53.580249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:22:53.580249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:22:53.580249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:22:53.580249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:22:53.580249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:22:53.580249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:22:53.580249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:22:53.580249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:22:53.580249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:22:53.580249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:22:53.580249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:22:53.581588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:22:53.581588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:22:53.581588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:22:53.581588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:22:53.581588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:22:53.581588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:53.581588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:22:53.581588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:53.581588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:22:53.581588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:53.581588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:22:53.581588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:53.581588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:22:53.581588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:53.581588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:22:53.581588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:22:53.581588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:53.581588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:22:53.582598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:53.582598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:22:53.582598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:22:53.582598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:53.582598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:22:53.582598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:53.582598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:22:53.583876 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:22:53.584191 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:22:53.584191 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:22:53.584307 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:22:53.585500 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:22:53.585776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:53.586290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:22:53.586290 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:53.586290 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:22:53.586290 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:22:53.586290 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:53.586290 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:22:53.586290 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:53.586290 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:22:53.586988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:53.586988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:22:53.586988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:22:53.586988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:22:53.586988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:53.586988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:22:53.586988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:53.586988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:22:53.586988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:53.586988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:22:53.586988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:53.586988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:22:53.586988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:53.586988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:22:53.586988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:22:53.587508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:53.587508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:22:53.587508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:53.587508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:22:53.587508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:22:53.587508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:53.587508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:22:53.587508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:53.587508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:22:53.587781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:53.587781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:22:53.587781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:22:53.587781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:22:53.587781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:22:53.587781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:22:53.587781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:22:53.587781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:22:53.587781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:22:53.587781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:53.587781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:22:53.588102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:53.588102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:22:53.588102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:53.588102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:22:53.588102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:53.588102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:22:53.588102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:53.588102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:22:53.588102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:22:53.588102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:53.588102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:22:53.588102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:53.588102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:22:53.588102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:22:53.588476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:53.588476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:22:53.588476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:53.588476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:22:53.588476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:53.588476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:22:53.588476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:22:53.588476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:22:53.588476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:22:53.588476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:22:53.588476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:22:53.588476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:22:53.588476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:22:53.588476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:22:53.588860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:22:53.588860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:22:53.588860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:22:53.588860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:22:53.588860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:22:53.588860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:22:53.588860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:22:53.588860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:22:53.588860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:22:53.588860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:22:53.588860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:53.588860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:22:53.588860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:53.588860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:22:53.588860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:53.588860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:22:53.588860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:53.588860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:22:53.588860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:53.589338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:22:53.589338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:22:53.589338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:53.589338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:22:53.589338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:53.589338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:22:53.589338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:22:53.589338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:53.589338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:22:53.589338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:53.589338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:22:53.591931 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:22:53.592528 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:22:53.592528 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:22:53.592528 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:22:53.592683 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:22:53.592962 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:22:53.593037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:53.593549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:22:53.593549 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:53.593549 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:22:53.593549 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:22:53.593549 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:53.593549 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:22:53.593549 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:53.593549 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:22:53.594206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:53.594206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:22:53.594206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:22:53.594206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:22:53.594206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:22:53.594206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:22:53.594206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:22:53.594206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:53.594206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:22:53.594206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:53.594206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:22:53.594206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:53.594813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:22:53.594813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:53.594813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:22:53.594813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:53.594813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:22:53.594813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:22:53.594813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:53.594813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:22:53.594813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:53.594813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:22:53.594813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:22:53.594813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:53.594813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:22:53.595137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:53.595137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:22:53.595137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:53.595137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:22:53.595137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:22:53.595137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:22:53.595137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:22:53.595137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:22:53.595137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:22:53.595137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:22:53.595137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:22:53.595137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:22:53.595137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:22:53.595137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:22:53.595137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:22:53.595507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:22:53.595507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:22:53.595507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:22:53.595507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:22:53.595507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:22:53.595507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:53.595507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:22:53.595507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:53.595507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:22:53.595507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:53.595507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:22:53.595507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:53.595507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:22:53.595507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:53.595507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:22:53.595507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:22:53.595507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:53.595507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:22:53.595507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:53.595920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:22:53.595920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:22:53.595920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:53.595920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:22:53.595920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:53.595920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:22:58.408089 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:22:58.408940 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:22:58.408940 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:22:58.409972 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:22:58.412479 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG dbcounter [-] [106314] Writing DB stats neutron:SELECT=239 {{(pid=106314) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 09 09:22:58.451823 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:22:58.452125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:58.453037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:22:58.453037 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:58.453037 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:22:58.453037 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:22:58.453037 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:58.453037 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:22:58.453037 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:58.453037 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:22:58.454926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.044 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:58.454926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:22:58.454926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:22:58.454926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:22:58.454926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:22:58.454926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:22:58.454926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:22:58.454926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:58.454926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:22:58.454926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:58.454926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:22:58.454926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:58.455495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:22:58.455495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:58.455495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:22:58.455495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:58.455495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:22:58.455495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:22:58.455495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:58.455495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:22:58.455495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:58.455495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:22:58.455495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:22:58.455495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:58.455495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:22:58.456056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:58.456056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:22:58.457424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:58.457424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:22:58.457424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:22:58.457424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:22:58.457424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:22:58.457424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:22:58.457424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:22:58.457424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:22:58.457424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:22:58.457424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:22:58.457424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:22:58.457424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:22:58.457424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:22:58.457424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:22:58.457424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:22:58.457424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:22:58.458078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:22:58.458078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:22:58.458078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:58.458078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:22:58.458078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:58.458078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:22:58.458078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:58.458078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:22:58.458078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:58.458078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:22:58.458078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:58.458078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:22:58.458078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:22:58.458078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:58.458078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:22:58.458078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:58.458078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:22:58.458078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:22:58.458078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:58.458697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:22:58.458697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:58.458697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:22:58.566841 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:22:58.567588 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:22:58.568324 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:22:58.569439 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:22:58.569773 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:22:58.569958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:58.571261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:22:58.571261 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:58.571261 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:22:58.571261 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:22:58.571261 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:58.571261 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:22:58.571261 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:58.571261 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:22:58.572615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:58.572615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:22:58.572615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:22:58.572615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:22:58.572615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:22:58.572615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:22:58.572615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:22:58.572615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:58.572615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:22:58.572615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:58.572615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:22:58.572615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:58.573237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:22:58.573237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:58.573237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:22:58.573237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:58.573237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:22:58.573237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:22:58.573237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:58.573237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:22:58.573237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:58.573237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:22:58.573237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:22:58.573237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:58.573237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:22:58.573875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:58.573875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:22:58.573875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:58.573875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:22:58.573875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:22:58.573875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:22:58.573875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:22:58.573875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:22:58.573875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:22:58.573875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:22:58.573875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:22:58.573875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:22:58.573875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:22:58.573875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:22:58.573875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:22:58.574517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:22:58.574517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:22:58.574517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:22:58.574517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:22:58.574517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:22:58.574517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:58.574517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:22:58.574517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:58.574517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:22:58.574517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:58.574517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:22:58.574517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:58.574517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:22:58.574517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:58.574517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:22:58.574517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:22:58.574517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:58.574517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:22:58.574517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:58.575420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:22:58.575420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:22:58.575420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:58.575420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:22:58.575420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:58.575420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:22:58.575945 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:22:58.576295 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:22:58.576401 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:22:58.576537 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:22:58.577002 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:22:58.577223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:58.577851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:22:58.577851 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:58.577851 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:22:58.577851 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:22:58.577851 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:58.577851 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:22:58.577851 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:58.577851 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:22:58.578690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:58.578690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:22:58.578690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:22:58.578690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:22:58.578690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:58.578690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:22:58.578690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:58.578690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:22:58.578690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:58.578690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:22:58.578690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:58.578690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:22:58.578690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:58.578690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:22:58.578690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:22:58.579370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:58.579370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:22:58.579370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:58.579370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:22:58.579370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:22:58.579370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:58.579370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:22:58.579370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:58.579370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:22:58.579908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:58.579908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:22:58.579908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:22:58.579908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:22:58.579908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:22:58.579908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:22:58.579908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:22:58.579908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:22:58.579908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:22:58.579908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:22:58.579908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:58.579908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:22:58.580577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:58.580577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:22:58.580577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:58.580577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:22:58.580577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:58.580577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:22:58.580577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:58.580577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:22:58.580577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:22:58.580577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:58.580577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:22:58.580577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:58.580577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:22:58.580577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:22:58.581198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:58.581198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:22:58.581198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:58.581198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:22:58.581198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:58.581198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:22:58.581198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:22:58.581198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:22:58.581198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:22:58.581198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:22:58.581198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:22:58.581198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:22:58.581198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:22:58.581198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:22:58.581856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:22:58.581856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:22:58.581856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:22:58.581856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:22:58.581856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:22:58.581856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:22:58.581856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:22:58.581856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:22:58.581856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:22:58.581856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:22:58.581856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:22:58.581856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:58.581856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:22:58.581856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:58.581856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:22:58.581856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:58.581856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:22:58.581856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:58.581856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:22:58.582671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:58.582671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:22:58.582671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:22:58.582671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:58.582671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:22:58.582671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:58.582671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:22:58.582671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:22:58.582671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:58.582671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:22:58.582671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:58.582671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:22:58.584214 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:22:58.584494 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:22:58.584611 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:22:58.585519 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:22:58.586264 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:22:58.586559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:58.588406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:22:58.588406 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:58.588406 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:22:58.588406 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:22:58.588406 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:58.588406 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:22:58.588406 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:58.588406 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:22:58.589396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:58.589396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:22:58.589396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:22:58.589396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:22:58.589396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:58.589396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:22:58.589396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:58.589396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:22:58.589396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:58.589396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:22:58.589396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:58.589396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:22:58.589396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:58.589396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:22:58.589396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:22:58.589670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:58.589670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:22:58.589670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:58.589670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:22:58.589670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:22:58.589670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:58.589670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:22:58.589670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:58.589670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:22:58.590430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:58.590430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:22:58.590430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:22:58.590430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:22:58.590430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:22:58.590430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:22:58.590430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:22:58.590430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:22:58.590430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:22:58.590430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:58.590430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:22:58.590664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:58.590664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:22:58.590664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:58.590664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:22:58.590664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:58.590664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:22:58.590664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:58.590664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:22:58.590664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:22:58.590664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:58.590664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:22:58.590664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:58.590664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:22:58.590664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:22:58.590967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:58.590967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:22:58.590967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:58.590967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:22:58.593449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:58.593449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:22:58.593449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:22:58.593449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:22:58.593449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:22:58.593449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:22:58.593449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:22:58.593449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:22:58.593449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:22:58.593449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:22:58.593449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:22:58.593449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:22:58.593449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:22:58.593449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:22:58.593449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:22:58.593449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:22:58.593449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:22:58.593879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:22:58.593879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:22:58.593879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:22:58.593879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:58.593879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:22:58.593879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:58.593879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:22:58.593879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:58.593879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:22:58.593879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:58.593879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:22:58.593879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:58.593879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:22:58.593879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:22:58.593879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:58.593879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:22:58.593879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:58.593879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:22:58.593879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:22:58.594467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:58.594467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:22:58.594467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:58.594467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:22:58.594467 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:22:58.594467 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:22:58.594467 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:22:58.594467 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:22:58.595177 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:22:58.595435 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:22:58.595598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:58.596211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:22:58.596211 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:58.596211 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:22:58.596211 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:22:58.596211 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:58.596211 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:22:58.596211 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:58.596211 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:22:58.597038 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:58.597038 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:22:58.597038 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:22:58.597038 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:22:58.597038 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:22:58.597038 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:22:58.597038 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:22:58.597038 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:58.597038 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:22:58.597038 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:58.597038 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:22:58.597038 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:58.597413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:22:58.597413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:58.597413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:22:58.597413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:58.597413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:22:58.597413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:22:58.597413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:58.597413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:22:58.597413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:58.597413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:22:58.597413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:22:58.597413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:58.597413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:22:58.597812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:58.597812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:22:58.597812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:58.597812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:22:58.597812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:22:58.597812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:22:58.597812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:22:58.597812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:22:58.597812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:22:58.597812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:22:58.597812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:22:58.597812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:22:58.597812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:22:58.597812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:22:58.597812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:22:58.598260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:22:58.598260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:22:58.598260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:22:58.598260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:22:58.598260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:22:58.598260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:58.598260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:22:58.598260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:22:58.598260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:22:58.598260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:22:58.598260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:22:58.598260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:22:58.598260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:22:58.598260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:22:58.598260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:22:58.598260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:22:58.598260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:22:58.598260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:22:58.598260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:22:58.598824 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:22:58.598824 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:22:58.598824 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:22:58.598824 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:22:58.598824 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:22:58.598824 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:23:03.408655 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:23:03.409345 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:23:03.409520 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:23:03.410714 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:23:03.455032 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:23:03.455587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:03.457322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:23:03.457322 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:03.457322 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:23:03.457322 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:23:03.457322 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:03.457322 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:23:03.457322 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:03.457322 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:23:03.458525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.048 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:03.458525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:23:03.458525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:23:03.458525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:23:03.458525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:23:03.458525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:23:03.458525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:23:03.458525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:03.458525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:23:03.458525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:03.458525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:23:03.458525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:03.458953 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:23:03.458953 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:03.458953 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:23:03.458953 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:03.458953 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:23:03.458953 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:23:03.458953 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:03.458953 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:23:03.458953 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:03.458953 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:23:03.458953 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:23:03.458953 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:03.458953 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:23:03.459355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:03.459355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:23:03.460882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:03.460882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:23:03.460882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:23:03.460882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:23:03.460882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:23:03.460882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:23:03.460882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:23:03.460882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:23:03.460882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:23:03.460882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:23:03.460882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:23:03.460882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:23:03.460882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:23:03.460882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:23:03.460882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:23:03.460882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:23:03.461550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:23:03.461550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:23:03.461550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:03.461550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:23:03.461550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:03.461550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:23:03.461550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:03.461550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:23:03.461550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:03.461550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:23:03.461550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:03.461550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:23:03.461550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:23:03.461550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:03.461550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:23:03.461550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:03.461550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:23:03.461550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:23:03.461550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:03.462605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:23:03.462605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:03.462605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:23:03.567438 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:23:03.567959 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:23:03.568097 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:23:03.568810 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:23:03.568993 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:23:03.569140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:03.569807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:23:03.569807 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:03.569807 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:23:03.569807 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:23:03.569807 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:03.569807 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:23:03.569807 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:03.569807 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:23:03.570840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:03.570840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:23:03.570840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:23:03.570840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:23:03.570840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:23:03.570840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:23:03.570840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:23:03.570840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:03.570840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:23:03.570840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:03.570840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:23:03.570840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:03.571175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:23:03.571175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:03.571175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:23:03.571175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:03.571175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:23:03.571175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:23:03.571175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:03.571175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:23:03.571175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:03.571175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:23:03.571175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:23:03.571175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:03.571175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:23:03.571474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:03.571474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:23:03.573252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:03.573252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:23:03.573252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:23:03.573252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:23:03.573252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:23:03.573252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:23:03.573252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:23:03.573252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:23:03.573252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:23:03.573252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:23:03.573252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:23:03.573252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:23:03.573252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:23:03.573252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:23:03.573252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:23:03.573252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:23:03.573710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:23:03.573710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:23:03.573710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:03.573710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:23:03.573710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:03.573710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:23:03.573710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:03.573710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:23:03.573710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:03.573710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:23:03.573710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:03.573710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:23:03.573710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:23:03.573710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:03.573710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:23:03.573710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:03.573710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:23:03.573710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:23:03.573710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:03.574170 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:23:03.574170 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:03.574170 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:23:03.576497 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:23:03.577008 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:23:03.577143 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:23:03.577273 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:23:03.577684 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:23:03.577964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:03.578542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:23:03.578542 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:03.578542 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:23:03.578542 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:23:03.578542 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:03.578542 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:23:03.578542 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:03.578542 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:23:03.579437 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:03.579437 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:23:03.579437 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:23:03.579437 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:23:03.579437 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:03.579437 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:23:03.579437 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:03.579437 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:23:03.579437 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:03.579437 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:23:03.579437 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:03.579437 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:23:03.579437 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:03.579437 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:23:03.579437 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:23:03.579740 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:03.579740 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:23:03.579740 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:03.579740 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:23:03.579740 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:23:03.579740 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:03.579740 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:23:03.579740 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:03.579740 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:23:03.580487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:03.580487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:23:03.580487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:23:03.580487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:23:03.580487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:23:03.580487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:23:03.580487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:23:03.580487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:23:03.580487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:23:03.580487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:23:03.580487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:03.580487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:23:03.580739 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:03.580739 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:23:03.580739 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:03.580739 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:23:03.580739 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:03.580739 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:23:03.580739 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:03.580739 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:23:03.580739 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:23:03.580739 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:03.580739 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:23:03.580739 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:03.580739 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:23:03.580739 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:23:03.581080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:03.581080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:23:03.581080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:03.581080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:23:03.581585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:03.581585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:23:03.581585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:23:03.581585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:23:03.581585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:23:03.581585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:23:03.581585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:23:03.581585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:23:03.581585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:23:03.581585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:23:03.581585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:23:03.581585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:23:03.581585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:23:03.581585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:23:03.581585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:23:03.581585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:23:03.581965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:23:03.581965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:23:03.581965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:23:03.581965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:23:03.581965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:23:03.581965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:03.581965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:23:03.581965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:03.581965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:23:03.581965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:03.581965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:23:03.581965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:03.581965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:23:03.581965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:03.581965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:23:03.581965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:23:03.581965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:03.581965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:23:03.582307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:03.582307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:23:03.582307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:23:03.582307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:03.582307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:23:03.582307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:03.582307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:23:03.584665 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:23:03.585017 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:23:03.585185 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:23:03.585362 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:23:03.585703 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:23:03.585973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:03.586484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:23:03.586484 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:03.586484 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:23:03.586484 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:23:03.586484 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:03.586484 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:23:03.586484 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:03.586484 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:23:03.587256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:03.587256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:23:03.587256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:23:03.587256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:23:03.587256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:03.587256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:23:03.587256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:03.587256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:23:03.587256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:03.587256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:23:03.587256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:03.587256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:23:03.587256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:03.587256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:23:03.587256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:23:03.587807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:03.587807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:23:03.587807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:03.587807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:23:03.587807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:23:03.587807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:03.587807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:23:03.587807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:03.587807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:23:03.588866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:03.588866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:23:03.588866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:23:03.588866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:23:03.588866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:23:03.588866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:23:03.588866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:23:03.588866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:23:03.588866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:23:03.588866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:03.588866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:23:03.589285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:03.589285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:23:03.589285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:03.589285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:23:03.589285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:03.589285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:23:03.589285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:03.589285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:23:03.589285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:23:03.589285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:03.589285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:23:03.589285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:03.589285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:23:03.589285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:23:03.589768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:03.589768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:23:03.589768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:03.589768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:23:03.589984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:03.589984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:23:03.589984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:23:03.589984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:23:03.589984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:23:03.589984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:23:03.589984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:23:03.589984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:23:03.589984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:23:03.589984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:23:03.589984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:23:03.589984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:23:03.589984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:23:03.589984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:23:03.589984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:23:03.589984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:23:03.589984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:23:03.590587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:23:03.590587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:23:03.590587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:23:03.590587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:03.590587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:23:03.590587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:03.590587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:23:03.590587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:03.590587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:23:03.590587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:03.590587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:23:03.590587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:03.590587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:23:03.590587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:23:03.590587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:03.590587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:23:03.590587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:03.590587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:23:03.590587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:23:03.591262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:03.591262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:23:03.591262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:03.591262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:23:03.594765 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:23:03.595201 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:23:03.595360 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:23:03.595589 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:23:03.595929 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:23:03.596158 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:23:03.596310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:03.596794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:23:03.596794 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:03.596794 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:23:03.596794 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:23:03.596794 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:03.596794 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:23:03.596794 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:03.596794 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:23:03.598000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:03.598000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:23:03.598000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:23:03.598000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:23:03.598000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:23:03.598000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:23:03.598000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:23:03.598000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:03.598000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:23:03.598000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:03.598000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:23:03.598000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:03.598433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:23:03.598433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:03.598433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:23:03.598433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:03.598433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:23:03.598433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:23:03.598433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:03.598433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:23:03.598433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:03.598433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:23:03.598433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:23:03.598433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:03.598433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:23:03.598904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:03.598904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:23:03.598993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:03.598993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:23:03.598993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:23:03.598993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:23:03.598993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:23:03.598993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:23:03.598993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:23:03.598993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:23:03.598993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:23:03.598993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:23:03.598993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:23:03.598993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:23:03.598993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:23:03.598993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:23:03.598993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:23:03.598993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:23:03.599752 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:23:03.599752 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:23:03.599752 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:03.599752 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:23:03.599752 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:03.599752 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:23:03.599752 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:03.599752 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:23:03.599752 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:03.599752 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:23:03.599752 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:03.599752 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:23:03.599752 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:23:03.599752 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:03.599752 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:23:03.599752 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:03.599752 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:23:03.599752 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:23:03.599752 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:03.600477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:23:03.600477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:03.600477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:23:08.409393 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:23:08.410164 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:23:08.410164 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:23:08.411697 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:23:08.454124 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:23:08.454124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:08.454344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:23:08.454344 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:08.454344 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:23:08.454344 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:23:08.454344 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:08.454344 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:23:08.454344 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:08.454344 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:23:08.455046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.044 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:08.455046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:23:08.455046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:23:08.455046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:23:08.455046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:23:08.455046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:23:08.455046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:23:08.455046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:08.455046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:23:08.455046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:08.455046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:23:08.455046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:08.455432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:23:08.455432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:08.455432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:23:08.455432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:08.455432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:23:08.455432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:23:08.455432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:08.455432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:23:08.455432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:08.455432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:23:08.455432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:23:08.455432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:08.455432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:23:08.456848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:08.456848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:23:08.456848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:08.456848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:23:08.456848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:23:08.456848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:23:08.456848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:23:08.456848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:23:08.456848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:23:08.456848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:23:08.456848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:23:08.456848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:23:08.456848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:23:08.456848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:23:08.456848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:23:08.457656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:23:08.457656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:23:08.457656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:23:08.457656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:23:08.457656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:23:08.457656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:08.457656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:23:08.457656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:08.457656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:23:08.457656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:08.457656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:23:08.457656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:08.457656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:23:08.457656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:08.457656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:23:08.457656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:23:08.457656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:08.457656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:23:08.457656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:08.458317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:23:08.458317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:23:08.458317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:08.458317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:23:08.458317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:08.458317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:23:08.568085 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:23:08.568461 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:23:08.568551 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:23:08.570168 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:23:08.570168 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:23:08.570168 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:08.570407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:23:08.570407 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:08.570407 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:23:08.570407 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:23:08.570407 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:08.570407 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:23:08.570407 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:08.570407 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:23:08.571324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:08.571324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:23:08.571324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:23:08.571324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:23:08.571324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:23:08.571324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:23:08.571324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:23:08.571324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:08.571324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:23:08.571324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:08.571324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:23:08.571324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:08.572010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:23:08.572010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:08.572010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:23:08.572010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:08.572010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:23:08.572010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:23:08.572010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:08.572010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:23:08.572010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:08.572010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:23:08.572010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:23:08.572010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:08.572010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:23:08.572508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:08.572508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:23:08.572508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:08.572508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:23:08.572508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:23:08.572508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:23:08.572508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:23:08.572508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:23:08.572508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:23:08.572508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:23:08.572508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:23:08.572508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:23:08.572508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:23:08.572508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:23:08.572508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:23:08.573165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:23:08.573165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:23:08.573165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:23:08.573165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:23:08.573165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:23:08.573165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:08.573165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:23:08.573165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:08.573165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:23:08.573165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:08.573165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:23:08.573165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:08.573165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:23:08.573165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:08.573165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:23:08.573165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:23:08.573165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:08.573165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:23:08.573165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:08.573994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:23:08.573994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:23:08.573994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:08.573994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:23:08.573994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:08.573994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:23:08.577044 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:23:08.579573 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:23:08.579716 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:23:08.579839 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:23:08.580733 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:23:08.582408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:08.582625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:23:08.582625 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:08.582625 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:23:08.582625 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:23:08.582625 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:08.582625 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:23:08.582625 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:08.582625 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:23:08.583828 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:08.583828 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:23:08.583828 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:23:08.583828 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:23:08.583828 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:08.583828 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:23:08.583828 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:08.583828 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:23:08.583828 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:08.583828 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:23:08.583828 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:08.583828 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:23:08.583828 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:08.583828 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:23:08.583828 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:23:08.585086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:08.585086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:23:08.585086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:08.585086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:23:08.585086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:23:08.585086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:08.585086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:23:08.585086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:08.585086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:23:08.585086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:08.585086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:23:08.585086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:23:08.585086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:23:08.585086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:23:08.587286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:23:08.587286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:23:08.587286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:23:08.587286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:23:08.587286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:23:08.587286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:08.587286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:23:08.587286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:08.587286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:23:08.587286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:08.587286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:23:08.587286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:08.587286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:23:08.588393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:08.588393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:23:08.588393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:23:08.588393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:08.588393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:23:08.588393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:08.588393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:23:08.588393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:23:08.588393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:08.588393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:23:08.588393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:08.588393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:23:08.590188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:08.590188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:23:08.590188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:23:08.590188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:23:08.590188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:23:08.590188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:23:08.590188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:23:08.590188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:23:08.590188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:23:08.590188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:23:08.590188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:23:08.590188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:23:08.590188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:23:08.590188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:23:08.590188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:23:08.590188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:23:08.591263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:23:08.591263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:23:08.591263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:23:08.591263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:23:08.591263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:23:08.591263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:08.591263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:23:08.591263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:08.591263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:23:08.591263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:08.591263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:23:08.591263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:08.591263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:23:08.591263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:08.591263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:23:08.591263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:23:08.591263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:08.591263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:23:08.591963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:08.591963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:23:08.591963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:23:08.591963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:08.591963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:23:08.591963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:08.591963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:23:08.591963 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:23:08.591963 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:23:08.591963 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:23:08.592234 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:23:08.592234 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:23:08.592234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:08.592234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:23:08.592234 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:08.592234 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:23:08.592234 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:23:08.592234 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:08.592234 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:23:08.592234 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:08.592234 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:23:08.592234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:08.592526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:23:08.592526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:23:08.592526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:23:08.592526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:08.592526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:23:08.592526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:08.592526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:23:08.592526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:08.592526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:23:08.592526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:08.592526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:23:08.592526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:08.592526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:23:08.592526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:23:08.592526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:08.592526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:23:08.592526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:08.593569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:23:08.593569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:23:08.593569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:08.593569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:23:08.593569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:08.593569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:23:08.593569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:08.593569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:23:08.593569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:23:08.593569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:23:08.593569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:23:08.593569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:23:08.593569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:23:08.594836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:23:08.594836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:23:08.594836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:08.594836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:23:08.594836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:08.594836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:23:08.594836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:08.594836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:23:08.594836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:08.594836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:23:08.594836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:08.594836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:23:08.594836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:23:08.595604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:08.595604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:23:08.595604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:08.595604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:23:08.595604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:23:08.595604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:08.595604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:23:08.595604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:08.595604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:23:08.595604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:08.595604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:23:08.595604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:23:08.596163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:23:08.596163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:23:08.596163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:23:08.596163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:23:08.596163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:23:08.596163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:23:08.596163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:23:08.596163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:23:08.596163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:23:08.596163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:23:08.596163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:23:08.596163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:23:08.596163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:23:08.596163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:23:08.596163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:23:08.596163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:23:08.596163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:23:08.596163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:08.596163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:23:08.596497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:08.596497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:23:08.596497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:08.596497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:23:08.596497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:08.596497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:23:08.596497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:08.596497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:23:08.596497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:23:08.596497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:08.596497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:23:08.596497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:08.596497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:23:08.596497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:23:08.596497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:08.596497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:23:08.596497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:08.596497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:23:08.597913 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:23:08.597913 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:23:08.597913 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:23:08.597913 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:23:08.597913 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:23:08.597913 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:23:08.598057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:08.598651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:23:08.598651 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:08.598651 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:23:08.598651 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:23:08.598651 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:08.598651 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:23:08.598651 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:08.598651 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:23:08.599600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:08.599600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:23:08.599600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:23:08.599600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:23:08.599600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:23:08.599600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:23:08.599600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:23:08.599600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:08.599600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:23:08.599600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:08.599600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:23:08.599600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:08.600341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:23:08.600341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:08.600341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:23:08.600341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:08.600341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:23:08.600341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:23:08.600341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:08.600341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:23:08.600341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:08.600341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:23:08.600341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:23:08.600341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:08.600341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:23:08.600639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:08.600639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:23:08.600986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:08.600986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:23:08.600986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:23:08.600986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:23:08.600986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:23:08.600986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:23:08.600986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:23:08.600986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:23:08.600986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:23:08.600986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:23:08.600986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:23:08.600986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:23:08.600986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:23:08.600986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:23:08.600986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:23:08.600986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:23:08.601284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:23:08.601284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:23:08.601284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:08.601284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:23:08.601284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:08.601284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:23:08.601284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:08.601284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:23:08.601284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:08.601284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:23:08.601284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:08.601284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:23:08.601284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:23:08.601284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:08.601284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:23:08.601284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:08.601284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:23:08.601284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:23:08.601284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:08.601651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:23:08.601651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:08.601651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:23:13.410447 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:23:13.411183 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:23:13.411183 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:23:13.411730 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:23:13.462029 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:23:13.462029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:13.462029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:23:13.462029 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:13.462029 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:23:13.462029 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:23:13.462029 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:13.462029 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:23:13.462029 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:13.462029 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:23:13.462624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.051 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:13.462624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:23:13.462624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:23:13.462624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:23:13.462624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:23:13.462624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:23:13.462624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:23:13.462624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:13.462624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:23:13.462624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:13.462624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:23:13.462624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:13.462948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:23:13.462948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:13.462948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:23:13.462948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:13.462948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:23:13.462948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:23:13.462948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:13.462948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:23:13.462948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:13.462948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:23:13.462948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:23:13.462948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:13.462948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:23:13.463213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:13.463213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:23:13.463457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:13.463457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:23:13.463457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:23:13.463457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:23:13.463457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:23:13.463457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:23:13.463457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:23:13.463457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:23:13.463457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:23:13.463457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:23:13.463457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:23:13.463457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:23:13.463457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:23:13.463457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:23:13.463457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:23:13.463457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:23:13.464575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:23:13.464575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:23:13.464575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:13.464575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:23:13.464575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:13.464575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:23:13.464575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:13.464575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:23:13.464575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:13.464575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:23:13.464575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:13.464575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:23:13.464575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:23:13.464575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:13.464575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:23:13.464575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:13.464575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:23:13.464575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:23:13.464575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:13.465311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:23:13.465311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:13.465311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:23:13.568606 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:23:13.569086 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:23:13.569338 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:23:13.571389 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:23:13.571389 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:23:13.571389 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:13.571586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:23:13.571586 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:13.571586 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:23:13.571586 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:23:13.571586 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:13.571586 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:23:13.571586 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:13.571586 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:23:13.572822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:13.572822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:23:13.572822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:23:13.572822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:23:13.572822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:23:13.572822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:23:13.572822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:23:13.572822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:13.572822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:23:13.572822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:13.572822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:23:13.572822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:13.573309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:23:13.573309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:13.573309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:23:13.573309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:13.573309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:23:13.573309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:23:13.573309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:13.573309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:23:13.573309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:13.573309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:23:13.573309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:23:13.573309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:13.573309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:23:13.573794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:13.573794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:23:13.577513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:13.577513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:23:13.577513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:23:13.577513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:23:13.577513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:23:13.577513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:23:13.577513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:23:13.577513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:23:13.577513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:23:13.577513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:23:13.577513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:23:13.577513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:23:13.577513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:23:13.577513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:23:13.577513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:23:13.577513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:23:13.578067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:23:13.578067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:23:13.578067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:13.578067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:23:13.578067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:13.578067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:23:13.578067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:13.578067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:23:13.578067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:13.578067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:23:13.578067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:13.578067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:23:13.578067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:23:13.578067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:13.578067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:23:13.578067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:13.578067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:23:13.578067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:23:13.578067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:13.578679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:23:13.578679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:13.578679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:23:13.579637 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:23:13.580000 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:23:13.580156 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:23:13.580330 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:23:13.580647 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:23:13.581011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:13.581625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:23:13.581625 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:13.581625 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:23:13.581625 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:23:13.581625 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:13.581625 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:23:13.581625 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:13.581625 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:23:13.582707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:13.582707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:23:13.582707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:23:13.582707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:23:13.582707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:13.582707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:23:13.582707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:13.582707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:23:13.582707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:13.582707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:23:13.582707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:13.582707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:23:13.582707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:13.582707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:23:13.582707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:23:13.583659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:13.583659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:23:13.583659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:13.583659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:23:13.583659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:23:13.583659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:13.583659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:23:13.583659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:13.583659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:23:13.585103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:13.585103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:23:13.585103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:23:13.585103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:23:13.585103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:23:13.585103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:23:13.585103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:23:13.585103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:23:13.585103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:23:13.585103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:23:13.585103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:13.585103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:23:13.585612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:13.585612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:23:13.585612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:13.585612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:23:13.585612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:13.585612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:23:13.585612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:13.585612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:23:13.585612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:23:13.585612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:13.585612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:23:13.585612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:13.585612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:23:13.585612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:23:13.586719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:13.586719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:23:13.586719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:13.586719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:23:13.586719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:13.586719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:23:13.586719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:23:13.586719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:23:13.586719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:23:13.586719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:23:13.586719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:23:13.586719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:23:13.586719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:23:13.586719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:23:13.588803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:23:13.588803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:23:13.588803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:23:13.588803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:23:13.588803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:23:13.588803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:23:13.588803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:23:13.588803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:23:13.588803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:23:13.588803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:23:13.588803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:23:13.588803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:13.588803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:23:13.588803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:13.588803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:23:13.588803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:13.588803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:23:13.588803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:13.588803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:23:13.589842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:13.589842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:23:13.589842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:23:13.589842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:13.589842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:23:13.589842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:13.589842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:23:13.589842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:23:13.589842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:13.589842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:23:13.589842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:13.589842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:23:13.589842 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:23:13.589842 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:23:13.590386 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:23:13.590386 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:23:13.590386 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:23:13.590386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:13.590386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:23:13.590386 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:13.590386 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:23:13.590386 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:23:13.590386 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:13.590386 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:23:13.590386 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:13.590386 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:23:13.591025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:13.591025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:23:13.591025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:23:13.591025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:23:13.591025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:13.591025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:23:13.591025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:13.591025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:23:13.591025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:13.591025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:23:13.591025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:13.591025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:23:13.591025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:13.591025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:23:13.591025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:23:13.591477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:13.591477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:23:13.591477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:13.591477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:23:13.591477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:23:13.591477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:13.591477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:23:13.591477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:13.591477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:23:13.591477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:13.591477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:23:13.591477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:23:13.591477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:23:13.591477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:23:13.591871 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:23:13.591871 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:23:13.591871 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:23:13.591871 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:23:13.591871 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:13.591871 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:23:13.591871 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:13.591871 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:23:13.591871 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:13.591871 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:23:13.591871 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:13.591871 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:23:13.591871 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:13.592320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:23:13.592320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:23:13.592320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:13.592320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:23:13.592320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:13.592320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:23:13.592320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:23:13.592320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:13.592320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:23:13.592320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:13.592320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:23:13.592320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:13.592543 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:23:13.592543 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:23:13.592543 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:23:13.592543 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:23:13.592543 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:23:13.592543 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:23:13.592543 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:23:13.592543 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:23:13.592543 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:23:13.592543 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:23:13.592543 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:23:13.592543 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:23:13.592543 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:23:13.592543 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:23:13.592543 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:23:13.592543 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:23:13.592543 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:23:13.592543 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:23:13.592543 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:23:13.594179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:13.594179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:23:13.594179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:13.594179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:23:13.594179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:13.594179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:23:13.594179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:13.594179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:23:13.594179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:13.594179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:23:13.594179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:23:13.594179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:13.594179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:23:13.594179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:13.594179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:23:13.594179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:23:13.594179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:13.594179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:23:13.594179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:13.594179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:23:13.595723 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:23:13.596035 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:23:13.596140 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:23:13.596296 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:23:13.596532 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:23:13.596808 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:23:13.597025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:13.597431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:23:13.597431 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:13.597431 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:23:13.597431 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:23:13.597431 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:13.597431 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:23:13.597431 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:13.597431 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:23:13.598191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:13.598191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:23:13.598191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:23:13.598191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:23:13.598191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:23:13.598191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:23:13.598191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:23:13.598191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:13.598191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:23:13.598191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:13.598191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:23:13.598191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:13.598696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:23:13.598696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:13.598696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:23:13.598696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:13.598696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:23:13.598696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:23:13.598696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:13.598696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:23:13.598696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:13.598696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:23:13.598696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:23:13.598696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:13.598696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:23:13.599154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:13.599154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:23:13.599223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:13.599223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:23:13.599223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:23:13.599223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:23:13.599223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:23:13.599223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:23:13.599223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:23:13.599223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:23:13.599223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:23:13.599223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:23:13.599223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:23:13.599223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:23:13.599223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:23:13.599223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:23:13.599223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:23:13.599223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:23:13.599576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:23:13.599576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:23:13.599576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:13.599576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:23:13.599576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:13.599576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:23:13.599576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:13.599576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:23:13.599576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:13.599576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:23:13.599576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:13.599576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:23:13.599576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:23:13.599576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:13.599576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:23:13.599576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:13.599576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:23:13.599576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:23:13.599576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:13.599975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:23:13.599975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:13.599975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:23:18.410893 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:23:18.411670 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:23:18.411670 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:23:18.412416 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:23:18.454987 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:23:18.454987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:18.455221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:23:18.455221 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:18.455221 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:23:18.455221 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:23:18.455221 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:18.455221 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:23:18.455221 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:18.455221 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:23:18.456214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.044 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:18.456214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:23:18.456214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:23:18.456214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:23:18.456214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:23:18.456214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:23:18.456214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:23:18.456214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:18.456214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:23:18.456214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:18.456214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:23:18.456214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:18.456578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:23:18.456578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:18.456578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:23:18.456578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:18.456578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:23:18.456578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:23:18.456578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:18.456578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:23:18.456578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:18.456578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:23:18.456578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:23:18.456578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:18.456578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:23:18.457600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:18.457600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:23:18.457600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:18.457600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:23:18.457600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:23:18.457600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:23:18.457600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:23:18.457600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:23:18.457600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:23:18.457600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:23:18.457600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:23:18.457600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:23:18.457600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:23:18.457600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:23:18.457600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:23:18.458132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:23:18.458132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:23:18.458132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:23:18.458132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:23:18.458132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:23:18.458132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:18.458132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:23:18.458132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:18.458132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:23:18.458132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:18.458132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:23:18.458132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:18.458132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:23:18.458132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:18.458132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:23:18.458132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:23:18.458132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:18.458132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:23:18.458132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:18.458682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:23:18.458682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:23:18.458682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:18.458682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:23:18.458682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:18.458682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:23:18.569243 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:23:18.569537 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:23:18.569616 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:23:18.572060 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:23:18.572060 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:23:18.572060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:18.572231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:23:18.572231 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:18.572231 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:23:18.572231 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:23:18.572231 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:18.572231 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:23:18.572231 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:18.572231 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:23:18.573311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:18.573311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:23:18.573311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:23:18.573311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:23:18.573311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:23:18.573311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:23:18.573311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:23:18.573311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:18.573311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:23:18.573311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:18.573311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:23:18.573311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:18.573619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:23:18.573619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:18.573619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:23:18.573619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:18.573619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:23:18.573619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:23:18.573619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:18.573619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:23:18.573619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:18.573619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:23:18.573619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:23:18.573619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:18.573619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:23:18.573916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:18.573916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:23:18.574499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:18.574499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:23:18.574499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:23:18.574499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:23:18.574499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:23:18.574499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:23:18.574499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:23:18.574499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:23:18.574499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:23:18.574499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:23:18.574499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:23:18.574499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:23:18.574499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:23:18.574499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:23:18.574499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:23:18.574499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:23:18.574850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:23:18.574850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:23:18.574850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:18.574850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:23:18.574850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:18.574850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:23:18.574850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:18.574850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:23:18.574850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:18.574850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:23:18.574850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:18.574850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:23:18.574850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:23:18.574850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:18.574850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:23:18.574850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:18.574850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:23:18.574850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:23:18.574850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:18.575303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:23:18.575303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:18.575303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:23:18.580144 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:23:18.580453 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:23:18.580572 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:23:18.580715 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:23:18.581631 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:23:18.581631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:18.581830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:23:18.581830 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:18.581830 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:23:18.581830 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:23:18.581830 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:18.581830 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:23:18.581830 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:18.581830 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:23:18.582697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:18.582697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:23:18.582697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:23:18.582697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:23:18.582697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:18.582697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:23:18.582697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:18.582697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:23:18.582697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:18.582697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:23:18.582697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:18.582697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:23:18.582697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:18.582697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:23:18.582697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:23:18.583556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:18.583556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:23:18.583556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:18.583556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:23:18.583556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:23:18.583556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:18.583556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:23:18.583556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:18.583556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:23:18.583556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:18.583556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:23:18.583556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:23:18.583556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:23:18.583556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:23:18.584732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:23:18.584732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:23:18.584732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:23:18.584732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:23:18.584732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:23:18.584732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:18.584732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:23:18.584732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:18.584732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:23:18.584732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:18.584732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:23:18.584732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:18.584732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:23:18.585853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:18.585853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:23:18.585853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:23:18.585853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:18.585853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:23:18.585853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:18.585853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:23:18.585853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:23:18.585853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:18.585853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:23:18.585853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:18.585853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:23:18.586695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:18.586695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:23:18.586695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:23:18.586695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:23:18.586695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:23:18.586695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:23:18.586695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:23:18.586695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:23:18.586695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:23:18.586695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:23:18.586695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:23:18.586695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:23:18.586695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:23:18.586695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:23:18.586695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:23:18.586695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:23:18.587790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:23:18.587790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:23:18.587790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:23:18.587790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:23:18.587790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:23:18.587790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:18.587790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:23:18.587790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:18.587790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:23:18.587790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:18.587790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:23:18.587790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:18.587790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:23:18.587790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:18.587790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:23:18.587790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:23:18.587790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:18.587790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:23:18.588505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:18.588505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:23:18.588505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:23:18.588505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:18.588505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:23:18.588505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:18.588505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:23:18.588505 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:23:18.588505 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:23:18.588505 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:23:18.588925 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:23:18.590226 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:23:18.590226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:18.590435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:23:18.590435 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:18.590435 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:23:18.590435 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:23:18.590435 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:18.590435 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:23:18.590435 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:18.590435 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:23:18.591415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:18.591415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:23:18.591415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:23:18.591415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:23:18.591415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:18.591415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:23:18.591415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:18.591415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:23:18.591415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:18.591415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:23:18.591415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:18.591415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:23:18.591415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:18.591415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:23:18.591415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:23:18.591871 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:18.591871 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:23:18.591871 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:18.591871 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:23:18.591871 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:23:18.591871 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:18.591871 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:23:18.591871 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:18.591871 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:23:18.593390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:18.593390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:23:18.593390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:23:18.593390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:23:18.593390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:23:18.593390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:23:18.593390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:23:18.593390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:23:18.593390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:23:18.593390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:18.593390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:23:18.593714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:18.593714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:23:18.593714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:18.593714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:23:18.593714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:18.593714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:23:18.593714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:18.593714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:23:18.593714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:23:18.593714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:18.593714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:23:18.593714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:18.593714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:23:18.593714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:23:18.594055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:18.594055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:23:18.594055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:18.594055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:23:18.594575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:18.594575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:23:18.594575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:23:18.594575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:23:18.594575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:23:18.594575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:23:18.594575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:23:18.594575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:23:18.594575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:23:18.594575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:23:18.594575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:23:18.594575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:23:18.594575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:23:18.594575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:23:18.594575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:23:18.594575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:23:18.594575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:23:18.595117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:23:18.595117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:23:18.595117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:23:18.595117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:18.595117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:23:18.595117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:18.595117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:23:18.595117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:18.595117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:23:18.595117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:18.595117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:23:18.595117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:18.595117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:23:18.595117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:23:18.595117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:18.595117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:23:18.595117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:18.595117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:23:18.595117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:23:18.595628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:18.595628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:23:18.595628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:18.595628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:23:18.596222 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:23:18.596524 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:23:18.596627 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:23:18.596865 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:23:18.597234 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:23:18.597571 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:23:18.598190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:18.598364 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:23:18.598364 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:18.598364 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:23:18.598364 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:23:18.598364 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:18.598364 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:23:18.598364 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:18.598364 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:23:18.599384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:18.599384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:23:18.599384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:23:18.599384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:23:18.599384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:23:18.599384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:23:18.599384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:23:18.599384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:18.599384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:23:18.599384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:18.599384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:23:18.599384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:18.599767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:23:18.599767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:18.599767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:23:18.599767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:18.599767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:23:18.599767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:23:18.599767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:18.599767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:23:18.599767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:18.599767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:23:18.599767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:23:18.599767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:18.599767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:23:18.600077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:18.600077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:23:18.601142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:18.601142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:23:18.601142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:23:18.601142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:23:18.601142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:23:18.601142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:23:18.601142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:23:18.601142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:23:18.601142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:23:18.601142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:23:18.601142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:23:18.601142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:23:18.601142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:23:18.601142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:23:18.601142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:23:18.601142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:23:18.601728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:23:18.601728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:23:18.601728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:18.601728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:23:18.601728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:18.601728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:23:18.601728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:18.601728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:23:18.601728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:18.601728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:23:18.601728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:18.601728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:23:18.601728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:23:18.601728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:18.601728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:23:18.601728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:18.601728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:23:18.601728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:23:18.601728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:18.602630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:23:18.602630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:18.602630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:23:23.411570 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:23:23.412408 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:23:23.412408 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:23:23.413558 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:23:23.453952 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:23:23.453952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:23.456249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:23:23.456249 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:23.456249 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:23:23.456249 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:23:23.456249 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:23.456249 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:23:23.456249 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:23.456249 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:23:23.456249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:23.456249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:23:23.456249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:23:23.456249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:23:23.456249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:23:23.456249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:23:23.456249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:23:23.456778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:23.456778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:23:23.456778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:23.456778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:23:23.456778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:23.456778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:23:23.456778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:23.456778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:23:23.456778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:23.456778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:23:23.456778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:23:23.456778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:23.456778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:23:23.456778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:23.457102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:23:23.457102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:23:23.457102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:23.457102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:23:23.457102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:23.457102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:23:23.457102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:23.457102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:23:23.457102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:23:23.457102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:23:23.457102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:23:23.457102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:23:23.457102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:23:23.457102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:23:23.457402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:23:23.457402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:23:23.457402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:23:23.457402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:23:23.457402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:23:23.457402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:23:23.457402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:23:23.457402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:23:23.457402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:23:23.457402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:23:23.457402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:23.457402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:23:23.457402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:23.457402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:23:23.457402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:23.457402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:23:23.457402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:23.457402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:23:23.457402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:23.457402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:23:23.457402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:23:23.457860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:23.457860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:23:23.457860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:23.457860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:23:23.457860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:23:23.457860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:23.457860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:23:23.457860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:23.457860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:23:23.569723 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:23:23.571171 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:23:23.571171 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:23:23.571893 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:23:23.572163 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:23:23.572417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:23.575861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:23:23.575861 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:23.575861 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:23:23.575861 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:23:23.575861 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:23.575861 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:23:23.575861 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:23.575861 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:23:23.575861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:23.575861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:23:23.575861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:23:23.575861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:23:23.575861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:23:23.575861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:23:23.575861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:23:23.576307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:23.576307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:23:23.576307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:23.576307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:23:23.576307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:23.576307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:23:23.576307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:23.576307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:23:23.576307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:23.576307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:23:23.576307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:23:23.576307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:23.576307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:23:23.576307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:23.576648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:23:23.576648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:23:23.576648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:23.576648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:23:23.576648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:23.576648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:23:23.576648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:23.576648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:23:23.576648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:23:23.576648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:23:23.576648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:23:23.576648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:23:23.576648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:23:23.576648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:23:23.578508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:23:23.578508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:23:23.578508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:23:23.578508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:23:23.578508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:23:23.578508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:23:23.578508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:23:23.578508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:23:23.578508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:23:23.578508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:23:23.578508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:23.578508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:23:23.578508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:23.578508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:23:23.578508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:23.578508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:23:23.578508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:23.578508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:23:23.578508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:23.578508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:23:23.579589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:23:23.579589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:23.579589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:23:23.579589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:23.579589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:23:23.579589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:23:23.579589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:23.579589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:23:23.579589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:23.579589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:23:23.580649 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:23:23.581044 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:23:23.581215 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:23:23.581400 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:23:23.581953 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:23:23.582251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:23.583176 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:23:23.583176 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:23.583176 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:23:23.583176 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:23:23.583176 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:23.583176 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:23:23.583176 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:23.583176 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:23:23.584083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:23.584083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:23:23.584083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:23:23.584083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:23:23.584083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:23.584083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:23:23.584083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:23.584083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:23:23.584083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:23.584083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:23:23.584083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:23.584083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:23:23.584083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:23.584083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:23:23.584083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:23:23.584574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:23.584574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:23:23.584574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:23.584574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:23:23.584574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:23:23.584574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:23.584574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:23:23.584574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:23.584574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:23:23.585316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:23.585316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:23:23.585316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:23:23.585316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:23:23.585316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:23:23.585316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:23:23.585316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:23:23.585316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:23:23.585316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:23:23.585316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:23:23.585316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:23.585316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:23:23.585637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:23.585637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:23:23.585637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:23.585637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:23:23.585637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:23.585637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:23:23.585637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:23.585637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:23:23.585637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:23:23.585637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:23.585637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:23:23.585637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:23.585637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:23:23.585637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:23:23.586064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:23.586064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:23:23.586064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:23.586064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:23:23.587873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:23.587873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:23:23.587873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:23:23.587873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:23:23.587873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:23:23.587873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:23:23.587873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:23:23.587873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:23:23.587873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:23:23.587873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:23:23.587873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:23:23.587873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:23:23.587873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:23:23.587873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:23:23.587873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:23:23.587873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:23:23.588423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:23:23.588423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:23:23.588423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:23:23.588423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:23:23.588423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:23:23.588423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:23.588423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:23:23.588423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:23.588423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:23:23.588423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:23.588423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:23:23.588423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:23.588423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:23:23.588423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:23.588423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:23:23.588423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:23:23.588423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:23.588423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:23:23.589030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:23.589030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:23:23.589030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:23:23.589030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:23.589030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:23:23.589030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:23.589030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:23:23.589030 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:23:23.589030 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:23:23.589030 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:23:23.589424 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:23:23.589424 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:23:23.589670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:23.590486 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:23:23.590486 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:23.590486 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:23:23.590486 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:23:23.590486 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:23.590486 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:23:23.590486 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:23.590486 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:23:23.591854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:23.591854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:23:23.591854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:23:23.591854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:23:23.591854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:23.591854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:23:23.591854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:23.591854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:23:23.591854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:23.591854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:23:23.591854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:23.591854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:23:23.591854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:23.591854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:23:23.591854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:23:23.592452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:23.592452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:23:23.592452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:23.592452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:23:23.592452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:23:23.592452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:23.592452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:23:23.592452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:23.592452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:23:23.592851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:23.592851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:23:23.592851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:23:23.592851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:23:23.592851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:23:23.592851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:23:23.592851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:23:23.592851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:23:23.592851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:23:23.592851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:23.592851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:23:23.593302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:23.593302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:23:23.593302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:23.593302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:23:23.593302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:23.593302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:23:23.593302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:23.593302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:23:23.593302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:23:23.593302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:23.593302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:23:23.593302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:23.593302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:23:23.593302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:23:23.593999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:23.593999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:23:23.593999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:23.593999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:23:23.593999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:23.593999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:23:23.593999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:23:23.593999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:23:23.593999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:23:23.593999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:23:23.593999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:23:23.593999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:23:23.593999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:23:23.593999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:23:23.594498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:23:23.594498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:23:23.594498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:23:23.594498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:23:23.594498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:23:23.594498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:23:23.594498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:23:23.594498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:23:23.594498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:23:23.594498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:23:23.594498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:23.594498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:23:23.594498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:23.594498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:23:23.594498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:23.594498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:23:23.594498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:23.594498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:23:23.594498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:23.595005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:23:23.595005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:23:23.595005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:23.595005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:23:23.595005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:23.595005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:23:23.595005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:23:23.595005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:23.595005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:23:23.595005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:23.595005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:23:23.596676 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:23:23.596958 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:23:23.597057 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:23:23.597219 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:23:23.597659 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:23:23.598055 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:23:23.598281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:23.599023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:23:23.599023 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:23.599023 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:23:23.599023 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:23:23.599023 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:23.599023 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:23:23.599023 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:23.599023 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:23:23.599939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:23.599939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:23:23.599939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:23:23.599939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:23:23.599939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:23:23.599939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:23:23.599939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:23:23.599939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:23.599939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:23:23.599939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:23.599939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:23:23.599939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:23.600412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:23:23.600412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:23.600412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:23:23.600412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:23.600412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:23:23.600412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:23:23.600412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:23.600412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:23:23.600412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:23.600412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:23:23.600412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:23:23.600412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:23.600412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:23:23.600857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:23.600857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:23:23.600941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:23.600941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:23:23.600941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:23:23.600941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:23:23.600941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:23:23.600941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:23:23.600941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:23:23.600941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:23:23.600941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:23:23.600941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:23:23.600941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:23:23.600941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:23:23.600941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:23:23.600941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:23:23.600941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:23:23.600941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:23:23.601474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:23:23.601474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:23:23.601474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:23.601474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:23:23.601474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:23.601474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:23:23.601474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:23.601474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:23:23.601474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:23.601474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:23:23.601474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:23.601474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:23:23.601474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:23:23.601474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:23.601474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:23:23.601474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:23.601474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:23:23.601474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:23:23.601474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:23.602148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:23:23.602148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:23.602148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:23:28.412090 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:23:28.412639 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:23:28.413318 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:23:28.414203 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:23:28.447265 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:23:28.447512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:28.450516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:23:28.450516 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:28.450516 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:23:28.450516 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:23:28.450516 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:28.450516 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:23:28.450516 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:28.450516 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:23:28.453304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:28.453304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:23:28.453304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:23:28.453304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:23:28.453304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:23:28.453304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:23:28.453304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:23:28.453304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:28.453304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:23:28.453304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:28.453304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:23:28.453304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:28.453688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:23:28.453688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:28.453688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:23:28.453688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:28.453688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:23:28.453688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:23:28.453688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:28.453688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:23:28.453688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:28.453688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:23:28.453688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:23:28.453688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:28.453688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:23:28.454099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:28.454099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:23:28.454099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:28.454099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:23:28.454099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:23:28.454099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:23:28.454099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:23:28.454099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:23:28.454099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:23:28.454099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:23:28.454099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:23:28.454099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:23:28.454099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:23:28.454099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:23:28.454099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:23:28.454448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:23:28.454448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:23:28.454448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:23:28.454448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:23:28.454448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:23:28.454448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:28.454448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:23:28.454448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:28.454448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:23:28.454448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:28.454448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:23:28.454448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:28.454448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:23:28.454448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:28.454448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:23:28.454448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:23:28.454448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:28.454448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:23:28.454448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:28.454989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:23:28.454989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:23:28.454989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:28.454989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:23:28.454989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:28.454989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:23:28.570419 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:23:28.571531 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:23:28.571531 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:23:28.571896 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:23:28.572087 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:23:28.572245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:28.572967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:23:28.572967 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:28.572967 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:23:28.572967 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:23:28.572967 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:28.572967 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:23:28.572967 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:28.572967 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:23:28.574206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:28.574206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:23:28.574206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:23:28.574206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:23:28.574206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:23:28.574206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:23:28.574206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:23:28.574206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:28.574206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:23:28.574206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:28.574206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:23:28.574206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:28.574630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:23:28.574630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:28.574630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:23:28.574630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:28.574630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:23:28.574630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:23:28.574630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:28.574630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:23:28.574630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:28.574630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:23:28.574630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:23:28.574630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:28.574630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:23:28.575110 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:28.575110 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:23:28.575291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:28.575291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:23:28.575291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:23:28.575291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:23:28.575291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:23:28.575291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:23:28.575291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:23:28.575291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:23:28.575291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:23:28.575291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:23:28.575291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:23:28.575291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:23:28.575291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:23:28.575291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:23:28.575291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:23:28.575291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:23:28.575811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:23:28.575811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:23:28.575811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:28.575811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:23:28.575811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:28.575811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:23:28.575811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:28.575811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:23:28.575811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:28.575811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:23:28.575811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:28.575811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:23:28.575811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:23:28.575811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:28.575811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:23:28.575811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:28.575811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:23:28.575811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:23:28.575811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:28.576351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:23:28.576351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:28.576351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:23:28.581297 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:23:28.581754 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:23:28.581856 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:23:28.582173 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:23:28.582369 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:23:28.582544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:28.583094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:23:28.583094 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:28.583094 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:23:28.583094 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:23:28.583094 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:28.583094 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:23:28.583094 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:28.583094 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:23:28.583701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:28.583701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:23:28.583701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:23:28.583701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:23:28.583701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:28.583701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:23:28.583701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:28.583701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:23:28.583701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:28.583701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:23:28.583701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:28.583701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:23:28.583701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:28.583701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:23:28.583701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:23:28.584243 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:28.584243 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:23:28.584243 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:28.584243 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:23:28.584243 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:23:28.584243 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:28.584243 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:23:28.584243 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:28.584243 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:23:28.584574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:28.584574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:23:28.584574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:23:28.584574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:23:28.584574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:23:28.584574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:23:28.584574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:23:28.584574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:23:28.584574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:23:28.584574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:23:28.584574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:28.584574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:23:28.585086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:28.585086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:23:28.585086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:28.585086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:23:28.585086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:28.585086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:23:28.585086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:28.585086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:23:28.585086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:23:28.585086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:28.585086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:23:28.585086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:28.585086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:23:28.585086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:23:28.585479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:28.585479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:23:28.585479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:28.585479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:23:28.585911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:28.585911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:23:28.585911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:23:28.585911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:23:28.585911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:23:28.585911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:23:28.585911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:23:28.585911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:23:28.585911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:23:28.585911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:23:28.585911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:23:28.585911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:23:28.585911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:23:28.585911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:23:28.585911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:23:28.585911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:23:28.586992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:23:28.586992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:23:28.586992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:23:28.586992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:23:28.586992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:23:28.586992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:28.586992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:23:28.586992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:28.586992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:23:28.586992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:28.586992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:23:28.586992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:28.586992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:23:28.586992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:28.586992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:23:28.586992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:23:28.586992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:28.586992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:23:28.587664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:28.587664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:23:28.587664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:23:28.587664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:28.587664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:23:28.587664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:28.587664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:23:28.590149 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:23:28.590149 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:23:28.590149 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:23:28.590149 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:23:28.590149 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:23:28.590429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:28.590861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:23:28.590861 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:28.590861 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:23:28.590861 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:23:28.590861 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:28.590861 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:23:28.590861 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:28.590861 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:23:28.591701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:28.591701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:23:28.591701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:23:28.591701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:23:28.591701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:28.591701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:23:28.591701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:28.591701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:23:28.591701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:28.591701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:23:28.591701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:28.591701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:23:28.591701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:28.591701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:23:28.591701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:23:28.592198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:28.592198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:23:28.592198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:28.592198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:23:28.592198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:23:28.592198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:28.592198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:23:28.592198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:28.592198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:23:28.592767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:28.592767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:23:28.592767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:23:28.592767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:23:28.592767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:23:28.592767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:23:28.592767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:23:28.592767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:23:28.592767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:23:28.592767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:28.592767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:23:28.593079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:28.593079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:23:28.593079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:28.593079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:23:28.593079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:28.593079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:23:28.593079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:28.593079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:23:28.593079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:23:28.593079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:28.593079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:23:28.593079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:28.593079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:23:28.593079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:23:28.593349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:28.593349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:23:28.593349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:28.593349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:23:28.594242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:28.594242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:23:28.594242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:23:28.594242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:23:28.594242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:23:28.594242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:23:28.594242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:23:28.594242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:23:28.594242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:23:28.594242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:23:28.594242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:23:28.594242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:23:28.594242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:23:28.594242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:23:28.594242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:23:28.594242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:23:28.594242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:23:28.594643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:23:28.594643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:23:28.594643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:23:28.594643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:28.594643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:23:28.594643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:28.594643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:23:28.594643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:28.594643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:23:28.594643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:28.594643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:23:28.594643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:28.594643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:23:28.594643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:23:28.594643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:28.594643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:23:28.594643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:28.594643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:23:28.594643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:23:28.595149 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:28.595149 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:23:28.595149 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:28.595149 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:23:28.597118 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:23:28.597433 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:23:28.597548 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:23:28.597722 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:23:28.598106 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:23:28.598323 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:23:28.598476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:28.599055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:23:28.599055 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:28.599055 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:23:28.599055 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:23:28.599055 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:28.599055 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:23:28.599055 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:28.599055 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:23:28.600266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:28.600266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:23:28.600266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:23:28.600266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:23:28.600266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:23:28.600266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:23:28.600266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:23:28.600266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:28.600266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:23:28.600266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:28.600266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:23:28.600266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:28.600944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:23:28.600944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:28.600944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:23:28.600944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:28.600944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:23:28.600944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:23:28.600944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:28.600944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:23:28.600944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:28.600944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:23:28.600944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:23:28.600944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:28.600944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:23:28.601320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:28.601320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:23:28.601411 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:28.601411 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:23:28.601411 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:23:28.601411 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:23:28.601411 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:23:28.601411 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:23:28.601411 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:23:28.601411 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:23:28.601411 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:23:28.601411 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:23:28.601411 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:23:28.601411 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:23:28.601411 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:23:28.601411 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:23:28.601411 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:23:28.601411 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:23:28.602460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:23:28.602460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:23:28.602460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:28.602460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:23:28.602460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:28.602460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:23:28.602460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:28.602460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:23:28.602460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:28.602460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:23:28.602460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:28.602460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:23:28.602460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:23:28.602460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:28.602460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:23:28.602460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:28.602460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:23:28.602460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:23:28.602460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:28.603071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:23:28.603071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:28.603071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:23:33.412703 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:23:33.413468 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:23:33.413539 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:23:33.414344 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:23:33.449500 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:23:33.449788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:33.450562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:23:33.450562 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:33.450562 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:23:33.450562 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:23:33.450562 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:33.450562 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:23:33.450562 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:33.450562 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:23:33.451998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:33.451998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:23:33.451998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:23:33.451998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:23:33.451998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:23:33.451998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:23:33.451998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:23:33.451998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:33.451998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:23:33.451998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:33.451998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:23:33.451998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:33.452429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:23:33.452429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:33.452429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:23:33.452429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:33.452429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:23:33.452429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:23:33.452429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:33.452429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:23:33.452429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:33.452429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:23:33.452429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:23:33.452429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:33.452429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:23:33.453029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:33.453029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:23:33.453437 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:33.453437 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:23:33.453437 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:23:33.453437 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:23:33.453437 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:23:33.453437 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:23:33.453437 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:23:33.453437 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:23:33.453437 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:23:33.453437 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:23:33.453437 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:23:33.453437 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:23:33.453437 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:23:33.453437 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:23:33.453437 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:23:33.453437 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:23:33.453957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:23:33.453957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:23:33.453957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:33.453957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:23:33.453957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:33.453957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:23:33.453957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:33.453957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:23:33.453957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:33.453957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:23:33.453957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:33.453957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:23:33.453957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:23:33.453957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:33.453957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:23:33.453957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:33.453957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:23:33.453957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:23:33.453957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:33.454595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:23:33.454595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:33.454595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:23:33.570959 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:23:33.571466 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:23:33.571582 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:23:33.572817 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:23:33.573150 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:23:33.573379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:33.574152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:23:33.574152 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:33.574152 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:23:33.574152 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:23:33.574152 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:33.574152 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:23:33.574152 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:33.574152 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:23:33.575018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:33.575018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:23:33.575018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:23:33.575018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:23:33.575018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:23:33.575018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:23:33.575018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:23:33.575018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:33.575018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:23:33.575018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:33.575018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:23:33.575018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:33.575360 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:23:33.575360 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:33.575360 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:23:33.575360 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:33.575360 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:23:33.575360 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:23:33.575360 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:33.575360 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:23:33.575360 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:33.575360 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:23:33.575360 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:23:33.575360 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:33.575360 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:23:33.575644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:33.575644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:23:33.575802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:33.575802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:23:33.575802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:23:33.575802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:23:33.575802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:23:33.575802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:23:33.575802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:23:33.575802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:23:33.575802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:23:33.575802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:23:33.575802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:23:33.575802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:23:33.575802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:23:33.575802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:23:33.575802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:23:33.575802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:23:33.576226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:23:33.576226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:23:33.576226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:33.576226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:23:33.576226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:33.576226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:23:33.576226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:33.576226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:23:33.576226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:33.576226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:23:33.576226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:33.576226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:23:33.576226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:23:33.576226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:33.576226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:23:33.576226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:33.576226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:23:33.576226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:23:33.576226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:33.576669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:23:33.576669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:33.576669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:23:33.581820 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:23:33.582119 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:23:33.582204 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:23:33.582348 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:23:33.583012 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:23:33.583291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:33.584043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:23:33.584043 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:33.584043 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:23:33.584043 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:23:33.584043 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:33.584043 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:23:33.584043 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:33.584043 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:23:33.584612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:33.584612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:23:33.584612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:23:33.584612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:23:33.584612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:33.584612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:23:33.584612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:33.584612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:23:33.584612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:33.584612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:23:33.584612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:33.584612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:23:33.584612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:33.584612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:23:33.584612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:23:33.584927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:33.584927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:23:33.584927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:33.584927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:23:33.584927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:23:33.584927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:33.584927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:23:33.584927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:33.584927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:23:33.585224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:33.585224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:23:33.585224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:23:33.585224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:23:33.585224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:23:33.585224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:23:33.585224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:23:33.585224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:23:33.585224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:23:33.585224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:23:33.585224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:33.585224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:23:33.585454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:33.585454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:23:33.585454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:33.585454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:23:33.585454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:33.585454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:23:33.585454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:33.585454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:23:33.585454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:23:33.585454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:33.585454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:23:33.585454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:33.585454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:23:33.585454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:23:33.585828 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:33.585828 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:23:33.585828 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:33.585828 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:23:33.585967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:33.585967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:23:33.585967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:23:33.585967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:23:33.585967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:23:33.585967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:23:33.585967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:23:33.585967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:23:33.585967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:23:33.585967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:23:33.585967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:23:33.585967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:23:33.585967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:23:33.585967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:23:33.585967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:23:33.585967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:23:33.586293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:23:33.586293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:23:33.586293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:23:33.586293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:23:33.586293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:23:33.586293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:33.586293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:23:33.586293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:33.586293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:23:33.586293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:33.586293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:23:33.586293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:33.586293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:23:33.586293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:33.586293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:23:33.586293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:23:33.586293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:33.586293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:23:33.586657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:33.586657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:23:33.586657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:23:33.586657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:33.586657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:23:33.586657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:33.586657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:23:33.589405 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:23:33.589827 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:23:33.590005 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:23:33.590374 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:23:33.591109 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:23:33.591482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:33.592309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:23:33.592309 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:33.592309 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:23:33.592309 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:23:33.592309 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:33.592309 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:23:33.592309 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:33.592309 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:23:33.593399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:33.593399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:23:33.593399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:23:33.593399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:23:33.593399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:33.593399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:23:33.593399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:33.593399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:23:33.593399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:33.593399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:23:33.593399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:33.593399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:23:33.593399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:33.593399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:23:33.593399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:23:33.593778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:33.593778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:23:33.593778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:33.593778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:23:33.593778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:23:33.593778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:33.593778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:23:33.593778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:33.593778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:23:33.594619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:33.594619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:23:33.594619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:23:33.594619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:23:33.594619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:23:33.594619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:23:33.594619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:23:33.594619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:23:33.594619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:23:33.594619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:33.594619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:23:33.595005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:33.595005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:23:33.595005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:33.595005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:23:33.595005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:33.595005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:23:33.595005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:33.595005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:23:33.595005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:23:33.595005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:33.595005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:23:33.595005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:33.595005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:23:33.595005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:23:33.595586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:33.595586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:23:33.595586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:33.595586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:23:33.596169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:33.596169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:23:33.596169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:23:33.596169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:23:33.596169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:23:33.596169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:23:33.596169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:23:33.596169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:23:33.596169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:23:33.596169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:23:33.596169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:23:33.596169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:23:33.596169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:23:33.596169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:23:33.596169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:23:33.596169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:23:33.596169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:23:33.596993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:23:33.596993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:23:33.596993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:23:33.596993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:33.596993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:23:33.596993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:33.596993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:23:33.596993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:33.596993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:23:33.596993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:33.596993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:23:33.596993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:33.596993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:23:33.596993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:23:33.596993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:33.596993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:23:33.596993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:33.596993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:23:33.596993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:23:33.597809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:33.597809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:23:33.597809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:33.597809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:23:33.597809 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:23:33.598034 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:23:33.598034 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:23:33.598300 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:23:33.598964 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:23:33.599165 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:23:33.599288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:33.599950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:23:33.599950 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:33.599950 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:23:33.599950 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:23:33.599950 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:33.599950 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:23:33.599950 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:33.599950 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:23:33.600807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:33.600807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:23:33.600807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:23:33.600807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:23:33.600807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:23:33.600807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:23:33.600807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:23:33.600807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:33.600807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:23:33.600807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:33.600807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:23:33.600807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:33.601156 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:23:33.601156 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:33.601156 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:23:33.601156 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:33.601156 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:23:33.601156 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:23:33.601156 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:33.601156 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:23:33.601156 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:33.601156 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:23:33.601156 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:23:33.601156 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:33.601156 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:23:33.601532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:33.601532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:23:33.601870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:33.601870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:23:33.601870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:23:33.601870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:23:33.601870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:23:33.601870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:23:33.601870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:23:33.601870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:23:33.601870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:23:33.601870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:23:33.601870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:23:33.601870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:23:33.601870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:23:33.601870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:23:33.601870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:23:33.601870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:23:33.602342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:23:33.602342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:23:33.602342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:33.602342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:23:33.602342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:33.602342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:23:33.602342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:33.602342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:23:33.602342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:33.602342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:23:33.602342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:33.602342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:23:33.602342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:23:33.602342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:33.602342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:23:33.602342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:33.602342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:23:33.602342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:23:33.602342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:33.602896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:23:33.602896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:33.602896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:23:38.413661 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:23:38.416844 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:23:38.416844 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:23:38.416844 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:23:38.461402 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:23:38.461635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:38.464789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:23:38.464789 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:38.464789 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:23:38.464789 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:23:38.464789 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:38.464789 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:23:38.464789 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:38.464789 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:23:38.464789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.047 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:38.464789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:23:38.464789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:23:38.464789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:23:38.464789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:23:38.464789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:23:38.464789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:23:38.465568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:38.465568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:23:38.465568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:38.465568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:23:38.465568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:38.465568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:23:38.465568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:38.465568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:23:38.465568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:38.465568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:23:38.465568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:23:38.465568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:38.465568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:23:38.465568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:38.466808 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:23:38.466808 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:23:38.466808 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:38.466808 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:23:38.466808 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:38.466808 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:23:38.466808 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:38.466808 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:23:38.466808 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:23:38.466808 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:23:38.466808 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:23:38.466808 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:23:38.466808 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:23:38.466808 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:23:38.467639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:23:38.467639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:23:38.467639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:23:38.467639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:23:38.467639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:23:38.467639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:23:38.467639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:23:38.467639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:23:38.467639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:23:38.467639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:23:38.467639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:38.467639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:23:38.467639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:38.467639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:23:38.467639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:38.467639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:23:38.467639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:38.467639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:23:38.467639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:38.467639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:23:38.467639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:23:38.468519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:38.468519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:23:38.468519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:38.468519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:23:38.468519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:23:38.468519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:38.468519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:23:38.468519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:38.468519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:23:38.572648 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:23:38.574895 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:23:38.574895 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:23:38.575261 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:23:38.575443 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:23:38.575578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:38.579836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:23:38.579836 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:38.579836 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:23:38.579836 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:23:38.579836 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:38.579836 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:23:38.579836 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:38.579836 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:23:38.579836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:38.579836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:23:38.579836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:23:38.579836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:23:38.579836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:23:38.579836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:23:38.579836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:23:38.580262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:38.580262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:23:38.580262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:38.580262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:23:38.580262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:38.580262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:23:38.580262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:38.580262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:23:38.580262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:38.580262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:23:38.580262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:23:38.580262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:38.580262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:23:38.580262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:38.580687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:23:38.580687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:23:38.580687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:38.580687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:23:38.580687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:38.580687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:23:38.580687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:38.580687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:23:38.580687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:23:38.580687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:23:38.580687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:23:38.580687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:23:38.580687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:23:38.580687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:23:38.581174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:23:38.581174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:23:38.581174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:23:38.581174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:23:38.581174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:23:38.581174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:23:38.581174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:23:38.581174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:23:38.581174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:23:38.581174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:23:38.581174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:38.581174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:23:38.581174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:38.581174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:23:38.581174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:38.581174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:23:38.581174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:38.581174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:23:38.581174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:38.581174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:23:38.581813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:23:38.581813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:38.581813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:23:38.581813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:38.581813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:23:38.581813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:23:38.581813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:38.581813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:23:38.581813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:38.581813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:23:38.582375 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:23:38.583069 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:23:38.583069 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:23:38.583069 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:23:38.583415 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:23:38.583618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:38.585169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:23:38.585169 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:38.585169 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:23:38.585169 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:23:38.585169 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:38.585169 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:23:38.585169 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:38.585169 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:23:38.585169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:38.585169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:23:38.585169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:23:38.585169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:23:38.585169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:38.585169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:23:38.585169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:38.585169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:23:38.585169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:38.585169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:23:38.586448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:38.586448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:23:38.586448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:38.586448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:23:38.586448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:23:38.586448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:38.586448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:23:38.586448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:38.586448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:23:38.586448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:23:38.586448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:38.586448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:23:38.586448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:38.586448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:23:38.586882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:38.586882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:23:38.586882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:23:38.586882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:23:38.586882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:23:38.586882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:23:38.586882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:23:38.586882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:23:38.586882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:23:38.586882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:23:38.586882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:38.586882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:23:38.587143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:38.587143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:23:38.587143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:38.587143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:23:38.587143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:38.587143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:23:38.587143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:38.587143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:23:38.587143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:23:38.587143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:38.587143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:23:38.587143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:38.587143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:23:38.587143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:23:38.587406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:38.587406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:23:38.587406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:38.587406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:23:38.588323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:38.588323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:23:38.588323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:23:38.588323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:23:38.588323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:23:38.588323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:23:38.588323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:23:38.588323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:23:38.588323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:23:38.588323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:23:38.588323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:23:38.588323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:23:38.588323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:23:38.588323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:23:38.588323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:23:38.588323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:23:38.588657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:23:38.588657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:23:38.588657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:23:38.588657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:23:38.588657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:23:38.588657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:38.588657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:23:38.588657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:38.588657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:23:38.588657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:38.588657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:23:38.588657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:38.588657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:23:38.588657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:38.588657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:23:38.588657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:23:38.588657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:38.588657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:23:38.589036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:38.589036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:23:38.589036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:23:38.589036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:38.589036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:23:38.589036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:38.589036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:23:38.589866 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:23:38.590389 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:23:38.590389 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:23:38.590389 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:23:38.590643 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:23:38.590908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:38.593382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:23:38.593382 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:38.593382 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:23:38.593382 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:23:38.593382 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:38.593382 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:23:38.593382 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:38.593382 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:23:38.593382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:38.593382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:23:38.593382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:23:38.593382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:23:38.593382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:38.593382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:23:38.593382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:38.593382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:23:38.593382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:38.593382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:23:38.593766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:38.593766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:23:38.593766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:38.593766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:23:38.593766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:23:38.593766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:38.593766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:23:38.593766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:38.593766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:23:38.593766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:23:38.593766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:38.593766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:23:38.593766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:38.593766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:23:38.593766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:38.594248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:23:38.594248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:23:38.594248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:23:38.594248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:23:38.594248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:23:38.594248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:23:38.594248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:23:38.594248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:23:38.594248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:38.594248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:23:38.594248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:38.594248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:23:38.594248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:38.594611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:23:38.594611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:38.594611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:23:38.594611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:38.594611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:23:38.594611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:23:38.594611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:38.594611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:23:38.594611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:38.594611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:23:38.594611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:23:38.594611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:38.594611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:23:38.595086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:38.595086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:23:38.595086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:38.595086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:23:38.595086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:23:38.595086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:23:38.595086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:23:38.595086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:23:38.595086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:23:38.595086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:23:38.595086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:23:38.595086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:23:38.595086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:23:38.595086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:23:38.595086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:23:38.595492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:23:38.595492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:23:38.595492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:23:38.595492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:23:38.595492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:23:38.595492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:23:38.595492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:23:38.595492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:38.595492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:23:38.595492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:38.595492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:23:38.595492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:38.595492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:23:38.595492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:38.595492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:23:38.595492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:38.595492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:23:38.595492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:23:38.595492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:38.595979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:23:38.595979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:38.595979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:23:38.595979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:23:38.595979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:38.595979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:23:38.595979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:38.595979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:23:38.598029 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:23:38.598691 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:23:38.598691 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:23:38.598691 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:23:38.599062 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:23:38.599379 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:23:38.599588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:38.602045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:23:38.602045 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:38.602045 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:23:38.602045 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:23:38.602045 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:38.602045 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:23:38.602045 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:38.602045 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:23:38.602045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:38.602045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:23:38.602045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:23:38.602045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:23:38.602045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:23:38.602045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:23:38.602045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:23:38.602552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:38.602552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:23:38.602552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:38.602552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:23:38.602552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:38.602552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:23:38.602552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:38.602552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:23:38.602552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:38.602552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:23:38.602552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:23:38.602552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:38.602552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:23:38.602552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:38.602920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:23:38.602920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:23:38.602920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:38.602920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:23:38.602920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:38.602920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:23:38.602920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:38.602920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:23:38.602920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:23:38.602920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:23:38.602920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:23:38.602920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:23:38.602920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:23:38.602920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:23:38.603183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:23:38.603183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:23:38.603183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:23:38.603183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:23:38.603183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:23:38.603183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:23:38.603183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:23:38.603183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:23:38.603183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:23:38.603183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:23:38.603183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:38.603183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:23:38.603183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:38.603183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:23:38.603183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:38.603183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:23:38.603183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:38.603183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:23:38.603183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:38.603183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:23:38.603622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:23:38.603622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:38.603622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:23:38.603622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:38.603622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:23:38.603622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:23:38.603622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:38.603622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:23:38.603622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:38.603622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:23:43.415053 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:23:43.418785 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:23:43.418785 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:23:43.418785 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:23:43.459767 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:23:43.460143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:43.460985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:23:43.460985 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:43.460985 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:23:43.460985 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:23:43.460985 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:43.460985 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:23:43.460985 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:43.460985 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:23:43.462247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.044 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:43.462247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:23:43.462247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:23:43.462247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:23:43.462247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:23:43.462247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:23:43.462247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:23:43.462247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:43.462247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:23:43.462247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:43.462247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:23:43.462247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:43.462945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:23:43.462945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:43.462945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:23:43.462945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:43.462945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:23:43.462945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:23:43.462945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:43.462945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:23:43.462945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:43.462945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:23:43.462945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:23:43.462945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:43.462945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:23:43.463691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:43.463691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:23:43.463691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:43.463691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:23:43.463691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:23:43.463691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:23:43.463691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:23:43.463691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:23:43.463691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:23:43.463691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:23:43.463691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:23:43.463691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:23:43.463691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:23:43.463691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:23:43.463691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:23:43.464515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:23:43.464515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:23:43.464515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:23:43.464515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:23:43.464515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:23:43.464515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:43.464515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:23:43.464515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:43.464515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:23:43.464515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:43.464515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:23:43.464515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:43.464515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:23:43.464515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:43.464515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:23:43.464515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:23:43.464515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:43.464515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:23:43.464515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:43.465492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:23:43.465492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:23:43.465492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:43.465492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:23:43.465492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:43.465492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:23:43.573413 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:23:43.573773 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:23:43.573925 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:23:43.575055 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:23:43.575484 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:23:43.575853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:43.576705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:23:43.576705 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:43.576705 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:23:43.576705 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:23:43.576705 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:43.576705 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:23:43.576705 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:43.576705 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:23:43.577976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:43.577976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:23:43.577976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:23:43.577976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:23:43.577976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:23:43.577976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:23:43.577976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:23:43.577976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:43.577976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:23:43.577976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:43.577976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:23:43.577976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:43.578356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:23:43.578356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:43.578356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:23:43.578356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:43.578356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:23:43.578356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:23:43.578356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:43.578356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:23:43.578356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:43.578356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:23:43.578356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:23:43.578356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:43.578356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:23:43.578718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:43.578718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:23:43.579098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:43.579098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:23:43.579098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:23:43.579098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:23:43.579098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:23:43.579098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:23:43.579098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:23:43.579098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:23:43.579098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:23:43.579098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:23:43.579098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:23:43.579098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:23:43.579098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:23:43.579098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:23:43.579098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:23:43.579098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:23:43.579800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:23:43.579800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:23:43.579800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:43.579800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:23:43.579800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:43.579800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:23:43.579800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:43.579800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:23:43.579800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:43.579800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:23:43.579800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:43.579800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:23:43.579800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:23:43.579800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:43.579800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:23:43.579800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:43.579800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:23:43.579800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:23:43.579800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:43.580875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:23:43.580875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:43.580875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:23:43.582778 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:23:43.583297 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:23:43.583297 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:23:43.583297 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:23:43.583832 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:23:43.584170 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:43.584912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:23:43.584912 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:43.584912 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:23:43.584912 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:23:43.584912 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:43.584912 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:23:43.584912 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:43.584912 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:23:43.585806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:43.585806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:23:43.585806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:23:43.585806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:23:43.585806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:43.585806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:23:43.585806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:43.585806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:23:43.585806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:43.585806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:23:43.585806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:43.585806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:23:43.585806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:43.585806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:23:43.585806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:23:43.586488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:43.586488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:23:43.586488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:43.586488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:23:43.586488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:23:43.586488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:43.586488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:23:43.586488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:43.586488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:23:43.586938 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:43.586938 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:23:43.586938 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:23:43.586938 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:23:43.586938 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:23:43.586938 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:23:43.586938 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:23:43.586938 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:23:43.586938 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:23:43.586938 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:23:43.586938 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:43.586938 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:23:43.587511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:43.587511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:23:43.587511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:43.587511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:23:43.587511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:43.587511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:23:43.587511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:43.587511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:23:43.587511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:23:43.587511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:43.587511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:23:43.587511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:43.587511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:23:43.587511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:23:43.588173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:43.588173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:23:43.588173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:43.588173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:23:43.588173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:43.588173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:23:43.588173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:23:43.588173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:23:43.588173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:23:43.588173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:23:43.588173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:23:43.588173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:23:43.588173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:23:43.588173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:23:43.589024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:23:43.589024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:23:43.589024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:23:43.589024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:23:43.589024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:23:43.589024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:23:43.589024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:23:43.589024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:23:43.589024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:23:43.589024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:23:43.589024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:23:43.589024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:43.589024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:23:43.589024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:43.589024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:23:43.589024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:43.589024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:23:43.589024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:43.589024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:23:43.590301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:43.590301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:23:43.590301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:23:43.590301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:43.590301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:23:43.590301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:43.590301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:23:43.590301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:23:43.590301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:43.590301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:23:43.590301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:43.590301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:23:43.590857 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:23:43.591313 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:23:43.591452 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:23:43.591598 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:23:43.592122 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:23:43.592510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:43.593227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:23:43.593227 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:43.593227 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:23:43.593227 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:23:43.593227 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:43.593227 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:23:43.593227 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:43.593227 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:23:43.594048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:43.594048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:23:43.594048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:23:43.594048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:23:43.594048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:43.594048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:23:43.594048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:43.594048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:23:43.594048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:43.594048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:23:43.594048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:43.594048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:23:43.594048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:43.594048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:23:43.594048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:23:43.594565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:43.594565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:23:43.594565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:43.594565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:23:43.594565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:23:43.594565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:43.594565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:23:43.594565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:43.594565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:23:43.595022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:43.595022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:23:43.595022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:23:43.595022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:23:43.595022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:23:43.595022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:23:43.595022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:23:43.595022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:23:43.595022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:23:43.595022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:43.595022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:23:43.595380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:43.595380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:23:43.595380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:43.595380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:23:43.595380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:43.595380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:23:43.595380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:43.595380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:23:43.595380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:23:43.595380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:43.595380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:23:43.595380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:43.595380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:23:43.595380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:23:43.595740 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:43.595740 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:23:43.595740 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:43.595740 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:23:43.596077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:43.596077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:23:43.596077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:23:43.596077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:23:43.596077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:23:43.596077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:23:43.596077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:23:43.596077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:23:43.596077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:23:43.596077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:23:43.596077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:23:43.596077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:23:43.596077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:23:43.596077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:23:43.596077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:23:43.596077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:23:43.596077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:23:43.596547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:23:43.596547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:23:43.596547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:23:43.596547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:43.596547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:23:43.596547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:43.596547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:23:43.596547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:43.596547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:23:43.596547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:43.596547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:23:43.596547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:43.596547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:23:43.596547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:23:43.596547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:43.596547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:23:43.596547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:43.596547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:23:43.596547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:23:43.597041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:43.597041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:23:43.597041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:43.597041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:23:43.598657 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:23:43.599165 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:23:43.599282 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:23:43.599441 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:23:43.600136 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:23:43.600505 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:23:43.600766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:43.601504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:23:43.601504 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:43.601504 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:23:43.601504 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:23:43.601504 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:43.601504 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:23:43.601504 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:43.601504 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:23:43.602784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:43.602784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:23:43.602784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:23:43.602784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:23:43.602784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:23:43.602784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:23:43.602784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:23:43.602784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:43.602784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:23:43.602784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:43.602784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:23:43.602784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:43.603224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:23:43.603224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:43.603224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:23:43.603224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:43.603224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:23:43.603224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:23:43.603224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:43.603224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:23:43.603224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:43.603224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:23:43.603224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:23:43.603224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:43.603224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:23:43.603654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:43.603654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:23:43.603774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:43.603774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:23:43.603774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:23:43.603774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:23:43.603774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:23:43.603774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:23:43.603774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:23:43.603774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:23:43.603774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:23:43.603774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:23:43.603774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:23:43.603774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:23:43.603774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:23:43.603774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:23:43.603774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:23:43.603774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:23:43.604359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:23:43.604359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:23:43.604359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:43.604359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:23:43.604359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:43.604359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:23:43.604359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:43.604359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:23:43.604359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:43.604359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:23:43.604359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:43.604359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:23:43.604359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:23:43.604359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:43.604359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:23:43.604359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:43.604359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:23:43.604359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:23:43.604359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:43.604869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:23:43.604869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:43.604869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:23:48.416525 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:23:48.417461 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:23:48.417461 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:23:48.419139 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:23:48.462939 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:23:48.462939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:48.463653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:23:48.463653 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:48.463653 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:23:48.463653 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:23:48.463653 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:48.463653 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:23:48.463653 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:48.463653 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:23:48.464802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.046 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:48.464802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:23:48.464802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:23:48.464802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:23:48.464802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:23:48.464802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:23:48.464802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:23:48.464802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:48.464802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:23:48.464802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:48.464802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:23:48.464802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:48.465365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:23:48.465365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:48.465365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:23:48.465365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:48.465365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:23:48.465365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:23:48.465365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:48.465365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:23:48.465365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:48.465365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:23:48.465365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:23:48.465365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:48.465365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:23:48.465950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:48.465950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:23:48.466334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:48.466334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:23:48.466334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:23:48.466334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:23:48.466334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:23:48.466334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:23:48.466334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:23:48.466334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:23:48.466334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:23:48.466334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:23:48.466334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:23:48.466334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:23:48.466334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:23:48.466334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:23:48.466334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:23:48.466334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:23:48.467046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:23:48.467046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:23:48.467046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:48.467046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:23:48.467046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:48.467046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:23:48.467046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:48.467046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:23:48.467046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:48.467046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:23:48.467046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:48.467046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:23:48.467046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:23:48.467046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:48.467046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:23:48.467046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:48.467046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:23:48.467046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:23:48.467046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:48.467862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:23:48.467862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:48.467862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:23:48.574267 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:23:48.574818 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:23:48.575021 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:23:48.575895 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:23:48.576273 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:23:48.576567 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:48.577499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:23:48.577499 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:48.577499 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:23:48.577499 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:23:48.577499 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:48.577499 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:23:48.577499 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:48.577499 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:23:48.578729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:48.578729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:23:48.578729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:23:48.578729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:23:48.578729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:23:48.578729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:23:48.578729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:23:48.578729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:48.578729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:23:48.578729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:48.578729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:23:48.578729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:48.579305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:23:48.579305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:48.579305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:23:48.579305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:48.579305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:23:48.579305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:23:48.579305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:48.579305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:23:48.579305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:48.579305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:23:48.579305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:23:48.579305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:48.579305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:23:48.579674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:48.579674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:23:48.580261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:48.580261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:23:48.580261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:23:48.580261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:23:48.580261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:23:48.580261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:23:48.580261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:23:48.580261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:23:48.580261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:23:48.580261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:23:48.580261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:23:48.580261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:23:48.580261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:23:48.580261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:23:48.580261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:23:48.580261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:23:48.580865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:23:48.580865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:23:48.580865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:48.580865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:23:48.580865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:48.580865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:23:48.580865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:48.580865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:23:48.580865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:48.580865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:23:48.580865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:48.580865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:23:48.580865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:23:48.580865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:48.580865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:23:48.580865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:48.580865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:23:48.580865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:23:48.580865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:48.581376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:23:48.581376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:48.581376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:23:48.583336 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:23:48.583701 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:23:48.583895 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:23:48.583948 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:23:48.584282 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:23:48.584474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:48.584928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:23:48.584928 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:48.584928 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:23:48.584928 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:23:48.584928 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:48.584928 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:23:48.584928 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:48.584928 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:23:48.585643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:48.585643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:23:48.585643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:23:48.585643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:23:48.585643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:48.585643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:23:48.585643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:48.585643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:23:48.585643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:48.585643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:23:48.585643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:48.585643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:23:48.585643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:48.585643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:23:48.585643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:23:48.586161 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:48.586161 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:23:48.586161 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:48.586161 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:23:48.586161 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:23:48.586161 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:48.586161 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:23:48.586161 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:48.586161 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:23:48.586505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:48.586505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:23:48.586505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:23:48.586505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:23:48.586505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:23:48.586505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:23:48.586505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:23:48.586505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:23:48.586505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:23:48.586505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:23:48.586505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:48.586505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:23:48.586920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:48.586920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:23:48.586920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:48.586920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:23:48.586920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:48.586920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:23:48.586920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:48.586920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:23:48.586920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:23:48.586920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:48.586920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:23:48.586920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:48.586920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:23:48.586920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:23:48.587276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:48.587276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:23:48.587276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:48.587276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:23:48.587565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:48.587565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:23:48.587565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:23:48.587565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:23:48.587565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:23:48.587565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:23:48.587565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:23:48.587565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:23:48.587565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:23:48.587565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:23:48.587565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:23:48.587565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:23:48.587565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:23:48.587565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:23:48.587565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:23:48.587565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:23:48.588070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:23:48.588070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:23:48.588070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:23:48.588070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:23:48.588070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:23:48.588070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:48.588070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:23:48.588070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:48.588070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:23:48.588070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:48.588070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:23:48.588070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:48.588070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:23:48.588070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:48.588070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:23:48.588070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:23:48.588070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:48.588070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:23:48.588580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:48.588580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:23:48.588580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:23:48.588580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:48.588580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:23:48.588580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:48.588580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:23:48.591299 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:23:48.591584 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:23:48.591871 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:23:48.591935 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:23:48.592259 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:23:48.592481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:48.592938 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:23:48.592938 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:48.592938 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:23:48.592938 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:23:48.592938 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:48.592938 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:23:48.592938 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:48.592938 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:23:48.593660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:48.593660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:23:48.593660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:23:48.593660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:23:48.593660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:48.593660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:23:48.593660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:48.593660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:23:48.593660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:48.593660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:23:48.593660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:48.593660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:23:48.593660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:48.593660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:23:48.593660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:23:48.594266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:48.594266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:23:48.594266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:48.594266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:23:48.594266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:23:48.594266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:48.594266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:23:48.594266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:48.594266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:23:48.594568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:48.594568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:23:48.594568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:23:48.594568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:23:48.594568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:23:48.594568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:23:48.594568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:23:48.594568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:23:48.594568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:23:48.594568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:48.594568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:23:48.595060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:48.595060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:23:48.595060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:48.595060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:23:48.595060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:48.595060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:23:48.595060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:48.595060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:23:48.595060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:23:48.595060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:48.595060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:23:48.595060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:48.595060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:23:48.595060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:23:48.595423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:48.595423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:23:48.595423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:48.595423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:23:48.595700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:48.595700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:23:48.595700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:23:48.595700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:23:48.595700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:23:48.595700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:23:48.595700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:23:48.595700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:23:48.595700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:23:48.595700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:23:48.595700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:23:48.595700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:23:48.595700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:23:48.595700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:23:48.595700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:23:48.595700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:23:48.595700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:23:48.596221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:23:48.596221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:23:48.596221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:23:48.596221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:48.596221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:23:48.596221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:48.596221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:23:48.596221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:48.596221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:23:48.596221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:48.596221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:23:48.596221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:48.596221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:23:48.596221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:23:48.596221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:48.596221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:23:48.596221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:48.596221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:23:48.596221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:23:48.596729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:48.596729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:23:48.596729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:48.596729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:23:48.599291 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:23:48.599537 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:23:48.599652 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:23:48.599878 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:23:48.600313 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:23:48.600706 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:23:48.601006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:48.601691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:23:48.601691 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:48.601691 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:23:48.601691 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:23:48.601691 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:48.601691 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:23:48.601691 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:48.601691 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:23:48.602711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:48.602711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:23:48.602711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:23:48.602711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:23:48.602711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:23:48.602711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:23:48.602711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:23:48.602711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:48.602711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:23:48.602711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:48.602711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:23:48.602711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:48.603650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:23:48.603650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:48.603650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:23:48.603650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:48.603650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:23:48.603650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:23:48.603650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:48.603650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:23:48.603650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:48.603650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:23:48.603650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:23:48.603650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:48.603650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:23:48.604648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:48.604648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:23:48.604648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:48.604648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:23:48.604648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:23:48.604648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:23:48.604648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:23:48.604648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:23:48.604648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:23:48.604648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:23:48.604648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:23:48.604648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:23:48.604648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:23:48.604648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:23:48.604648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:23:48.605573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:23:48.605573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:23:48.605573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:23:48.605573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:23:48.605573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:23:48.605573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:48.605573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:23:48.605573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:48.605573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:23:48.605573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:48.605573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:23:48.605573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:48.605573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:23:48.605573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:48.605573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:23:48.605573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:23:48.605573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:48.605573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:23:48.605573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:48.606499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:23:48.606499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:23:48.606499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:48.606499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:23:48.606499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:48.606499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:23:53.417205 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:23:53.418234 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:23:53.418234 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:23:53.418840 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:23:53.459375 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:23:53.459992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:53.460305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:23:53.460305 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:53.460305 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:23:53.460305 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:23:53.460305 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:53.460305 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:23:53.460305 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:53.460305 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:23:53.461276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:53.461276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:23:53.461276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:23:53.461276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:23:53.461276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:23:53.461276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:23:53.461276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:23:53.461276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:53.461276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:23:53.461276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:53.461276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:23:53.461276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:53.462281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:23:53.462281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:53.462281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:23:53.462281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:53.462281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:23:53.462281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:23:53.462281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:53.462281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:23:53.462281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:53.462281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:23:53.462281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:23:53.462281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:53.462281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:23:53.463440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:53.463440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:23:53.463440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:53.463440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:23:53.463440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:23:53.463440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:23:53.463440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:23:53.463440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:23:53.463440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:23:53.463440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:23:53.463440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:23:53.463440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:23:53.463440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:23:53.463440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:23:53.463440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:23:53.465261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:23:53.465261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:23:53.465261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:23:53.465261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:23:53.465261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:23:53.465261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:53.465261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:23:53.465261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:53.465261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:23:53.465261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:53.465261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:23:53.465261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:53.465261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:23:53.465261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:53.465261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:23:53.465261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:23:53.465261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:53.465261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:23:53.465261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:53.466638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:23:53.466638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:23:53.466638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:53.466638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:23:53.466638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:53.466638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:23:53.574916 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:23:53.575641 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:23:53.576180 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:23:53.577317 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:23:53.577716 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:23:53.578054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:53.579166 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:23:53.579166 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:53.579166 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:23:53.579166 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:23:53.579166 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:53.579166 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:23:53.579166 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:53.579166 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:23:53.581025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:53.581025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:23:53.581025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:23:53.581025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:23:53.581025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:23:53.581025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:23:53.581025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:23:53.581025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:53.581025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:23:53.581025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:53.581025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:23:53.581025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:53.581511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:23:53.581511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:53.581511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:23:53.581511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:53.581511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:23:53.581511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:23:53.581511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:53.581511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:23:53.581511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:53.581511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:23:53.581511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:23:53.581511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:53.581511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:23:53.581958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:53.581958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:23:53.583325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:53.583325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:23:53.583325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:23:53.583325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:23:53.583325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:23:53.583325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:23:53.583325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:23:53.583325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:23:53.583325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:23:53.583325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:23:53.583325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:23:53.583325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:23:53.583325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:23:53.583325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:23:53.583325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:23:53.583325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:23:53.583950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:23:53.583950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:23:53.583950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:53.583950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:23:53.583950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:53.583950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:23:53.583950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:53.583950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:23:53.583950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:53.583950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:23:53.583950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:53.583950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:23:53.583950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:23:53.583950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:53.583950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:23:53.583950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:53.583950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:23:53.583950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:23:53.583950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:53.584681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:23:53.584681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:53.584681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:23:53.584681 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:23:53.584681 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:23:53.584681 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:23:53.584958 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:23:53.585541 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:23:53.585837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:53.586857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:23:53.586857 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:53.586857 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:23:53.586857 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:23:53.586857 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:53.586857 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:23:53.586857 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:53.586857 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:23:53.587552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:53.587552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:23:53.587552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:23:53.587552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:23:53.587552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:53.587552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:23:53.587552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:53.587552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:23:53.587552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:53.587552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:23:53.587552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:53.587552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:23:53.587552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:53.587552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:23:53.587552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:23:53.587984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:53.587984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:23:53.587984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:53.587984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:23:53.587984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:23:53.587984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:53.587984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:23:53.587984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:53.587984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:23:53.588387 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:53.588387 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:23:53.588387 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:23:53.588387 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:23:53.588387 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:23:53.588387 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:23:53.588387 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:23:53.588387 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:23:53.588387 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:23:53.588387 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:23:53.588387 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:53.588387 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:23:53.588808 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:53.588808 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:23:53.588808 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:53.588808 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:23:53.588808 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:53.588808 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:23:53.588808 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:53.588808 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:23:53.588808 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:23:53.588808 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:53.588808 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:23:53.588808 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:53.588808 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:23:53.588808 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:23:53.589119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:53.589119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:23:53.589119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:53.589119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:23:53.589496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:53.589496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:23:53.589496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:23:53.589496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:23:53.589496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:23:53.589496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:23:53.589496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:23:53.589496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:23:53.589496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:23:53.589496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:23:53.589496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:23:53.589496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:23:53.589496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:23:53.589496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:23:53.589496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:23:53.589496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:23:53.590139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:23:53.590139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:23:53.590139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:23:53.590139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:23:53.590139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:23:53.590139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:53.590139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:23:53.590139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:53.590139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:23:53.590139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:53.590139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:23:53.590139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:53.590139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:23:53.590139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:53.590139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:23:53.590139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:23:53.590139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:53.590139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:23:53.590993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:53.590993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:23:53.590993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:23:53.590993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:53.590993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:23:53.590993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:53.590993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:23:53.591673 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:23:53.591949 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:23:53.592118 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:23:53.592268 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:23:53.592930 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:23:53.593333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:53.594226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:23:53.594226 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:53.594226 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:23:53.594226 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:23:53.594226 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:53.594226 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:23:53.594226 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:53.594226 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:23:53.595083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:53.595083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:23:53.595083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:23:53.595083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:23:53.595083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:53.595083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:23:53.595083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:53.595083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:23:53.595083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:53.595083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:23:53.595083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:53.595083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:23:53.595083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:53.595083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:23:53.595083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:23:53.595525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:53.595525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:23:53.595525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:53.595525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:23:53.595525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:23:53.595525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:53.595525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:23:53.595525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:53.595525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:23:53.596059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:53.596059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:23:53.596059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:23:53.596059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:23:53.596059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:23:53.596059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:23:53.596059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:23:53.596059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:23:53.596059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:23:53.596059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:53.596059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:23:53.596425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:53.596425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:23:53.596425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:53.596425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:23:53.596425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:53.596425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:23:53.596425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:53.596425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:23:53.596425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:23:53.596425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:53.596425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:23:53.596425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:53.596425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:23:53.596425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:23:53.596843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:53.596843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:23:53.596843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:53.596843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:23:53.597150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:53.597150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:23:53.597150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:23:53.597150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:23:53.597150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:23:53.597150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:23:53.597150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:23:53.597150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:23:53.597150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:23:53.597150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:23:53.597150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:23:53.597150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:23:53.597150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:23:53.597150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:23:53.597150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:23:53.597150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:23:53.597150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:23:53.597677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:23:53.597677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:23:53.597677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:23:53.597677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:53.597677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:23:53.597677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:53.597677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:23:53.597677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:53.597677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:23:53.597677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:53.597677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:23:53.597677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:53.597677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:23:53.597677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:23:53.597677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:53.597677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:23:53.597677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:53.597677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:23:53.597677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:23:53.598399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:53.598399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:23:53.598399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:53.598399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:23:53.599657 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:23:53.599959 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:23:53.600073 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:23:53.600264 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:23:53.600890 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:23:53.601288 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:23:53.601528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:53.602357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:23:53.602357 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:53.602357 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:23:53.602357 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:23:53.602357 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:53.602357 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:23:53.602357 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:53.602357 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:23:53.603391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:53.603391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:23:53.603391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:23:53.603391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:23:53.603391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:23:53.603391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:23:53.603391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:23:53.603391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:53.603391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:23:53.603391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:53.603391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:23:53.603391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:53.603927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:23:53.603927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:53.603927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:23:53.603927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:53.603927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:23:53.603927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:23:53.603927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:53.603927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:23:53.603927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:53.603927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:23:53.603927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:23:53.603927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:53.603927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:23:53.604352 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:53.604352 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:23:53.604954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:53.604954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:23:53.604954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:23:53.604954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:23:53.604954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:23:53.604954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:23:53.604954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:23:53.604954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:23:53.604954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:23:53.604954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:23:53.604954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:23:53.604954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:23:53.604954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:23:53.604954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:23:53.604954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:23:53.604954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:23:53.605586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:23:53.605586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:23:53.605586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:53.605586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:23:53.605586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:53.605586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:23:53.605586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:53.605586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:23:53.605586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:53.605586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:23:53.605586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:53.605586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:23:53.605586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:23:53.605586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:53.605586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:23:53.605586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:53.605586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:23:53.605586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:23:53.605586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:53.606206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:23:53.606206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:53.606206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:23:58.417735 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:23:58.418542 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:23:58.418542 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:23:58.419436 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:23:58.425854 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG dbcounter [-] [106314] Writing DB stats neutron:SELECT=241 {{(pid=106314) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 09 09:23:58.456944 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:23:58.457272 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:58.458146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:23:58.458146 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:58.458146 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:23:58.458146 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:23:58.458146 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:58.458146 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:23:58.458146 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:58.458146 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:23:58.459183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:58.459183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:23:58.459183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:23:58.459183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:23:58.459183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:23:58.459183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:23:58.459183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:23:58.459183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:58.459183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:23:58.459183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:58.459183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:23:58.459183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:58.460256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:23:58.460256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:58.460256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:23:58.460256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:58.460256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:23:58.460256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:23:58.460256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:58.460256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:23:58.460256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:58.460256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:23:58.460256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:23:58.460256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:58.460256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:23:58.460974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:58.460974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:23:58.460974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:58.460974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:23:58.460974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:23:58.460974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:23:58.460974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:23:58.460974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:23:58.460974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:23:58.460974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:23:58.460974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:23:58.460974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:23:58.460974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:23:58.460974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:23:58.460974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:23:58.461624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:23:58.461624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:23:58.461624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:23:58.461624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:23:58.461624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:23:58.461624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:58.461624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:23:58.461624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:58.461624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:23:58.461624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:58.461624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:23:58.461624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:58.461624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:23:58.461624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:58.461624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:23:58.461624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:23:58.461624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:58.461624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:23:58.461624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:58.462444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:23:58.462444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:23:58.462444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:58.462444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:23:58.462444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:58.462444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:23:58.575572 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:23:58.576036 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:23:58.576202 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:23:58.577496 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:23:58.577889 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:23:58.578116 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:58.578973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:23:58.578973 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:58.578973 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:23:58.578973 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:23:58.578973 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:58.578973 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:23:58.578973 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:58.578973 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:23:58.580041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:58.580041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:23:58.580041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:23:58.580041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:23:58.580041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:23:58.580041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:23:58.580041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:23:58.580041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:58.580041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:23:58.580041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:58.580041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:23:58.580041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:58.580786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:23:58.580786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:58.580786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:23:58.580786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:58.580786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:23:58.580786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:23:58.580786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:58.580786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:23:58.580786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:58.580786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:23:58.580786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:23:58.580786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:58.580786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:23:58.581599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:58.581599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:23:58.581599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:58.581599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:23:58.581599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:23:58.581599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:23:58.581599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:23:58.581599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:23:58.581599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:23:58.581599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:23:58.581599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:23:58.581599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:23:58.581599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:23:58.581599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:23:58.581599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:23:58.582375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:23:58.582375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:23:58.582375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:23:58.582375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:23:58.582375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:23:58.582375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:58.582375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:23:58.582375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:58.582375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:23:58.582375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:58.582375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:23:58.582375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:58.582375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:23:58.582375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:58.582375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:23:58.582375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:23:58.582375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:58.582375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:23:58.582375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:58.583294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:23:58.583294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:23:58.583294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:58.583294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:23:58.583294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:58.583294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:23:58.584642 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:23:58.585141 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:23:58.585494 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:23:58.585494 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:23:58.586253 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:23:58.587862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:58.590077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:23:58.590077 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:58.590077 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:23:58.590077 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:23:58.590077 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:58.590077 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:23:58.590077 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:58.590077 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:23:58.591003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:58.591003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:23:58.591003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:23:58.591003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:23:58.591003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:58.591003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:23:58.591003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:58.591003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:23:58.591003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:58.591003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:23:58.591003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:58.591003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:23:58.591003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:58.591003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:23:58.591003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:23:58.591403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:58.591403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:23:58.591403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:58.591403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:23:58.591403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:23:58.591403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:58.591403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:23:58.591403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:58.591403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:23:58.592152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.006 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:58.592152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:23:58.592152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:23:58.592152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:23:58.592152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:23:58.592152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:23:58.592152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:23:58.592152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:23:58.592152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:23:58.592152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:23:58.592152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:58.592152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:23:58.592544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:58.592544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:23:58.592544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:58.592544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:23:58.592544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:58.592544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:23:58.592544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:58.592544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:23:58.592544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:23:58.592544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:58.592544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:23:58.592544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:58.592544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:23:58.592544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:23:58.592943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:58.592943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:23:58.592943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:58.592943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:23:58.593917 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:23:58.594711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:58.594711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:23:58.594711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:23:58.594711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:23:58.594711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:23:58.594711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:23:58.594711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:23:58.594711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:23:58.594711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:23:58.594711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:23:58.594711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:23:58.594711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:23:58.594711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:23:58.594711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:23:58.594711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:23:58.594711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:23:58.595168 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:23:58.595168 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:23:58.595168 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:23:58.595168 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:23:58.595168 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:23:58.595168 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:58.595168 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:23:58.595168 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:58.595168 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:23:58.595168 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:58.595168 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:23:58.595168 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:58.595168 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:23:58.595168 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:58.595168 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:23:58.595168 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:23:58.595168 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:58.595168 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:23:58.595658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:58.595658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:23:58.595658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:23:58.595658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:58.595658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:23:58.595658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:58.595658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:23:58.595658 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:23:58.595658 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:23:58.595658 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:23:58.595997 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:23:58.595997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:58.596635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:23:58.596635 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:58.596635 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:23:58.596635 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:23:58.596635 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:58.596635 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:23:58.596635 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:58.596635 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:23:58.597304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:58.597304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:23:58.597304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:23:58.597304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:23:58.597304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:58.597304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:23:58.597304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:58.597304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:23:58.597304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:58.597304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:23:58.597304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:58.597304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:23:58.597304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:58.597304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:23:58.597304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:23:58.598131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:58.598131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:23:58.598131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:58.598131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:23:58.598131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:23:58.598131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:58.598131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:23:58.598131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:58.598131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:23:58.598131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:58.598131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:23:58.598131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:23:58.598131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:23:58.598131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:23:58.598515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:23:58.598515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:23:58.598515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:23:58.598515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:23:58.598515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:58.598515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:23:58.598515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:58.598515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:23:58.598515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:58.598515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:23:58.598515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:58.598515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:23:58.598515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:58.598881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:23:58.598881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:23:58.598881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:58.598881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:23:58.598881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:58.598881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:23:58.598881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:23:58.598881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:58.598881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:23:58.598881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:58.598881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:23:58.599363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:58.599363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:23:58.599363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:23:58.599363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:23:58.599363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:23:58.599363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:23:58.599363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:23:58.599363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:23:58.599363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:23:58.599363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:23:58.599363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:23:58.599363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:23:58.599363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:23:58.599363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:23:58.599363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:23:58.599363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:23:58.599363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:23:58.600023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:23:58.600023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:23:58.600023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:23:58.600023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:58.600023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:23:58.600023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:58.600023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:23:58.600023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:58.600023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:23:58.600023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:58.600023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:23:58.600023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:58.600023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:23:58.600023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:23:58.600023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:58.600023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:23:58.600023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:58.600023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:23:58.600023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:23:58.600584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:58.600584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:23:58.600584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:58.600584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:23:58.600584 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:23:58.603061 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:23:58.603176 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:23:58.603396 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:23:58.603841 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:23:58.604141 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:23:58.604611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:58.605223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:23:58.605223 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:58.605223 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:23:58.605223 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:23:58.605223 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:58.605223 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:23:58.605223 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:58.605223 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:23:58.605955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:58.605955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:23:58.605955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:23:58.605955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:23:58.605955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:23:58.605955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:23:58.605955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:23:58.605955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:58.605955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:23:58.605955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:58.605955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:23:58.605955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:58.606322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:23:58.606322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:58.606322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:23:58.606322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:58.606322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:23:58.606322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:23:58.606322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:58.606322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:23:58.606322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:58.606322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:23:58.606322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:23:58.606322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:58.606322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:23:58.606765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:58.606765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:23:58.606883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:58.606883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:23:58.606883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:23:58.606883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:23:58.606883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:23:58.606883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:23:58.606883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:23:58.606883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:23:58.606883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:23:58.606883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:23:58.606883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:23:58.606883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:23:58.606883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:23:58.606883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:23:58.606883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:23:58.606883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:23:58.607705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:23:58.607705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:23:58.607705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:58.607705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:23:58.607705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:23:58.607705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:23:58.607705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:23:58.607705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:23:58.607705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:23:58.607705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:23:58.607705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:23:58.607705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:23:58.607705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:23:58.607705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:23:58.607705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:23:58.607705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:23:58.607705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:23:58.607705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:23:58.607705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:23:58.608365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:23:58.608365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:23:58.608365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:24:03.418419 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:24:03.419173 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:24:03.419937 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:24:03.421219 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:24:03.461717 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:24:03.461958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:03.463981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:24:03.463981 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:03.463981 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:24:03.463981 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:24:03.463981 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:03.463981 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:24:03.463981 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:03.463981 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:24:03.463981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.043 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:03.463981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:24:03.463981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:24:03.463981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:24:03.463981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:24:03.463981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:24:03.463981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:24:03.465010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:03.465010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:24:03.465010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:03.465010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:24:03.465010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:03.465010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:24:03.465010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:03.465010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:24:03.465010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:03.465010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:24:03.465010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:24:03.465010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:03.465010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:24:03.465010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:03.465548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:24:03.465548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:24:03.465548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:03.465548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:24:03.465548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:03.465548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:24:03.465548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:03.465548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:24:03.465548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:24:03.465548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:24:03.465548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:24:03.465548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:24:03.465548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:24:03.465548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:24:03.466181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:24:03.466181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:24:03.466181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:24:03.466181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:24:03.466181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:24:03.466181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:24:03.466181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:24:03.466181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:24:03.466181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:24:03.466181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:24:03.466181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:03.466181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:24:03.466181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:03.466181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:24:03.466181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:03.466181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:24:03.466181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:03.466181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:24:03.466181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:03.466181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:24:03.466181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:24:03.466966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:03.466966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:24:03.466966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:03.466966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:24:03.466966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:24:03.466966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:03.466966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:24:03.466966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:03.466966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:24:03.576214 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:24:03.576968 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:24:03.577185 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:24:03.578552 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:24:03.578927 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:24:03.579185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:03.580430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:24:03.580430 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:03.580430 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:24:03.580430 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:24:03.580430 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:03.580430 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:24:03.580430 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:03.580430 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:24:03.582795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:03.582795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:24:03.582795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:24:03.582795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:24:03.582795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:24:03.582795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:24:03.582795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:24:03.582795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:03.582795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:24:03.582795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:03.582795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:24:03.582795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:03.583403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:24:03.583403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:03.583403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:24:03.583403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:03.583403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:24:03.583403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:24:03.583403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:03.583403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:24:03.583403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:03.583403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:24:03.583403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:24:03.583403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:03.583403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:24:03.584210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:03.584210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:24:03.585384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:03.585384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:24:03.585384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:24:03.585384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:24:03.585384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:24:03.585384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:24:03.585384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:24:03.585384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:24:03.585384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:24:03.585384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:24:03.585384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:24:03.585384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:24:03.585384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:24:03.585384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:24:03.585384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:24:03.585384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:24:03.586474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:24:03.586474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:24:03.586474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:03.586474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:24:03.586474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:03.586474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:24:03.586474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:03.586474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:24:03.586474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:03.586474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:24:03.586474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:03.586474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:24:03.586474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:24:03.586474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:03.586474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:24:03.586474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:03.586474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:24:03.586474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:24:03.586474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:03.587336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:24:03.587336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:03.587336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:24:03.587336 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:24:03.587336 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:24:03.587336 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:24:03.587336 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:24:03.588967 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:24:03.588967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:03.588967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:24:03.588967 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:03.588967 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:24:03.588967 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:24:03.588967 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:03.588967 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:24:03.588967 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:03.588967 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:24:03.589632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:03.589632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:24:03.589632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:24:03.589632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:24:03.589632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:03.589632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:24:03.589632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:03.589632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:24:03.589632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:03.589632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:24:03.589632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:03.589632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:24:03.589632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:03.589632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:24:03.589632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:24:03.590833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:03.590833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:24:03.590833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:03.590833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:24:03.590833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:24:03.590833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:03.590833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:24:03.590833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:03.590833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:24:03.590833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:03.590833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:24:03.590833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:24:03.590833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:24:03.590833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:24:03.591674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:24:03.591674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:24:03.591674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:24:03.591674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:24:03.591674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:24:03.591674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:03.591674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:24:03.591674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:03.591674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:24:03.591674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:03.591674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:24:03.591674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:03.591674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:24:03.592559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:03.592559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:24:03.592559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:24:03.592559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:03.592559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:24:03.592559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:03.592559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:24:03.592559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:24:03.592559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:03.592559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:24:03.592559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:03.592559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:24:03.593298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:03.593298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:24:03.593298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:24:03.593298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:24:03.593298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:24:03.593298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:24:03.593298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:24:03.593298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:24:03.593298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:24:03.593298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:24:03.593298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:24:03.593298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:24:03.593298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:24:03.593298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:24:03.593298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:24:03.593298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:24:03.594265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:24:03.594265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:24:03.594265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:24:03.594265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:24:03.594265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:24:03.594265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:03.594265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:24:03.594265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:03.594265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:24:03.594265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:03.594265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:24:03.594265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:03.594265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:24:03.594265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:03.594265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:24:03.594265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:24:03.594265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:03.594265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:24:03.595230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:03.595230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:24:03.595230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:24:03.595230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:03.595230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:24:03.595230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:03.595230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:24:03.595230 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:24:03.595688 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:24:03.595688 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:24:03.595688 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:24:03.596063 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:24:03.596291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:03.596797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:24:03.596797 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:03.596797 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:24:03.596797 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:24:03.596797 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:03.596797 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:24:03.596797 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:03.596797 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:24:03.597401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:03.597401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:24:03.597401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:24:03.597401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:24:03.597401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:03.597401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:24:03.597401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:03.597401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:24:03.597401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:03.597401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:24:03.597401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:03.597401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:24:03.597401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:03.597401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:24:03.597401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:24:03.597806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:03.597806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:24:03.597806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:03.597806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:24:03.597806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:24:03.597806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:03.597806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:24:03.597806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:03.597806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:24:03.598313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:03.598313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:24:03.598313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:24:03.598313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:24:03.598313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:24:03.598313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:24:03.598313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:24:03.598313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:24:03.598313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:24:03.598313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:03.598313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:24:03.598766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:03.598766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:24:03.598766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:03.598766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:24:03.598766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:03.598766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:24:03.598766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:03.598766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:24:03.598766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:24:03.598766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:03.598766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:24:03.598766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:03.598766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:24:03.598766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:24:03.599153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:03.599153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:24:03.599153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:03.599153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:24:03.599460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:03.599460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:24:03.599460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:24:03.599460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:24:03.599460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:24:03.599460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:24:03.599460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:24:03.599460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:24:03.599460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:24:03.599460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:24:03.599460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:24:03.599460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:24:03.599460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:24:03.599460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:24:03.599460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:24:03.599460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:24:03.599460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:24:03.600171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:24:03.600171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:24:03.600171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:24:03.600171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:03.600171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:24:03.600171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:03.600171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:24:03.600171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:03.600171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:24:03.600171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:03.600171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:24:03.600171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:03.600171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:24:03.600171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:24:03.600171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:03.600171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:24:03.600171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:03.600171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:24:03.600171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:24:03.600891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:03.600891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:24:03.600891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:03.600891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:24:03.603245 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:24:03.603485 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:24:03.603599 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:24:03.603842 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:24:03.604331 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:24:03.604720 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:24:03.604970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:03.605665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:24:03.605665 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:03.605665 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:24:03.605665 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:24:03.605665 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:03.605665 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:24:03.605665 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:03.605665 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:24:03.606514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:03.606514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:24:03.606514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:24:03.606514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:24:03.606514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:24:03.606514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:24:03.606514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:24:03.606514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:03.606514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:24:03.606514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:03.606514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:24:03.606514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:03.606879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:24:03.606879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:03.606879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:24:03.606879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:03.606879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:24:03.606879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:24:03.606879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:03.606879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:24:03.606879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:03.606879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:24:03.606879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:24:03.606879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:03.606879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:24:03.607230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:03.607230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:24:03.607527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:03.607527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:24:03.607527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:24:03.607527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:24:03.607527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:24:03.607527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:24:03.607527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:24:03.607527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:24:03.607527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:24:03.607527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:24:03.607527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:24:03.607527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:24:03.607527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:24:03.607527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:24:03.607527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:24:03.607527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:24:03.607993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:24:03.607993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:24:03.607993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:03.607993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:24:03.607993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:03.607993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:24:03.607993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:03.607993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:24:03.607993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:03.607993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:24:03.607993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:03.607993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:24:03.607993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:24:03.607993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:03.607993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:24:03.607993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:03.607993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:24:03.607993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:24:03.607993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:03.608542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:24:03.608542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:03.608542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:24:08.419172 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:24:08.420098 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:24:08.420098 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:24:08.420623 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:24:08.458251 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:24:08.458617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:08.459363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:24:08.459363 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:08.459363 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:24:08.459363 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:24:08.459363 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:08.459363 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:24:08.459363 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:08.459363 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:24:08.460239 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:08.460239 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:24:08.460239 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:24:08.460239 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:24:08.460239 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:24:08.460239 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:24:08.460239 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:24:08.460239 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:08.460239 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:24:08.460239 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:08.460239 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:24:08.460239 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:08.460692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:24:08.460692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:08.460692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:24:08.460692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:08.460692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:24:08.460692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:24:08.460692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:08.460692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:24:08.460692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:08.460692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:24:08.460692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:24:08.460692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:08.460692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:24:08.461264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:08.461264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:24:08.461264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:08.461264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:24:08.461264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:24:08.461264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:24:08.461264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:24:08.461264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:24:08.461264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:24:08.461264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:24:08.461264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:24:08.461264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:24:08.461264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:24:08.461264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:24:08.461264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:24:08.461777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:24:08.461777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:24:08.461777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:24:08.461777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:24:08.461777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:24:08.461777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:08.461777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:24:08.461777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:08.461777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:24:08.461777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:08.461777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:24:08.461777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:08.461777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:24:08.461777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:08.461777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:24:08.461777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:24:08.461777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:08.461777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:24:08.461777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:08.462308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:24:08.462308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:24:08.462308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:08.462308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:24:08.462308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:08.462308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:24:08.576964 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:24:08.577250 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:24:08.577329 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:24:08.578372 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:24:08.578690 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:24:08.578943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:08.579774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:24:08.579774 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:08.579774 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:24:08.579774 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:24:08.579774 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:08.579774 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:24:08.579774 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:08.579774 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:24:08.581212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:08.581212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:24:08.581212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:24:08.581212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:24:08.581212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:24:08.581212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:24:08.581212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:24:08.581212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:08.581212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:24:08.581212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:08.581212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:24:08.581212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:08.581833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:24:08.581833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:08.581833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:24:08.581833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:08.581833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:24:08.581833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:24:08.581833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:08.581833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:24:08.581833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:08.581833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:24:08.581833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:24:08.581833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:08.581833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:24:08.582379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:08.582379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:24:08.582379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:08.582379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:24:08.582379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:24:08.582379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:24:08.582379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:24:08.582379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:24:08.582379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:24:08.582379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:24:08.582379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:24:08.582379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:24:08.582379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:24:08.582379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:24:08.582379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:24:08.582995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:24:08.582995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:24:08.582995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:24:08.582995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:24:08.582995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:24:08.582995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:08.582995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:24:08.582995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:08.582995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:24:08.582995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:08.582995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:24:08.582995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:08.582995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:24:08.582995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:08.582995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:24:08.582995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:24:08.582995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:08.582995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:24:08.582995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:08.583776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:24:08.583776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:24:08.583776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:08.583776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:24:08.583776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:08.583776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:24:08.586321 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:24:08.586583 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:24:08.586687 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:24:08.586879 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:24:08.587736 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:24:08.588018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:08.588569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:24:08.588569 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:08.588569 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:24:08.588569 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:24:08.588569 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:08.588569 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:24:08.588569 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:08.588569 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:24:08.589383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:08.589383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:24:08.589383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:24:08.589383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:24:08.589383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:08.589383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:24:08.589383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:08.589383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:24:08.589383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:08.589383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:24:08.589383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:08.589383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:24:08.589383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:08.589383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:24:08.589383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:24:08.590028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:08.590028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:24:08.590028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:08.590028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:24:08.590028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:24:08.590028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:08.590028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:24:08.590028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:08.590028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:24:08.590404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:08.590404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:24:08.590404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:24:08.590404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:24:08.590404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:24:08.590404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:24:08.590404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:24:08.590404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:24:08.590404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:24:08.590404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:24:08.590404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:08.590404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:24:08.591171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:08.591171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:24:08.591171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:08.591171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:24:08.591171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:08.591171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:24:08.591171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:08.591171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:24:08.591171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:24:08.591171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:08.591171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:24:08.591171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:08.591171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:24:08.591171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:24:08.591799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:08.591799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:24:08.591799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:08.591799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:24:08.591799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:08.591799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:24:08.591799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:24:08.591799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:24:08.591799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:24:08.591799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:24:08.591799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:24:08.591799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:24:08.591799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:24:08.591799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:24:08.592408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:24:08.592408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:24:08.592408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:24:08.592408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:24:08.592408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:24:08.592408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:24:08.592408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:24:08.592408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:24:08.592408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:24:08.592408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:24:08.592408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:24:08.592408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:08.592408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:24:08.592408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:08.592408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:24:08.592408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:08.592408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:24:08.592408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:08.592408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:24:08.593214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:08.593214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:24:08.593214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:24:08.593214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:08.593214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:24:08.593214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:08.593214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:24:08.593214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:24:08.593214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:08.593214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:24:08.593214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:08.593214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:24:08.595530 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:24:08.596052 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:24:08.596159 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:24:08.596300 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:24:08.596770 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:24:08.596999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:08.597514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:24:08.597514 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:08.597514 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:24:08.597514 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:24:08.597514 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:08.597514 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:24:08.597514 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:08.597514 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:24:08.598320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:08.598320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:24:08.598320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:24:08.598320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:24:08.598320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:08.598320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:24:08.598320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:08.598320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:24:08.598320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:08.598320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:24:08.598320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:08.598320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:24:08.598320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:08.598320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:24:08.598320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:24:08.598988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:08.598988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:24:08.598988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:08.598988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:24:08.598988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:24:08.598988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:08.598988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:24:08.598988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:08.598988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:24:08.599367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:08.599367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:24:08.599367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:24:08.599367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:24:08.599367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:24:08.599367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:24:08.599367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:24:08.599367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:24:08.599367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:24:08.599367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:08.599367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:24:08.599876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:08.599876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:24:08.599876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:08.599876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:24:08.599876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:08.599876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:24:08.599876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:08.599876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:24:08.599876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:24:08.599876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:08.599876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:24:08.599876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:08.599876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:24:08.599876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:24:08.600448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:08.600448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:24:08.600448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:08.600448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:24:08.600448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:08.600448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:24:08.600448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:24:08.600448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:24:08.600448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:24:08.600448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:24:08.600448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:24:08.600448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:24:08.600448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:24:08.600448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:24:08.601086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:24:08.601086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:24:08.601086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:24:08.601086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:24:08.601086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:24:08.601086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:24:08.601086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:24:08.601086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:24:08.601086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:24:08.601086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:24:08.601086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:08.601086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:24:08.601086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:08.601086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:24:08.601086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:08.601086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:24:08.601086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:08.601086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:24:08.601086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:08.601908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:24:08.601908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:24:08.601908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:08.601908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:24:08.601908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:08.601908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:24:08.601908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:24:08.601908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:08.601908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:24:08.601908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:08.601908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:24:08.603540 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:24:08.603818 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:24:08.603915 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:24:08.604082 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:24:08.607923 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:24:08.607923 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:24:08.607923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:08.607923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:24:08.607923 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:08.607923 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:24:08.607923 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:24:08.607923 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:08.607923 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:24:08.607923 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:08.607923 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:24:08.608493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:08.608493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:24:08.608493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:24:08.608493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:24:08.608493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:24:08.608493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:24:08.608493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:24:08.608493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:08.608493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:24:08.608493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:08.608493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:24:08.608493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:08.609118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:24:08.609118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:08.609118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:24:08.609118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:08.609118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:24:08.609118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:24:08.609118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:08.609118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:24:08.609118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:08.609118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:24:08.609118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:24:08.609118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:08.609118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:24:08.609645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:08.609645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:24:08.609645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:08.609645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:24:08.609645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:24:08.609645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:24:08.609645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:24:08.609645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:24:08.609645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:24:08.609645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:24:08.609645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:24:08.609645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:24:08.609645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:24:08.609645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:24:08.609645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:24:08.610260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:24:08.610260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:24:08.610260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:24:08.610260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:24:08.610260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:24:08.610260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:08.610260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:24:08.610260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:08.610260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:24:08.610260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:08.610260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:24:08.610260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:08.610260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:24:08.610260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:08.610260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:24:08.610260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:24:08.610260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:08.610260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:24:08.610260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:08.611052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:24:08.611052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:24:08.611052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:08.611052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:24:08.611052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:08.611052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:24:13.419658 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:24:13.420350 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:24:13.420350 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:24:13.422802 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:24:13.470688 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:24:13.471132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:13.472121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:24:13.472121 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:13.472121 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:24:13.472121 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:24:13.472121 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:13.472121 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:24:13.472121 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:13.472121 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:24:13.473211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.052 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:13.473211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:24:13.473211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:24:13.473211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:24:13.473211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:24:13.473211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:24:13.473211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:24:13.473211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:13.473211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:24:13.473211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:13.473211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:24:13.473211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:13.473555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:24:13.473555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:13.473555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:24:13.473555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:13.473555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:24:13.473555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:24:13.473555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:13.473555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:24:13.473555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:13.473555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:24:13.473555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:24:13.473555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:13.473555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:24:13.473856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:13.473856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:24:13.474185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:13.474185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:24:13.474185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:24:13.474185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:24:13.474185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:24:13.474185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:24:13.474185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:24:13.474185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:24:13.474185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:24:13.474185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:24:13.474185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:24:13.474185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:24:13.474185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:24:13.474185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:24:13.474185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:24:13.474185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:24:13.474564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:24:13.474564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:24:13.474564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:13.474564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:24:13.474564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:13.474564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:24:13.474564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:13.474564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:24:13.474564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:13.474564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:24:13.474564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:13.474564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:24:13.474564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:24:13.474564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:13.474564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:24:13.474564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:13.474564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:24:13.474564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:24:13.474564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:13.474972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:24:13.474972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:13.474972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:24:13.577403 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:24:13.578441 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:24:13.578441 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:24:13.578784 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:24:13.578937 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:24:13.579059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:13.579666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:24:13.579666 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:13.579666 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:24:13.579666 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:24:13.579666 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:13.579666 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:24:13.579666 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:13.579666 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:24:13.580908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:13.580908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:24:13.580908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:24:13.580908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:24:13.580908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:24:13.580908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:24:13.580908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:24:13.580908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:13.580908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:24:13.580908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:13.580908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:24:13.580908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:13.581724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:24:13.581724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:13.581724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:24:13.581724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:13.581724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:24:13.581724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:24:13.581724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:13.581724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:24:13.581724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:13.581724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:24:13.581724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:24:13.581724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:13.581724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:24:13.582433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:13.582433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:24:13.582433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:13.582433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:24:13.582433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:24:13.582433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:24:13.582433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:24:13.582433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:24:13.582433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:24:13.582433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:24:13.582433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:24:13.582433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:24:13.582433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:24:13.582433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:24:13.582433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:24:13.583235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:24:13.583235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:24:13.583235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:24:13.583235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:24:13.583235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:24:13.583235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:13.583235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:24:13.583235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:13.583235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:24:13.583235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:13.583235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:24:13.583235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:13.583235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:24:13.583235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:13.583235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:24:13.583235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:24:13.583235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:13.583235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:24:13.583235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:13.585243 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:24:13.585243 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:24:13.585243 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:13.585243 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:24:13.585243 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:13.585243 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:24:13.586881 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:24:13.586881 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:24:13.586992 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:24:13.587093 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:24:13.587565 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:24:13.588075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:13.588787 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:24:13.588787 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:13.588787 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:24:13.588787 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:24:13.588787 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:13.588787 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:24:13.588787 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:13.588787 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:24:13.589606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:13.589606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:24:13.589606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:24:13.589606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:24:13.589606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:13.589606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:24:13.589606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:13.589606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:24:13.589606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:13.589606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:24:13.589606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:13.589606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:24:13.589606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:13.589606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:24:13.589606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:24:13.590131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:13.590131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:24:13.590131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:13.590131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:24:13.590131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:24:13.590131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:13.590131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:24:13.590131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:13.590131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:24:13.590404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:13.590404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:24:13.590404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:24:13.590404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:24:13.590404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:24:13.590404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:24:13.590404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:24:13.590404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:24:13.590404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:24:13.590404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:24:13.590404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:13.590404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:24:13.590760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:13.590760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:24:13.590760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:13.590760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:24:13.590760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:13.590760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:24:13.590760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:13.590760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:24:13.590760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:24:13.590760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:13.590760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:24:13.590760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:13.590760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:24:13.590760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:24:13.591127 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:13.591127 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:24:13.591127 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:13.591127 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:24:13.591242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:13.591242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:24:13.591242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:24:13.591242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:24:13.591242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:24:13.591242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:24:13.591242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:24:13.591242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:24:13.591242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:24:13.591242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:24:13.591242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:24:13.591242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:24:13.591242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:24:13.591242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:24:13.591242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:24:13.591242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:24:13.591658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:24:13.591658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:24:13.591658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:24:13.591658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:24:13.591658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:24:13.591658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:13.591658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:24:13.591658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:13.591658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:24:13.591658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:13.591658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:24:13.591658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:13.591658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:24:13.591658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:13.591658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:24:13.591658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:24:13.591658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:13.591658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:24:13.592213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:13.592213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:24:13.592213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:24:13.592213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:13.592213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:24:13.592213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:13.592213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:24:13.596199 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:24:13.596395 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:24:13.596508 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:24:13.597389 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:24:13.597389 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:24:13.597509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:13.598234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:24:13.598234 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:13.598234 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:24:13.598234 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:24:13.598234 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:13.598234 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:24:13.598234 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:13.598234 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:24:13.598946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:13.598946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:24:13.598946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:24:13.598946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:24:13.598946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:13.598946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:24:13.598946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:13.598946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:24:13.598946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:13.598946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:24:13.598946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:13.598946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:24:13.598946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:13.598946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:24:13.598946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:24:13.599362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:13.599362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:24:13.599362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:13.599362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:24:13.599362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:24:13.599362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:13.599362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:24:13.599362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:13.599362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:24:13.599805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:13.599805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:24:13.599805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:24:13.599805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:24:13.599805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:24:13.599805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:24:13.599805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:24:13.599805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:24:13.599805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:24:13.599805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:13.599805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:24:13.600145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:13.600145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:24:13.600145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:13.600145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:24:13.600145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:13.600145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:24:13.600145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:13.600145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:24:13.600145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:24:13.600145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:13.600145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:24:13.600145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:13.600145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:24:13.600145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:24:13.600469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:13.600469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:24:13.600469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:13.600469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:24:13.601716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:13.601716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:24:13.601716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:24:13.601716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:24:13.601716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:24:13.601716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:24:13.601716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:24:13.601716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:24:13.601716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:24:13.601716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:24:13.601716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:24:13.601716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:24:13.601716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:24:13.601716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:24:13.601716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:24:13.601716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:24:13.601716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:24:13.602204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:24:13.602204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:24:13.602204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:24:13.602204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:13.602204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:24:13.602204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:13.602204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:24:13.602204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:13.602204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:24:13.602204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:13.602204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:24:13.602204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:13.602204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:24:13.602204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:24:13.602204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:13.602204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:24:13.602204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:13.602204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:24:13.602204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:24:13.602683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:13.602683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:24:13.602683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:13.602683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:24:13.603966 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:24:13.604209 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:24:13.604332 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:24:13.604538 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:24:13.605221 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:24:13.605342 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:24:13.605569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:13.606303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:24:13.606303 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:13.606303 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:24:13.606303 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:24:13.606303 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:13.606303 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:24:13.606303 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:13.606303 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:24:13.607122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:13.607122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:24:13.607122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:24:13.607122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:24:13.607122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:24:13.607122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:24:13.607122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:24:13.607122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:13.607122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:24:13.607122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:13.607122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:24:13.607122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:13.607561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:24:13.607561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:13.607561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:24:13.607561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:13.607561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:24:13.607561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:24:13.607561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:13.607561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:24:13.607561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:13.607561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:24:13.607561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:24:13.607561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:13.607561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:24:13.607919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:13.607919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:24:13.608218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:13.608218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:24:13.608218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:24:13.608218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:24:13.608218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:24:13.608218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:24:13.608218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:24:13.608218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:24:13.608218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:24:13.608218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:24:13.608218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:24:13.608218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:24:13.608218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:24:13.608218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:24:13.608218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:24:13.608218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:24:13.608611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:24:13.608611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:24:13.608611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:13.608611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:24:13.608611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:13.608611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:24:13.608611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:13.608611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:24:13.608611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:13.608611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:24:13.608611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:13.608611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:24:13.608611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:24:13.608611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:13.608611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:24:13.608611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:13.608611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:24:13.608611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:24:13.608611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:13.609531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:24:13.609531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:13.609531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:24:18.421560 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:24:18.422173 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:24:18.422173 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:24:18.422869 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:24:18.452202 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:24:18.452202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:18.453381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:24:18.453381 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:18.453381 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:24:18.453381 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:24:18.453381 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:18.453381 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:24:18.453381 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:18.453381 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:24:18.454144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.031 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:18.454144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:24:18.454144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:24:18.454144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:24:18.454144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:24:18.454144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:24:18.454144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:24:18.454144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:18.454144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:24:18.454144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:18.454144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:24:18.454144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:18.454663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:24:18.454663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:18.454663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:24:18.454663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:18.454663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:24:18.454663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:24:18.454663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:18.454663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:24:18.454663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:18.454663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:24:18.454663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:24:18.454663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:18.454663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:24:18.455189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:18.455189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:24:18.455189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:18.455189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:24:18.455189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:24:18.455189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:24:18.455189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:24:18.455189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:24:18.455189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:24:18.455189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:24:18.455189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:24:18.455189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:24:18.455189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:24:18.455189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:24:18.455189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:24:18.455730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:24:18.455730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:24:18.455730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:24:18.455730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:24:18.455730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:24:18.455730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:18.455730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:24:18.455730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:18.455730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:24:18.455730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:18.455730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:24:18.455730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:18.455730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:24:18.455730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:18.455730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:24:18.455730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:24:18.455730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:18.455730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:24:18.455730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:18.456439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:24:18.456439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:24:18.456439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:18.456439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:24:18.456439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:18.456439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:24:18.578084 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:24:18.578347 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:24:18.578449 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:24:18.579845 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:24:18.579845 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:24:18.579845 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:18.580821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:24:18.580821 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:18.580821 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:24:18.580821 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:24:18.580821 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:18.580821 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:24:18.580821 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:18.580821 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:24:18.581251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:18.581251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:24:18.581251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:24:18.581251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:24:18.581251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:24:18.581251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:24:18.581251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:24:18.581251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:18.581251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:24:18.581251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:18.581251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:24:18.581251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:18.581554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:24:18.581554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:18.581554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:24:18.581554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:18.581554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:24:18.581554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:24:18.581554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:18.581554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:24:18.581554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:18.581554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:24:18.581554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:24:18.581554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:18.581554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:24:18.581918 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:18.581918 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:24:18.583037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:18.583037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:24:18.583037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:24:18.583037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:24:18.583037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:24:18.583037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:24:18.583037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:24:18.583037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:24:18.583037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:24:18.583037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:24:18.583037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:24:18.583037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:24:18.583037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:24:18.583037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:24:18.583037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:24:18.583037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:24:18.583605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:24:18.583605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:24:18.583605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:18.583605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:24:18.583605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:18.583605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:24:18.583605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:18.583605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:24:18.583605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:18.583605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:24:18.583605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:18.583605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:24:18.583605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:24:18.583605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:18.583605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:24:18.583605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:18.583605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:24:18.583605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:24:18.583605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:18.584171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:24:18.584171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:18.584171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:24:18.587074 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:24:18.587474 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:24:18.587732 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:24:18.588022 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:24:18.588957 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:24:18.588957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:18.589216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:24:18.589216 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:18.589216 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:24:18.589216 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:24:18.589216 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:18.589216 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:24:18.589216 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:18.589216 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:24:18.590303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:18.590303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:24:18.590303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:24:18.590303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:24:18.590303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:18.590303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:24:18.590303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:18.590303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:24:18.590303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:18.590303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:24:18.590303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:18.590303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:24:18.590303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:18.590303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:24:18.590303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:24:18.590878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:18.590878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:24:18.590878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:18.590878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:24:18.590878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:24:18.590878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:18.590878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:24:18.590878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:18.590878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:24:18.591463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:18.591463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:24:18.591463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:24:18.591463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:24:18.591463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:24:18.591463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:24:18.591463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:24:18.591463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:24:18.591463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:24:18.591463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:24:18.591463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:18.591463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:24:18.591828 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:18.591828 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:24:18.591828 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:18.591828 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:24:18.591828 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:18.591828 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:24:18.591828 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:18.591828 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:24:18.591828 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:24:18.591828 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:18.591828 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:24:18.591828 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:18.591828 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:24:18.591828 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:24:18.592162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:18.592162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:24:18.592162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:18.592162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:24:18.592956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:18.592956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:24:18.592956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:24:18.592956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:24:18.592956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:24:18.592956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:24:18.592956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:24:18.592956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:24:18.592956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:24:18.592956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:24:18.592956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:24:18.592956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:24:18.592956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:24:18.592956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:24:18.592956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:24:18.592956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:24:18.593423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:24:18.593423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:24:18.593423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:24:18.593423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:24:18.593423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:24:18.593423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:18.593423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:24:18.593423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:18.593423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:24:18.593423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:18.593423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:24:18.593423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:18.593423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:24:18.593423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:18.593423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:24:18.593423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:24:18.593423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:18.593423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:24:18.593946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:18.593946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:24:18.593946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:24:18.593946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:18.593946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:24:18.593946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:18.593946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:24:18.596485 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:24:18.596737 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:24:18.596880 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:24:18.597022 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:24:18.598054 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:24:18.598054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:18.598293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:24:18.598293 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:18.598293 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:24:18.598293 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:24:18.598293 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:18.598293 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:24:18.598293 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:18.598293 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:24:18.599662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:18.599662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:24:18.599662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:24:18.599662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:24:18.599662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:18.599662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:24:18.599662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:18.599662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:24:18.599662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:18.599662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:24:18.599662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:18.599662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:24:18.599662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:18.599662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:24:18.599662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:24:18.600289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:18.600289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:24:18.600289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:18.600289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:24:18.600289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:24:18.600289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:18.600289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:24:18.600289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:18.600289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:24:18.600842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:18.600842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:24:18.600842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:24:18.600842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:24:18.600842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:24:18.600842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:24:18.600842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:24:18.600842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:24:18.600842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:24:18.600842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:18.600842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:24:18.601280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:18.601280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:24:18.601280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:18.601280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:24:18.601280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:18.601280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:24:18.601280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:18.601280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:24:18.601280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:24:18.601280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:18.601280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:24:18.601280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:18.601280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:24:18.601280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:24:18.601806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:18.601806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:24:18.601806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:18.601806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:24:18.602292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:18.602292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:24:18.602292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:24:18.602292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:24:18.602292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:24:18.602292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:24:18.602292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:24:18.602292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:24:18.602292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:24:18.602292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:24:18.602292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:24:18.602292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:24:18.602292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:24:18.602292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:24:18.602292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:24:18.602292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:24:18.602292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:24:18.602799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:24:18.602799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:24:18.602799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:24:18.602799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:18.602799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:24:18.602799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:18.602799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:24:18.602799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:18.602799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:24:18.602799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:18.602799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:24:18.602799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:18.602799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:24:18.602799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:24:18.602799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:18.602799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:24:18.602799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:18.602799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:24:18.602799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:24:18.603425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:18.603425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:24:18.603425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:18.603425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:24:18.604344 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:24:18.604677 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:24:18.604870 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:24:18.605064 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:24:18.605492 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:24:18.605867 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:24:18.606105 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:18.606838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:24:18.606838 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:18.606838 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:24:18.606838 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:24:18.606838 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:18.606838 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:24:18.606838 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:18.606838 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:24:18.608365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:18.608365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:24:18.608365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:24:18.608365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:24:18.608365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:24:18.608365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:24:18.608365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:24:18.608365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:18.608365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:24:18.608365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:18.608365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:24:18.608365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:18.608739 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:24:18.608739 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:18.608739 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:24:18.608739 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:18.608739 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:24:18.608739 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:24:18.608739 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:18.608739 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:24:18.608739 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:18.608739 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:24:18.608739 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:24:18.608739 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:18.608739 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:24:18.609129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:18.609129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:24:18.609792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:18.609792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:24:18.609792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:24:18.609792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:24:18.609792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:24:18.609792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:24:18.609792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:24:18.609792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:24:18.609792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:24:18.609792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:24:18.609792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:24:18.609792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:24:18.609792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:24:18.609792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:24:18.609792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:24:18.609792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:24:18.610349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:24:18.610349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:24:18.610349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:18.610349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:24:18.610349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:18.610349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:24:18.610349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:18.610349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:24:18.610349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:18.610349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:24:18.610349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:18.610349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:24:18.610349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:24:18.610349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:18.610349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:24:18.610349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:18.610349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:24:18.610349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:24:18.610349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:18.611002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:24:18.611002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:18.611002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:24:23.422178 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:24:23.422728 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:24:23.422728 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:24:23.423520 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:24:23.469269 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:24:23.469269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:23.469498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:24:23.469498 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:23.469498 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:24:23.469498 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:24:23.469498 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:23.469498 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:24:23.469498 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:23.469498 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:24:23.470446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.047 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:23.470446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:24:23.470446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:24:23.470446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:24:23.470446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:24:23.470446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:24:23.470446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:24:23.470446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:23.470446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:24:23.470446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:23.470446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:24:23.470446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:23.470726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:24:23.470726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:23.470726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:24:23.470726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:23.470726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:24:23.470726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:24:23.470726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:23.470726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:24:23.470726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:23.470726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:24:23.470726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:24:23.470726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:23.470726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:24:23.471044 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:23.471044 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:24:23.471542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:23.471542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:24:23.471542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:24:23.471542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:24:23.471542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:24:23.471542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:24:23.471542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:24:23.471542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:24:23.471542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:24:23.471542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:24:23.471542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:24:23.471542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:24:23.471542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:24:23.471542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:24:23.471542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:24:23.471542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:24:23.471895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:24:23.471895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:24:23.471895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:23.471895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:24:23.471895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:23.471895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:24:23.471895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:23.471895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:24:23.471895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:23.471895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:24:23.471895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:23.471895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:24:23.471895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:24:23.471895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:23.471895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:24:23.471895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:23.471895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:24:23.471895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:24:23.471895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:23.472241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:24:23.472241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:23.472241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:24:23.578601 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:24:23.579202 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:24:23.579303 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:24:23.581951 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:24:23.581951 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:24:23.581951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:23.581951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:24:23.581951 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:23.581951 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:24:23.581951 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:24:23.581951 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:23.581951 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:24:23.581951 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:23.581951 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:24:23.582980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:23.582980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:24:23.582980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:24:23.582980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:24:23.582980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:24:23.582980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:24:23.582980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:24:23.582980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:23.582980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:24:23.582980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:23.582980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:24:23.582980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:23.583292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:24:23.583292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:23.583292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:24:23.583292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:23.583292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:24:23.583292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:24:23.583292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:23.583292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:24:23.583292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:23.583292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:24:23.583292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:24:23.583292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:23.583292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:24:23.583536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:23.583536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:24:23.584641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:23.584641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:24:23.584641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:24:23.584641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:24:23.584641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:24:23.584641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:24:23.584641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:24:23.584641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:24:23.584641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:24:23.584641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:24:23.584641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:24:23.584641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:24:23.584641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:24:23.584641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:24:23.584641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:24:23.584641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:24:23.584999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:24:23.584999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:24:23.584999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:23.584999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:24:23.584999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:23.584999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:24:23.584999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:23.584999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:24:23.584999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:23.584999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:24:23.584999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:23.584999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:24:23.584999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:24:23.584999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:23.584999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:24:23.584999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:23.584999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:24:23.584999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:24:23.584999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:23.585383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:24:23.585383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:23.585383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:24:23.587913 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:24:23.587913 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:24:23.588089 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:24:23.588160 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:24:23.588589 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:24:23.588799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:23.589409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:24:23.589409 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:23.589409 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:24:23.589409 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:24:23.589409 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:23.589409 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:24:23.589409 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:23.589409 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:24:23.590354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:23.590354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:24:23.590354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:24:23.590354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:24:23.590354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:23.590354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:24:23.590354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:23.590354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:24:23.590354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:23.590354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:24:23.590354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:23.590354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:24:23.590354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:23.590354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:24:23.590354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:24:23.591023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:23.591023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:24:23.591023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:23.591023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:24:23.591023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:24:23.591023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:23.591023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:24:23.591023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:23.591023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:24:23.591471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:23.591471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:24:23.591471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:24:23.591471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:24:23.591471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:24:23.591471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:24:23.591471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:24:23.591471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:24:23.591471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:24:23.591471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:24:23.591471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:23.591471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:24:23.591947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:23.591947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:24:23.591947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:23.591947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:24:23.591947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:23.591947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:24:23.591947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:23.591947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:24:23.591947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:24:23.591947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:23.591947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:24:23.591947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:23.591947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:24:23.591947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:24:23.592457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:23.592457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:24:23.592457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:23.592457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:24:23.592939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:23.592939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:24:23.592939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:24:23.592939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:24:23.592939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:24:23.592939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:24:23.592939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:24:23.592939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:24:23.592939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:24:23.592939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:24:23.592939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:24:23.592939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:24:23.592939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:24:23.592939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:24:23.592939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:24:23.592939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:24:23.593499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:24:23.593499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:24:23.593499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:24:23.593499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:24:23.593499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:24:23.593499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:23.593499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:24:23.593499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:23.593499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:24:23.593499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:23.593499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:24:23.593499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:23.593499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:24:23.593499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:23.593499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:24:23.593499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:24:23.593499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:23.593499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:24:23.594167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:23.594167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:24:23.594167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:24:23.594167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:23.594167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:24:23.594167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:23.594167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:24:23.596900 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:24:23.597274 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:24:23.597357 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:24:23.597516 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:24:23.598117 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:24:23.598518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:23.599671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:24:23.599671 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:23.599671 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:24:23.599671 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:24:23.599671 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:23.599671 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:24:23.599671 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:23.599671 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:24:23.600543 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:23.600543 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:24:23.600543 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:24:23.600543 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:24:23.600543 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:23.600543 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:24:23.600543 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:23.600543 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:24:23.600543 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:23.600543 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:24:23.600543 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:23.600543 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:24:23.600543 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:23.600543 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:24:23.600543 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:24:23.601053 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:23.601053 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:24:23.601053 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:23.601053 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:24:23.601053 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:24:23.601053 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:23.601053 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:24:23.601053 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:23.601053 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:24:23.601698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:23.601698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:24:23.601698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:24:23.601698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:24:23.601698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:24:23.601698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:24:23.601698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:24:23.601698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:24:23.601698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:24:23.601698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:23.601698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:24:23.602178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:23.602178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:24:23.602178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:23.602178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:24:23.602178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:23.602178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:24:23.602178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:23.602178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:24:23.602178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:24:23.602178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:23.602178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:24:23.602178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:23.602178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:24:23.602178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:24:23.602914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:23.602914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:24:23.602914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:23.602914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:24:23.603081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:23.603081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:24:23.603081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:24:23.603081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:24:23.603081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:24:23.603081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:24:23.603081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:24:23.603081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:24:23.603081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:24:23.603081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:24:23.603081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:24:23.603081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:24:23.603081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:24:23.603081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:24:23.603081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:24:23.603081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:24:23.603081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:24:23.603791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:24:23.603791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:24:23.603791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:24:23.603791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:23.603791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:24:23.603791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:23.603791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:24:23.603791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:23.603791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:24:23.603791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:23.603791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:24:23.603791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:23.603791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:24:23.603791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:24:23.603791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:23.603791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:24:23.603791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:23.603791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:24:23.603791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:24:23.604403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:23.604403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:24:23.604403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:23.604403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:24:23.604720 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:24:23.605062 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:24:23.605191 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:24:23.605363 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:24:23.605807 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:24:23.605975 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:24:23.606163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:23.606870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:24:23.606870 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:23.606870 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:24:23.606870 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:24:23.606870 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:23.606870 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:24:23.606870 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:23.606870 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:24:23.608135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:23.608135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:24:23.608135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:24:23.608135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:24:23.608135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:24:23.608135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:24:23.608135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:24:23.608135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:23.608135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:24:23.608135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:23.608135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:24:23.608135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:23.608701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:24:23.608701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:23.608701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:24:23.608701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:23.608701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:24:23.608701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:24:23.608701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:23.608701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:24:23.608701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:23.608701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:24:23.608701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:24:23.608701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:23.608701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:24:23.609115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:23.609115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:24:23.609420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:23.609420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:24:23.609420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:24:23.609420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:24:23.609420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:24:23.609420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:24:23.609420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:24:23.609420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:24:23.609420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:24:23.609420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:24:23.609420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:24:23.609420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:24:23.609420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:24:23.609420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:24:23.609420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:24:23.609420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:24:23.609895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:24:23.609895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:24:23.609895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:23.609895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:24:23.609895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:23.609895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:24:23.609895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:23.609895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:24:23.609895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:23.609895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:24:23.609895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:23.609895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:24:23.609895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:24:23.609895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:23.609895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:24:23.609895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:23.609895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:24:23.609895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:24:23.609895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:23.610562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:24:23.610562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:23.610562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:24:28.422906 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:24:28.423556 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:24:28.423556 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:24:28.424681 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:24:28.468214 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:24:28.468425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:28.468997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:24:28.468997 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:28.468997 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:24:28.468997 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:24:28.468997 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:28.468997 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:24:28.468997 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:28.468997 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:24:28.469715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.046 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:28.469715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:24:28.469715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:24:28.469715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:24:28.469715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:24:28.469715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:24:28.469715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:24:28.469715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:28.469715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:24:28.469715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:28.469715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:24:28.469715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:28.470072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:24:28.470072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:28.470072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:24:28.470072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:28.470072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:24:28.470072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:24:28.470072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:28.470072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:24:28.470072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:28.470072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:24:28.470072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:24:28.470072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:28.470072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:24:28.470431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:28.470431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:24:28.470492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:28.470492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:24:28.470492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:24:28.470492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:24:28.470492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:24:28.470492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:24:28.470492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:24:28.470492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:24:28.470492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:24:28.470492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:24:28.470492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:24:28.470492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:24:28.470492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:24:28.470492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:24:28.470492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:24:28.470492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:24:28.470884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:24:28.470884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:24:28.470884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:28.470884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:24:28.470884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:28.470884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:24:28.470884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:28.470884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:24:28.470884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:28.470884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:24:28.470884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:28.470884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:24:28.470884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:24:28.470884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:28.470884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:24:28.470884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:28.470884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:24:28.470884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:24:28.470884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:28.471284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:24:28.471284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:28.471284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:24:28.579271 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:24:28.579652 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:24:28.579872 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:24:28.580800 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:24:28.580989 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:24:28.581106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:28.581925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:24:28.581925 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:28.581925 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:24:28.581925 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:24:28.581925 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:28.581925 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:24:28.581925 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:28.581925 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:24:28.583050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:28.583050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:24:28.583050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:24:28.583050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:24:28.583050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:24:28.583050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:24:28.583050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:24:28.583050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:28.583050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:24:28.583050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:28.583050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:24:28.583050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:28.583886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:24:28.583886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:28.583886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:24:28.583886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:28.583886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:24:28.583886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:24:28.583886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:28.583886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:24:28.583886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:28.583886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:24:28.583886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:24:28.583886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:28.583886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:24:28.584679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:28.584679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:24:28.584679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:28.584679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:24:28.584679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:24:28.584679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:24:28.584679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:24:28.584679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:24:28.584679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:24:28.584679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:24:28.584679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:24:28.584679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:24:28.584679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:24:28.584679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:24:28.584679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:24:28.585886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:24:28.585886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:24:28.585886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:24:28.585886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:24:28.585886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:24:28.585886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:28.585886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:24:28.585886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:28.585886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:24:28.585886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:28.585886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:24:28.585886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:28.585886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:24:28.585886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:28.585886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:24:28.585886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:24:28.585886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:28.585886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:24:28.585886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:28.587182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:24:28.587182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:24:28.587182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:28.587182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:24:28.587182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:28.587182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:24:28.587978 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:24:28.588209 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:24:28.588507 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:24:28.588507 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:24:28.588925 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:24:28.588999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:28.589464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:24:28.589464 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:28.589464 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:24:28.589464 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:24:28.589464 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:28.589464 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:24:28.589464 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:28.589464 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:24:28.590497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:28.590497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:24:28.590497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:24:28.590497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:24:28.590497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:28.590497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:24:28.590497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:28.590497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:24:28.590497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:28.590497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:24:28.590497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:28.590497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:24:28.590497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:28.590497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:24:28.590497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:24:28.591316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:28.591316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:24:28.591316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:28.591316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:24:28.591316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:24:28.591316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:28.591316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:24:28.591316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:28.591316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:24:28.591654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:28.591654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:24:28.591654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:24:28.591654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:24:28.591654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:24:28.591654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:24:28.591654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:24:28.591654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:24:28.591654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:24:28.591654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:24:28.591654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:28.591654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:24:28.592081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:28.592081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:24:28.592081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:28.592081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:24:28.592081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:28.592081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:24:28.592081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:28.592081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:24:28.592081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:24:28.592081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:28.592081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:24:28.592081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:28.592081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:24:28.592081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:24:28.592481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:28.592481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:24:28.592481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:28.592481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:24:28.592481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:28.592481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:24:28.592481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:24:28.592481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:24:28.592481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:24:28.592481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:24:28.592481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:24:28.592481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:24:28.592481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:24:28.592481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:24:28.593275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:24:28.593275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:24:28.593275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:24:28.593275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:24:28.593275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:24:28.593275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:24:28.593275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:24:28.593275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:24:28.593275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:24:28.593275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:24:28.593275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:24:28.593275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:28.593275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:24:28.593275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:28.593275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:24:28.593275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:28.593275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:24:28.593275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:28.593275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:24:28.593891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:28.593891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:24:28.593891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:24:28.593891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:28.593891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:24:28.593891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:28.593891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:24:28.593891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:24:28.593891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:28.593891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:24:28.593891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:28.593891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:24:28.597302 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:24:28.597484 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:24:28.597605 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:24:28.597724 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:24:28.598187 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:24:28.598339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:28.598803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:24:28.598803 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:28.598803 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:24:28.598803 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:24:28.598803 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:28.598803 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:24:28.598803 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:28.598803 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:24:28.599539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:28.599539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:24:28.599539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:24:28.599539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:24:28.599539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:28.599539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:24:28.599539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:28.599539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:24:28.599539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:28.599539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:24:28.599539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:28.599539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:24:28.599539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:28.599539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:24:28.599539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:24:28.599992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:28.599992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:24:28.599992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:28.599992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:24:28.599992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:24:28.599992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:28.599992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:24:28.599992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:28.599992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:24:28.600400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:28.600400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:24:28.600400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:24:28.600400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:24:28.600400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:24:28.600400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:24:28.600400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:24:28.600400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:24:28.600400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:24:28.600400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:28.600400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:24:28.600738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:28.600738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:24:28.600738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:28.600738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:24:28.600738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:28.600738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:24:28.600738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:28.600738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:24:28.600738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:24:28.600738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:28.600738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:24:28.600738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:28.600738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:24:28.600738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:24:28.601128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:28.601128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:24:28.601128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:28.601128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:24:28.601456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:28.601456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:24:28.601456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:24:28.601456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:24:28.601456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:24:28.601456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:24:28.601456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:24:28.601456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:24:28.601456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:24:28.601456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:24:28.601456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:24:28.601456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:24:28.601456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:24:28.601456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:24:28.601456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:24:28.601456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:24:28.601456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:24:28.601954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:24:28.601954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:24:28.601954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:24:28.601954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:28.601954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:24:28.601954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:28.601954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:24:28.601954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:28.601954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:24:28.601954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:28.601954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:24:28.601954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:28.601954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:24:28.601954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:24:28.601954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:28.601954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:24:28.601954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:28.601954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:24:28.601954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:24:28.602433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:28.602433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:24:28.602433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:28.602433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:24:28.605020 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:24:28.605205 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:24:28.605300 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:24:28.605443 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:24:28.606071 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:24:28.606284 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:24:28.606400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:28.606846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:24:28.606846 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:28.606846 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:24:28.606846 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:24:28.606846 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:28.606846 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:24:28.606846 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:28.606846 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:24:28.607729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:28.607729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:24:28.607729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:24:28.607729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:24:28.607729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:24:28.607729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:24:28.607729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:24:28.607729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:28.607729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:24:28.607729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:28.607729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:24:28.607729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:28.608108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:24:28.608108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:28.608108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:24:28.608108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:28.608108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:24:28.608108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:24:28.608108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:28.608108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:24:28.608108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:28.608108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:24:28.608108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:24:28.608108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:28.608108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:24:28.608430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:28.608430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:24:28.608698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:28.608698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:24:28.608698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:24:28.608698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:24:28.608698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:24:28.608698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:24:28.608698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:24:28.608698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:24:28.608698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:24:28.608698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:24:28.608698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:24:28.608698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:24:28.608698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:24:28.608698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:24:28.608698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:24:28.608698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:24:28.609183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:24:28.609183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:24:28.609183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:28.609183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:24:28.609183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:28.609183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:24:28.609183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:28.609183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:24:28.609183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:28.609183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:24:28.609183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:28.609183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:24:28.609183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:24:28.609183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:28.609183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:24:28.609183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:28.609183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:24:28.609183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:24:28.609183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:28.609669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:24:28.609669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:28.609669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:24:33.423655 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:24:33.424199 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:24:33.424232 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:24:33.425769 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:24:33.462852 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:24:33.462852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:33.463515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:24:33.463515 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:33.463515 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:24:33.463515 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:24:33.463515 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:33.463515 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:24:33.463515 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:33.463515 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:24:33.464995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:33.464995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:24:33.464995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:24:33.464995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:24:33.464995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:24:33.464995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:24:33.464995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:24:33.464995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:33.464995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:24:33.464995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:33.464995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:24:33.464995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:33.465535 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:24:33.465535 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:33.465535 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:24:33.465535 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:33.465535 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:24:33.465535 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:24:33.465535 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:33.465535 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:24:33.465535 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:33.465535 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:24:33.465535 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:24:33.465535 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:33.465535 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:24:33.466334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:33.466334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:24:33.466722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:33.466722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:24:33.466722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:24:33.466722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:24:33.466722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:24:33.466722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:24:33.466722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:24:33.466722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:24:33.466722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:24:33.466722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:24:33.466722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:24:33.466722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:24:33.466722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:24:33.466722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:24:33.466722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:24:33.466722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:24:33.467945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:24:33.467945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:24:33.467945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:33.467945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:24:33.467945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:33.467945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:24:33.467945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:33.467945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:24:33.467945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:33.467945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:24:33.467945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:33.467945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:24:33.467945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:24:33.467945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:33.467945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:24:33.467945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:33.467945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:24:33.467945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:24:33.467945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:33.469626 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:24:33.469626 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:33.469626 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:24:33.579935 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:24:33.580411 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:24:33.580550 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:24:33.581895 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:24:33.581895 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:24:33.582240 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:33.582905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:24:33.582905 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:33.582905 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:24:33.582905 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:24:33.582905 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:33.582905 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:24:33.582905 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:33.582905 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:24:33.584398 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:33.584398 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:24:33.584398 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:24:33.584398 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:24:33.584398 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:24:33.584398 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:24:33.584398 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:24:33.584398 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:33.584398 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:24:33.584398 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:33.584398 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:24:33.584398 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:33.584967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:24:33.584967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:33.584967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:24:33.584967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:33.584967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:24:33.584967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:24:33.584967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:33.584967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:24:33.584967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:33.584967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:24:33.584967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:24:33.584967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:33.584967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:24:33.585443 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:33.585443 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:24:33.586354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:33.586354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:24:33.586354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:24:33.586354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:24:33.586354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:24:33.586354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:24:33.586354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:24:33.586354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:24:33.586354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:24:33.586354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:24:33.586354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:24:33.586354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:24:33.586354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:24:33.586354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:24:33.586354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:24:33.586354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:24:33.586987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:24:33.586987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:24:33.586987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:33.586987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:24:33.586987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:33.586987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:24:33.586987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:33.586987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:24:33.586987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:33.586987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:24:33.586987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:33.586987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:24:33.586987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:24:33.586987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:33.586987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:24:33.586987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:33.586987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:24:33.586987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:24:33.586987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:33.587641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:24:33.587641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:33.587641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:24:33.588249 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:24:33.588484 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:24:33.588634 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:24:33.588634 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:24:33.589018 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:24:33.589220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:33.589778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:24:33.589778 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:33.589778 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:24:33.589778 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:24:33.589778 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:33.589778 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:24:33.589778 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:33.589778 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:24:33.590558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:33.590558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:24:33.590558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:24:33.590558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:24:33.590558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:33.590558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:24:33.590558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:33.590558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:24:33.590558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:33.590558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:24:33.590558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:33.590558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:24:33.590558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:33.590558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:24:33.590558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:24:33.591097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:33.591097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:24:33.591097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:33.591097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:24:33.591097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:24:33.591097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:33.591097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:24:33.591097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:33.591097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:24:33.591408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:33.591408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:24:33.591408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:24:33.591408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:24:33.591408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:24:33.591408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:24:33.591408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:24:33.591408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:24:33.591408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:24:33.591408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:24:33.591408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:33.591408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:24:33.591790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:33.591790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:24:33.591790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:33.591790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:24:33.591790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:33.591790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:24:33.591790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:33.591790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:24:33.591790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:24:33.591790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:33.591790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:24:33.591790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:33.591790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:24:33.591790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:24:33.592134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:33.592134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:24:33.592134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:33.592134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:24:33.592505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:33.592505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:24:33.592505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:24:33.592505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:24:33.592505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:24:33.592505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:24:33.592505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:24:33.592505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:24:33.592505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:24:33.592505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:24:33.592505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:24:33.592505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:24:33.592505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:24:33.592505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:24:33.592505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:24:33.592505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:24:33.592941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:24:33.592941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:24:33.592941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:24:33.592941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:24:33.592941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:24:33.592941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:33.592941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:24:33.592941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:33.592941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:24:33.592941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:33.592941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:24:33.592941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:33.592941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:24:33.592941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:33.592941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:24:33.592941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:24:33.592941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:33.592941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:24:33.593394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:33.593394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:24:33.593394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:24:33.593394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:33.593394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:24:33.593394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:33.593394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:24:33.597604 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:24:33.597883 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:24:33.598022 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:24:33.598204 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:24:33.598542 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:24:33.598854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:33.599404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:24:33.599404 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:33.599404 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:24:33.599404 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:24:33.599404 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:33.599404 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:24:33.599404 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:33.599404 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:24:33.600141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:33.600141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:24:33.600141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:24:33.600141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:24:33.600141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:33.600141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:24:33.600141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:33.600141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:24:33.600141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:33.600141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:24:33.600141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:33.600141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:24:33.600141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:33.600141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:24:33.600141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:24:33.600484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:33.600484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:24:33.600484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:33.600484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:24:33.600484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:24:33.600484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:33.600484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:24:33.600484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:33.600484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:24:33.601118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:33.601118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:24:33.601118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:24:33.601118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:24:33.601118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:24:33.601118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:24:33.601118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:24:33.601118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:24:33.601118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:24:33.601118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:33.601118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:24:33.601520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:33.601520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:24:33.601520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:33.601520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:24:33.601520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:33.601520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:24:33.601520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:33.601520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:24:33.601520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:24:33.601520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:33.601520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:24:33.601520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:33.601520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:24:33.601520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:24:33.601899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:33.601899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:24:33.601899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:33.601899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:24:33.602403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:33.602403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:24:33.602403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:24:33.602403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:24:33.602403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:24:33.602403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:24:33.602403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:24:33.602403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:24:33.602403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:24:33.602403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:24:33.602403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:24:33.602403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:24:33.602403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:24:33.602403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:24:33.602403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:24:33.602403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:24:33.602403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:24:33.602838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:24:33.602838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:24:33.602838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:24:33.602838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:33.602838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:24:33.602838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:33.602838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:24:33.602838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:33.602838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:24:33.602838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:33.602838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:24:33.602838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:33.602838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:24:33.602838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:24:33.602838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:33.602838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:24:33.602838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:33.602838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:24:33.602838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:24:33.603319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:33.603319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:24:33.603319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:33.603319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:24:33.605555 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:24:33.606770 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:24:33.606770 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:24:33.606770 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:24:33.606876 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:24:33.607157 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:24:33.607328 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:33.608015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:24:33.608015 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:33.608015 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:24:33.608015 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:24:33.608015 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:33.608015 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:24:33.608015 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:33.608015 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:24:33.608927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:33.608927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:24:33.608927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:24:33.608927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:24:33.608927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:24:33.608927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:24:33.608927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:24:33.608927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:33.608927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:24:33.608927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:33.608927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:24:33.608927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:33.609235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:24:33.609235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:33.609235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:24:33.609235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:33.609235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:24:33.609235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:24:33.609235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:33.609235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:24:33.609235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:33.609235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:24:33.609235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:24:33.609235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:33.609235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:24:33.609631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:33.609631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:24:33.610025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:33.610025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:24:33.610025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:24:33.610025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:24:33.610025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:24:33.610025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:24:33.610025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:24:33.610025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:24:33.610025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:24:33.610025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:24:33.610025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:24:33.610025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:24:33.610025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:24:33.610025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:24:33.610025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:24:33.610025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:24:33.610545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:24:33.610545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:24:33.610545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:33.610545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:24:33.610545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:33.610545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:24:33.610545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:33.610545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:24:33.610545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:33.610545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:24:33.610545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:33.610545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:24:33.610545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:24:33.610545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:33.610545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:24:33.610545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:33.610545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:24:33.610545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:24:33.610545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:33.611052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:24:33.611052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:33.611052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:24:38.425190 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:24:38.425980 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:24:38.425980 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:24:38.426733 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:24:38.463393 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:24:38.463872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:38.464803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:24:38.464803 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:38.464803 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:24:38.464803 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:24:38.464803 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:38.464803 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:24:38.464803 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:38.464803 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:24:38.465578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:38.465578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:24:38.465578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:24:38.465578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:24:38.465578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:24:38.465578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:24:38.465578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:24:38.465578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:38.465578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:24:38.465578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:38.465578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:24:38.465578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:38.465947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:24:38.465947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:38.465947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:24:38.465947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:38.465947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:24:38.465947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:24:38.465947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:38.465947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:24:38.465947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:38.465947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:24:38.465947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:24:38.465947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:38.465947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:24:38.466373 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:38.466373 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:24:38.466373 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:38.466373 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:24:38.466373 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:24:38.466373 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:24:38.466373 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:24:38.466373 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:24:38.466373 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:24:38.466373 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:24:38.466373 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:24:38.466373 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:24:38.466373 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:24:38.466373 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:24:38.466373 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:24:38.466914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:24:38.466914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:24:38.466914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:24:38.466914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:24:38.466914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:24:38.466914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:38.466914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:24:38.466914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:38.466914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:24:38.466914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:38.466914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:24:38.466914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:38.466914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:24:38.466914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:38.466914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:24:38.466914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:24:38.466914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:38.466914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:24:38.466914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:38.467655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:24:38.467655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:24:38.467655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:38.467655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:24:38.467655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:38.467655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:24:38.580832 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:24:38.582195 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:24:38.582379 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:24:38.583451 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:24:38.583805 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:24:38.584034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:38.587558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:24:38.587558 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:38.587558 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:24:38.587558 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:24:38.587558 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:38.587558 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:24:38.587558 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:38.587558 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:24:38.587558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:38.587558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:24:38.587558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:24:38.587558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:24:38.587558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:24:38.587558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:24:38.587558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:24:38.588553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:38.588553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:24:38.588553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:38.588553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:24:38.588553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:38.588553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:24:38.588553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:38.588553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:24:38.588553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:38.588553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:24:38.588553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:24:38.588553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:38.588553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:24:38.588553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:38.588975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:24:38.588975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:24:38.588975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:38.588975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:24:38.588975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:38.588975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:24:38.589535 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:38.589535 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:24:38.589535 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:24:38.589535 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:24:38.589535 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:24:38.589535 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:24:38.589535 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:24:38.589535 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:24:38.589535 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:24:38.589535 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:24:38.589535 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:24:38.589535 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:24:38.589535 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:24:38.589535 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:24:38.589535 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:24:38.589535 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:24:38.590309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:24:38.590309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:24:38.590309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:38.590309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:24:38.590309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:38.590309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:24:38.590309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:38.590309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:24:38.590309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:38.590309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:24:38.590309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:38.590309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:24:38.590309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:24:38.590309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:38.590309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:24:38.590309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:38.590309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:24:38.590309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:24:38.590309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:38.590880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:24:38.590880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:38.590880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:24:38.592339 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:24:38.593141 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:24:38.593141 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:24:38.593141 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:24:38.593977 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:24:38.594158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:38.597151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:24:38.597151 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:38.597151 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:24:38.597151 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:24:38.597151 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:38.597151 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:24:38.597151 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:38.597151 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:24:38.597151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:38.597151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:24:38.597151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:24:38.597151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:24:38.597151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:38.597151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:24:38.597151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:38.597151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:24:38.597151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:38.597151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:24:38.597684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:38.597684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:24:38.597684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:38.597684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:24:38.597684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:24:38.597684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:38.597684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:24:38.597684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:38.597684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:24:38.597684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:24:38.597684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:38.597684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:24:38.597684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:38.597684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:24:38.597684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:38.598260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:24:38.598260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:24:38.598260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:24:38.598260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:24:38.598260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:24:38.598260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:24:38.598260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:24:38.598260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:24:38.598260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:24:38.598260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:38.598260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:24:38.598260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:38.598260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:24:38.598733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:38.598733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:24:38.598733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:38.598733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:24:38.598733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:38.598733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:24:38.598733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:24:38.598733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:38.598733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:24:38.598733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:38.598733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:24:38.598733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:24:38.598733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:38.598733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:24:38.599084 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:38.599084 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:24:38.599084 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:38.599084 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:24:38.599084 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:24:38.599084 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:24:38.599084 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:24:38.599084 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:24:38.599084 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:24:38.599084 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:24:38.599084 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:24:38.599084 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:24:38.599084 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:24:38.599084 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:24:38.599084 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:24:38.599364 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:24:38.599364 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:24:38.599364 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:24:38.599364 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:24:38.599364 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:24:38.599364 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:24:38.599364 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:24:38.599364 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:24:38.599364 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:38.599364 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:24:38.599364 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:38.599364 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:24:38.599364 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:38.599364 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:24:38.599364 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:38.599364 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:24:38.599364 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:38.599364 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:24:38.599364 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:24:38.599811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:38.599811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:24:38.599811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:38.599811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:24:38.599811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:24:38.599811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:38.599811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:24:38.599811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:38.599811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:24:38.599811 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:24:38.600612 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:24:38.600869 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:24:38.601147 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:24:38.601640 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:24:38.601999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:38.603265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:24:38.603265 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:38.603265 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:24:38.603265 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:24:38.603265 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:38.603265 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:24:38.603265 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:38.603265 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:24:38.603265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:38.603265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:24:38.603265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:24:38.603265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:24:38.603265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:38.603265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:24:38.603265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:38.603265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:24:38.603265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:38.603265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:24:38.603999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:38.603999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:24:38.603999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:38.603999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:24:38.603999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:24:38.603999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:38.603999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:24:38.603999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:38.603999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:24:38.603999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:24:38.603999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:38.603999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:24:38.603999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:38.603999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:24:38.603999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:38.604574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:24:38.604574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:24:38.604574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:24:38.604574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:24:38.604574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:24:38.604574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:24:38.604574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:24:38.604574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:24:38.604574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:38.604574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:24:38.604574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:38.604574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:24:38.604574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:38.606261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:24:38.606261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:38.606261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:24:38.606261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:38.606261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:24:38.606261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:24:38.606261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:38.606261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:24:38.606261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:38.606261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:24:38.606261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:24:38.606261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:38.606261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:24:38.607375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:38.607375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:24:38.607375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:38.607375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:24:38.607375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:24:38.607375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:24:38.607375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:24:38.607375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:24:38.607375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:24:38.607375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:24:38.607375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:24:38.607375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:24:38.607375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:24:38.607375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:24:38.607375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:24:38.608111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:24:38.608111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:24:38.608111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:24:38.608111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:24:38.608111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:24:38.608111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:24:38.608111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:24:38.608111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:38.608111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:24:38.608111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:38.608111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:24:38.608111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:38.608111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:24:38.608111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:38.608111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:24:38.608111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:38.608111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:24:38.608111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:24:38.608111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:38.608647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:24:38.608647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:38.608647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:24:38.608647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:24:38.608647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:38.608647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:24:38.608647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:38.608647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:24:38.608647 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:24:38.608647 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:24:38.608647 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:24:38.609013 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:24:38.609013 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:24:38.609013 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:24:38.609013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:38.609013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:24:38.609013 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:38.609013 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:24:38.609013 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:24:38.609013 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:38.609013 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:24:38.609013 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:38.609013 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:24:38.609384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:38.609384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:24:38.609384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:24:38.609384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:24:38.609384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:24:38.609384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:24:38.609384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:24:38.609384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:38.609384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:24:38.609384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:38.609384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:24:38.609384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:38.609696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:24:38.609696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:38.609696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:24:38.609696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:38.609696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:24:38.609696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:24:38.609696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:38.609696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:24:38.609696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:38.609696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:24:38.609696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:24:38.609696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:38.609696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:24:38.610298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:38.610298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:24:38.610298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:38.610298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:24:38.610298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:24:38.610298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:24:38.610298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:24:38.610298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:24:38.610298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:24:38.610298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:24:38.610298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:24:38.610298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:24:38.610298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:24:38.610298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:24:38.610298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:24:38.610699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:24:38.610699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:24:38.610699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:24:38.610699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:24:38.610699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:24:38.610699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:38.610699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:24:38.610699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:38.610699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:24:38.610699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:38.610699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:24:38.610699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:38.610699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:24:38.610699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:38.610699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:24:38.610699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:24:38.610699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:38.610699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:24:38.610699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:38.612962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:24:38.612962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:24:38.612962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:38.612962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:24:38.612962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:38.612962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:24:43.426055 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:24:43.426964 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:24:43.426964 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:24:43.428029 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:24:43.462519 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:24:43.462736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:43.464732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:24:43.464732 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:43.464732 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:24:43.464732 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:24:43.464732 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:43.464732 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:24:43.464732 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:43.464732 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:24:43.465392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:43.465392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:24:43.465392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:24:43.465392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:24:43.465392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:24:43.465392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:24:43.465392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:24:43.465392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:43.465392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:24:43.465392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:43.465392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:24:43.465392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:43.466003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:24:43.466003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:43.466003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:24:43.466003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:43.466003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:24:43.466003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:24:43.466003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:43.466003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:24:43.466003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:43.466003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:24:43.466003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:24:43.466003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:43.466003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:24:43.466552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:43.466552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:24:43.466552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:43.466552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:24:43.466552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:24:43.466552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:24:43.466552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:24:43.466552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:24:43.466552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:24:43.466552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:24:43.466552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:24:43.466552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:24:43.466552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:24:43.466552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:24:43.466552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:24:43.467308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:24:43.467308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:24:43.467308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:24:43.467308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:24:43.467308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:24:43.467308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:43.467308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:24:43.467308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:43.467308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:24:43.467308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:43.467308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:24:43.467308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:43.467308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:24:43.467308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:43.467308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:24:43.467308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:24:43.467308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:43.467308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:24:43.467308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:43.468142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:24:43.468142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:24:43.468142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:43.468142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:24:43.468142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:43.468142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:24:43.582242 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:24:43.583177 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:24:43.583305 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:24:43.584509 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:24:43.584897 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:24:43.585157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:43.586217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:24:43.586217 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:43.586217 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:24:43.586217 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:24:43.586217 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:43.586217 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:24:43.586217 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:43.586217 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:24:43.588027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:43.588027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:24:43.588027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:24:43.588027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:24:43.588027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:24:43.588027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:24:43.588027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:24:43.588027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:43.588027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:24:43.588027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:43.588027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:24:43.588027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:43.588648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:24:43.588648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:43.588648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:24:43.588648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:43.588648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:24:43.588648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:24:43.588648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:43.588648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:24:43.588648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:43.588648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:24:43.588648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:24:43.588648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:43.588648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:24:43.589331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:43.589331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:24:43.590382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:43.590382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:24:43.590382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:24:43.590382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:24:43.590382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:24:43.590382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:24:43.590382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:24:43.590382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:24:43.590382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:24:43.590382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:24:43.590382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:24:43.590382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:24:43.590382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:24:43.590382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:24:43.590382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:24:43.590382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:24:43.591460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:24:43.591460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:24:43.591460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:43.591460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:24:43.591460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:43.591460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:24:43.591460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:43.591460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:24:43.591460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:43.591460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:24:43.591460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:43.591460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:24:43.591460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:24:43.591460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:43.591460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:24:43.591460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:43.591460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:24:43.591460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:24:43.591460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:43.592566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:24:43.592566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:43.592566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:24:43.592956 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:24:43.593347 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:24:43.593526 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:24:43.593726 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:24:43.594181 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:24:43.594402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:43.595019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:24:43.595019 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:43.595019 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:24:43.595019 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:24:43.595019 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:43.595019 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:24:43.595019 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:43.595019 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:24:43.595984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:43.595984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:24:43.595984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:24:43.595984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:24:43.595984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:43.595984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:24:43.595984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:43.595984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:24:43.595984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:43.595984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:24:43.595984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:43.595984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:24:43.595984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:43.595984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:24:43.595984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:24:43.596446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:43.596446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:24:43.596446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:43.596446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:24:43.596446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:24:43.596446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:43.596446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:24:43.596446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:43.596446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:24:43.597273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:43.597273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:24:43.597273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:24:43.597273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:24:43.597273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:24:43.597273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:24:43.597273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:24:43.597273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:24:43.597273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:24:43.597273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:24:43.597273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:43.597273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:24:43.597794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:43.597794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:24:43.597794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:43.597794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:24:43.597794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:43.597794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:24:43.597794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:43.597794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:24:43.597794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:24:43.597794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:43.597794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:24:43.597794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:43.597794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:24:43.597794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:24:43.598229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:43.598229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:24:43.598229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:43.598229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:24:43.598763 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:43.598763 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:24:43.598763 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:24:43.598763 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:24:43.598763 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:24:43.598763 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:24:43.598763 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:24:43.598763 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:24:43.598763 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:24:43.598763 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:24:43.598763 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:24:43.598763 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:24:43.598763 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:24:43.598763 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:24:43.598763 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:24:43.598763 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:24:43.599222 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:24:43.599222 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:24:43.599222 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:24:43.599222 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:24:43.599222 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:24:43.599222 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:43.599222 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:24:43.599222 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:43.599222 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:24:43.599222 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:43.599222 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:24:43.599222 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:43.599222 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:24:43.599222 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:43.599222 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:24:43.599222 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:24:43.599222 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:43.599222 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:24:43.599711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:43.599711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:24:43.599711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:24:43.599711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:43.599711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:24:43.599711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:43.599711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:24:43.600788 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:24:43.601208 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:24:43.601402 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:24:43.601604 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:24:43.602150 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:24:43.602523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:43.603328 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:24:43.603328 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:43.603328 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:24:43.603328 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:24:43.603328 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:43.603328 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:24:43.603328 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:43.603328 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:24:43.604194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:43.604194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:24:43.604194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:24:43.604194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:24:43.604194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:43.604194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:24:43.604194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:43.604194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:24:43.604194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:43.604194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:24:43.604194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:43.604194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:24:43.604194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:43.604194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:24:43.604194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:24:43.604588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:43.604588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:24:43.604588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:43.604588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:24:43.604588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:24:43.604588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:43.604588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:24:43.604588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:43.604588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:24:43.605368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:43.605368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:24:43.605368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:24:43.605368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:24:43.605368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:24:43.605368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:24:43.605368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:24:43.605368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:24:43.605368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:24:43.605368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:43.605368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:24:43.605716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:43.605716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:24:43.605716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:43.605716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:24:43.605716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:43.605716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:24:43.605716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:43.605716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:24:43.605716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:24:43.605716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:43.605716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:24:43.605716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:43.605716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:24:43.605716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:24:43.606118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:43.606118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:24:43.606118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:43.606118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:24:43.606442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:43.606442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:24:43.606442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:24:43.606442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:24:43.606442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:24:43.606442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:24:43.606442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:24:43.606442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:24:43.606442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:24:43.606442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:24:43.606442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:24:43.606442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:24:43.606442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:24:43.606442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:24:43.606442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:24:43.606442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:24:43.606442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:24:43.606955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:24:43.606955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:24:43.606955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:24:43.606955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:43.606955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:24:43.606955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:43.606955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:24:43.606955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:43.606955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:24:43.606955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:43.606955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:24:43.606955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:43.606955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:24:43.606955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:24:43.606955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:43.606955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:24:43.606955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:43.606955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:24:43.606955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:24:43.607443 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:43.607443 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:24:43.607443 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:43.607443 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:24:43.607443 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:24:43.607443 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:24:43.607443 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:24:43.607443 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:24:43.607930 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:24:43.608294 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:24:43.608532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:43.609340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:24:43.609340 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:43.609340 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:24:43.609340 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:24:43.609340 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:43.609340 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:24:43.609340 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:43.609340 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:24:43.610097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:43.610097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:24:43.610097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:24:43.610097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:24:43.610097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:24:43.610097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:24:43.610097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:24:43.610097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:43.610097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:24:43.610097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:43.610097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:24:43.610097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:43.610453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:24:43.610453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:43.610453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:24:43.610453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:43.610453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:24:43.610453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:24:43.610453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:43.610453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:24:43.610453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:43.610453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:24:43.610453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:24:43.610453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:43.610453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:24:43.610811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:43.610811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:24:43.610952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:43.610952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:24:43.610952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:24:43.610952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:24:43.610952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:24:43.610952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:24:43.610952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:24:43.610952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:24:43.610952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:24:43.610952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:24:43.610952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:24:43.610952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:24:43.610952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:24:43.610952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:24:43.610952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:24:43.610952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:24:43.611432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:24:43.611432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:24:43.611432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:43.611432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:24:43.611432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:43.611432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:24:43.611432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:43.611432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:24:43.611432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:43.611432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:24:43.611432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:43.611432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:24:43.611432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:24:43.611432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:43.611432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:24:43.611432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:43.611432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:24:43.611432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:24:43.611432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:43.611935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:24:43.611935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:43.611935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:24:47.981273 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_for_inconsistencies' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:24:47.982181 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_for_inconsistencies {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:24:47.982181 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Checking Neutron and OVN revision consistency. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:24:47.993462 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Maintenance task: Synchronizing Neutron and OVN databases started {{(pid=106314) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:440}}
May 09 09:24:47.993653 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Maintenance task: Number of inconsistencies found at create/update: networks=4, routers=1, router_ports=1 {{(pid=106314) _log /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:419}}
May 09 09:24:47.993798 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Maintenance task: Fixing resource f8a38fcb-e8a2-421a-a340-bf7f2f5cc72b (type: networks) at create/update {{(pid=106314) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 09 09:24:48.031091 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-f8a38fcb-e8a2-421a-a340-bf7f2f5cc72b, resource={'id': 'f8a38fcb-e8a2-421a-a340-bf7f2f5cc72b', 'name': 'private', 'tenant_id': 'f4d54dfe001d43fbb344093a1ccabd6a', 'project_id': 'f4d54dfe001d43fbb344093a1ccabd6a', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['96e72a03-97b8-412a-b2b9-1bf048ce1066', 'bbd6957a-953f-4fc1-8761-5d93128564d9'], 'standard_attr_id': 12, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-05-09T09:04:34Z', 'updated_at': '2026-05-09T09:04:37Z', 'revision_number': 3, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 3730}, resource_type=networks, if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:24:48.031358 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-f8a38fcb-e8a2-421a-a340-bf7f2f5cc72b, col_values=(('external_ids', {'neutron:network_name': 'private', 'neutron:mtu': '1372', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:24:48.031794 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-78a32c7c-7637-4b83-b0ab-62aee8bb3038, external_ids_update=None, columns={'tag': 3730}, if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:24:48.031910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:48.032651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:24:48.032651 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:48.032651 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:24:48.032651 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:24:48.032651 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:48.032651 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:24:48.032651 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:48.032651 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:24:48.034378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Maintenance task: Failed to fix resource f8a38fcb-e8a2-421a-a340-bf7f2f5cc72b (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:48.034378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:24:48.034378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 09 09:24:48.034378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 09 09:24:48.034378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 09 09:24:48.034378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 09 09:24:48.034378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2332, in update_network
May 09 09:24:48.034378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:24:48.034378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:48.034378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:24:48.034378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:48.035519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:24:48.035519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:48.035519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:24:48.035519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:48.035519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:24:48.035519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:48.035519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:24:48.035519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:24:48.035519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:48.035519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:24:48.035519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:48.035519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:24:48.035519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:24:48.035519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:48.036876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:24:48.036876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:48.036876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:24:48.036876 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Maintenance task: Fixing resource d18a9fac-2ebf-4d1e-a86c-0f33fba8eeda (type: networks) at create/update {{(pid=106314) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 09 09:24:48.079791 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-d18a9fac-2ebf-4d1e-a86c-0f33fba8eeda, resource={'id': 'd18a9fac-2ebf-4d1e-a86c-0f33fba8eeda', 'name': 'public', 'tenant_id': 'f4d54dfe001d43fbb344093a1ccabd6a', 'project_id': 'f4d54dfe001d43fbb344093a1ccabd6a', 'admin_state_up': True, 'mtu': 1430, 'status': 'ACTIVE', 'subnets': ['088bbd0f-7e81-4a76-8399-24d15c984009', 'bbd6a8cc-80e6-4336-a70b-1c962c561590'], 'standard_attr_id': 18, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': True, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'is_default': True, 'tags': [], 'created_at': '2026-05-09T09:04:40Z', 'updated_at': '2026-05-09T09:04:50Z', 'revision_number': 3, 'provider:network_type': 'flat', 'provider:physical_network': 'public', 'provider:segmentation_id': None}, resource_type=networks, if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:24:48.079791 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-d18a9fac-2ebf-4d1e-a86c-0f33fba8eeda, col_values=(('external_ids', {'neutron:network_name': 'public', 'neutron:mtu': '1430', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'flat', 'neutron:provnet-physical-network': 'public'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false', 'fdb_age_threshold': '300', 'broadcast-arps-to-all-routers': 'true'})), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:24:48.080100 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-1163f525-70cd-4b84-ae9b-7d56378305e9, external_ids_update=None, columns={'tag': []}, if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:24:48.080100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:48.080206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:24:48.080206 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:48.080206 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:24:48.080206 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:24:48.080206 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:48.080206 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:24:48.080206 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:48.080206 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:24:48.081972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Maintenance task: Failed to fix resource d18a9fac-2ebf-4d1e-a86c-0f33fba8eeda (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:48.081972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:24:48.081972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 09 09:24:48.081972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 09 09:24:48.081972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 09 09:24:48.081972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 09 09:24:48.081972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2332, in update_network
May 09 09:24:48.081972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:24:48.081972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:48.081972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:24:48.081972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:48.082570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:24:48.082570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:48.082570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:24:48.082570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:48.082570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:24:48.082570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:48.082570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:24:48.082570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:24:48.082570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:48.082570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:24:48.082570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:48.082570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:24:48.082570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:24:48.082570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:48.083199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:24:48.083199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:48.083199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:24:48.083199 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Maintenance task: Fixing resource 417da935-2882-452f-8ba7-9076d68fcd2b (type: networks) at create/update {{(pid=106314) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 09 09:24:48.117731 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-417da935-2882-452f-8ba7-9076d68fcd2b, resource={'id': '417da935-2882-452f-8ba7-9076d68fcd2b', 'name': 'admin_net', 'tenant_id': '38e536cfc9c94b6cbf529caf55cc7739', 'project_id': '38e536cfc9c94b6cbf529caf55cc7739', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['d3c53d14-f490-4615-a986-3715a315f827'], 'standard_attr_id': 42, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-05-09T09:10:38Z', 'updated_at': '2026-05-09T09:10:39Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 32823}, resource_type=networks, if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:24:48.118261 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-417da935-2882-452f-8ba7-9076d68fcd2b, col_values=(('external_ids', {'neutron:network_name': 'admin_net', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:24:48.119042 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-5738ab2a-b471-447b-9292-7f8c6bc97168, external_ids_update=None, columns={'tag': 32823}, if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:24:48.119042 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:48.119823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:24:48.119823 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:48.119823 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:24:48.119823 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:24:48.119823 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:48.119823 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:24:48.119823 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:48.119823 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:24:48.121157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Maintenance task: Failed to fix resource 417da935-2882-452f-8ba7-9076d68fcd2b (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:48.121157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:24:48.121157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 09 09:24:48.121157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 09 09:24:48.121157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 09 09:24:48.121157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 09 09:24:48.121157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2332, in update_network
May 09 09:24:48.121157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:24:48.121157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:48.121157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:24:48.121157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:48.121557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:24:48.121557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:48.121557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:24:48.121557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:48.121557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:24:48.121557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:48.121557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:24:48.121557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:24:48.121557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:48.121557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:24:48.121557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:48.121557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:24:48.121557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:24:48.121557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:48.122070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:24:48.122070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:48.122070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:24:48.122070 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Maintenance task: Fixing resource 433617e8-cdf8-414a-8cc6-63a7b5d0c678 (type: networks) at create/update {{(pid=106314) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 09 09:24:48.156431 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-433617e8-cdf8-414a-8cc6-63a7b5d0c678, resource={'id': '433617e8-cdf8-414a-8cc6-63a7b5d0c678', 'name': 'k8s-clusterapi-cluster-magnum-system-kube-whtb8', 'tenant_id': '3e013d45a44544c3b50eb0e83e4b1efc', 'project_id': '3e013d45a44544c3b50eb0e83e4b1efc', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['9187f065-9264-427a-8df8-672a6981191d'], 'standard_attr_id': 52, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-05-09T09:11:20Z', 'updated_at': '2026-05-09T09:11:20Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 32644}, resource_type=networks, if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:24:48.156899 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-433617e8-cdf8-414a-8cc6-63a7b5d0c678, col_values=(('external_ids', {'neutron:network_name': 'k8s-clusterapi-cluster-magnum-system-kube-whtb8', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:24:48.157472 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-8c3fe586-c182-47ba-9845-d7608113cb71, external_ids_update=None, columns={'tag': 32644}, if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:24:48.157721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:48.158864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:24:48.158864 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:48.158864 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:24:48.158864 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:24:48.158864 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:48.158864 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:24:48.158864 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:48.158864 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:24:48.160332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Maintenance task: Failed to fix resource 433617e8-cdf8-414a-8cc6-63a7b5d0c678 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:48.160332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:24:48.160332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 09 09:24:48.160332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 09 09:24:48.160332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 09 09:24:48.160332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 09 09:24:48.160332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2332, in update_network
May 09 09:24:48.160332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:24:48.160332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:48.160332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:24:48.160332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:48.160871 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:24:48.160871 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:48.160871 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:24:48.160871 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:48.160871 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:24:48.160871 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:48.160871 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:24:48.160871 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:24:48.160871 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:48.160871 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:24:48.160871 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:48.160871 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:24:48.160871 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:24:48.160871 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:48.161425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:24:48.161425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:48.161425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:24:48.161425 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Maintenance task: Fixing resource cab10384-14c5-492c-8bd0-061f5838310c (type: routers) at create/update {{(pid=106314) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 09 09:24:48.240141 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-cab10384-14c5-492c-8bd0-061f5838310c, resource={'id': 'cab10384-14c5-492c-8bd0-061f5838310c', 'name': 'k8s-clusterapi-cluster-magnum-system-kube-whtb8', 'project_id': '3e013d45a44544c3b50eb0e83e4b1efc', 'admin_state_up': True, 'status': 'ACTIVE', 'enable_snat': True, 'external_gateway_info': {'network_id': 'd18a9fac-2ebf-4d1e-a86c-0f33fba8eeda', 'external_fixed_ips': [{'subnet_id': 'bbd6a8cc-80e6-4336-a70b-1c962c561590', 'ip_address': '172.24.5.116'}, {'subnet_id': '088bbd0f-7e81-4a76-8399-24d15c984009', 'ip_address': '2001:db8::1b7'}], 'enable_snat': True}, 'gw_port_id': 'af118f11-dc01-429a-b20a-e97b77b8d5ec', 'description': 'Created by cluster-api-provider-openstack cluster magnum-system-kube-whtb8', 'availability_zones': [], 'distributed': False, 'ha': True, 'ha_vr_id': 0, 'availability_zone_hints': [], 'enable_default_route_ecmp': False, 'enable_default_route_bfd': False, 'external_gateways': [{'network_id': 'd18a9fac-2ebf-4d1e-a86c-0f33fba8eeda', 'external_fixed_ips': [{'ip_address': '172.24.5.116', 'subnet_id': 'bbd6a8cc-80e6-4336-a70b-1c962c561590'}, {'ip_address': '2001:db8::1b7', 'subnet_id': '088bbd0f-7e81-4a76-8399-24d15c984009'}]}], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2026-05-09T09:11:21Z', 'updated_at': '2026-05-09T09:11:24Z', 'revision_number': 3, 'tenant_id': '3e013d45a44544c3b50eb0e83e4b1efc'}, resource_type=routers, if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:24:48.240398 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterCommand(_result=None, name=neutron-cab10384-14c5-492c-8bd0-061f5838310c, columns={'external_ids': {'neutron:router_name': 'k8s-clusterapi-cluster-magnum-system-kube-whtb8', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': ''}, 'enabled': True}, if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:24:48.240488 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): QoSDelCommand(_result=None, switch=neutron-d18a9fac-2ebf-4d1e-a86c-0f33fba8eeda, conditions=[('direction', '=', 'from-lport'), ('priority', '=', 2002), ('match', '=', 'inport == "af118f11-dc01-429a-b20a-e97b77b8d5ec"')], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:24:48.240619 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): QoSDelCommand(_result=None, switch=neutron-d18a9fac-2ebf-4d1e-a86c-0f33fba8eeda, conditions=[('direction', '=', 'to-lport'), ('priority', '=', 2002), ('match', '=', 'outport == "af118f11-dc01-429a-b20a-e97b77b8d5ec"')], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:24:48.240766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:48.241738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:24:48.241738 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:48.241738 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:24:48.241738 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:24:48.241738 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:48.241738 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:24:48.241738 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:48.241738 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:24:48.241738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_client [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Unable to update router cab10384-14c5-492c-8bd0-061f5838310c. Error: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:48.243765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Maintenance task: Failed to fix resource cab10384-14c5-492c-8bd0-061f5838310c (type: routers): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:48.243765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:24:48.243765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 09 09:24:48.243765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 09 09:24:48.243765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 09 09:24:48.243765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 09 09:24:48.243765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1635, in update_router
May 09 09:24:48.243765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with excutils.save_and_reraise_exception():
May 09 09:24:48.243765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 271, in __exit__
May 09 09:24:48.243765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.force_reraise()
May 09 09:24:48.243765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 233, in force_reraise
May 09 09:24:48.244346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise self.value
May 09 09:24:48.244346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1569, in update_router
May 09 09:24:48.244346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:24:48.244346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:48.244346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:24:48.244346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:48.244346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:24:48.244346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:48.244346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:24:48.244346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:48.244346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:24:48.244346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:48.244346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:24:48.244346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:24:48.245071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:48.245071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:24:48.245071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:48.245071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:24:48.245071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:24:48.245071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:48.245071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:24:48.245071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:48.245071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:24:48.245071 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Maintenance task: Fixing resource af118f11-dc01-429a-b20a-e97b77b8d5ec (type: router_ports) at create/update {{(pid=106314) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 09 09:24:48.364114 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=lrp-af118f11-dc01-429a-b20a-e97b77b8d5ec, resource={'id': 'af118f11-dc01-429a-b20a-e97b77b8d5ec', 'name': '', 'network_id': 'd18a9fac-2ebf-4d1e-a86c-0f33fba8eeda', 'project_id': '', 'mac_address': 'fa:16:3e:51:04:94', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'cab10384-14c5-492c-8bd0-061f5838310c', 'device_owner': 'network:router_gateway', 'standard_attr_id': 57, 'fixed_ips': [{'subnet_id': 'bbd6a8cc-80e6-4336-a70b-1c962c561590', 'ip_address': '172.24.5.116'}, {'subnet_id': '088bbd0f-7e81-4a76-8399-24d15c984009', 'ip_address': '2001:db8::1b7'}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'np0000173033', 'binding:vif_type': 'ovs', 'binding:vif_details': {'port_filter': True, 'connectivity': 'l2', 'ovs_create_tap': False, 'bridge_name': 'br-int', 'datapath_type': 'system', 'bound_drivers': {'0': 'ovn'}}, 'port_security_enabled': False, 'tags': [], 'created_at': '2026-05-09T09:11:21Z', 'updated_at': '2026-05-09T09:11:24Z', 'revision_number': 5, 'tenant_id': ''}, resource_type=router_ports, if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:24:48.364530 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterPortCommand(_result=None, name=lrp-af118f11-dc01-429a-b20a-e97b77b8d5ec, columns={'external_ids': {'neutron:revision_number': '5', 'neutron:subnet_ids': 'bbd6a8cc-80e6-4336-a70b-1c962c561590 088bbd0f-7e81-4a76-8399-24d15c984009', 'neutron:network_name': 'neutron-d18a9fac-2ebf-4d1e-a86c-0f33fba8eeda', 'neutron:is_ext_gw': 'True', 'neutron:router_name': 'neutron-cab10384-14c5-492c-8bd0-061f5838310c'}, 'options': {'reside-on-redirect-chassis': 'true', 'gateway_mtu': '1372'}, 'networks': ['172.24.5.116/24', '2001:db8::1b7/64'], 'ipv6_ra_configs': {}}, if_exists=False) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:24:48.364866 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLRouterPortInLSwitchPortCommand(_result=None, lswitch_port=af118f11-dc01-429a-b20a-e97b77b8d5ec, lrouter_port=lrp-af118f11-dc01-429a-b20a-e97b77b8d5ec, is_gw_port=True, if_exists=True, lsp_address=router) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:24:48.364965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:48.365960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:24:48.365960 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:48.365960 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:24:48.365960 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:24:48.365960 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:48.365960 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:24:48.365960 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:48.365960 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:24:48.366411 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Maintenance task: Failed to fix resource af118f11-dc01-429a-b20a-e97b77b8d5ec (type: router_ports): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:48.366411 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:24:48.366411 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 09 09:24:48.366411 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 09 09:24:48.366411 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 09 09:24:48.366411 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 09 09:24:48.366411 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1976, in update_router_port
May 09 09:24:48.366411 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:24:48.366411 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:48.366411 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:24:48.366411 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:48.366954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:24:48.366954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:48.366954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:24:48.366954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:48.366954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:24:48.366954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:48.366954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:24:48.366954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:24:48.366954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:48.366954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:24:48.366954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:48.366954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:24:48.366954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:24:48.366954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:48.367326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:24:48.367326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:48.367326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:24:48.367326 np0000173033 neutron-ovn-maintenance-worker[106314]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_for_inconsistencies finished in 0.384 seconds
May 09 09:24:48.426536 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:24:48.427196 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:24:48.427388 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:24:48.428957 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:24:48.465376 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:24:48.465650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:48.466431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:24:48.466431 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:48.466431 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:24:48.466431 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:24:48.466431 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:48.466431 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:24:48.466431 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:48.466431 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:24:48.469783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:48.469783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:24:48.469783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:24:48.469783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:24:48.469783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:24:48.469783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:24:48.469783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:24:48.469783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:48.469783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:24:48.469783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:48.469783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:24:48.469783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:48.470350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:24:48.470350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:48.470350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:24:48.470350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:48.470350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:24:48.470350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:24:48.470350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:48.470350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:24:48.470350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:48.470350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:24:48.470350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:24:48.470350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:48.470350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:24:48.470845 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:48.470845 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:24:48.470845 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:48.470845 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:24:48.470845 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:24:48.470845 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:24:48.470845 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:24:48.470845 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:24:48.470845 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:24:48.470845 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:24:48.470845 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:24:48.470845 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:24:48.470845 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:24:48.470845 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:24:48.470845 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:24:48.471496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:24:48.471496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:24:48.471496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:24:48.471496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:24:48.471496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:24:48.471496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:48.471496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:24:48.471496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:48.471496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:24:48.471496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:48.471496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:24:48.471496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:48.471496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:24:48.471496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:48.471496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:24:48.471496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:24:48.471496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:48.471496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:24:48.471496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:48.472268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:24:48.472268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:24:48.472268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:48.472268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:24:48.472268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:48.472268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:24:48.582979 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:24:48.583583 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:24:48.583843 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:24:48.585365 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:24:48.585365 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:24:48.585516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:48.586416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:24:48.586416 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:48.586416 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:24:48.586416 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:24:48.586416 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:48.586416 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:24:48.586416 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:48.586416 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:24:48.588074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:48.588074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:24:48.588074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:24:48.588074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:24:48.588074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:24:48.588074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:24:48.588074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:24:48.588074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:48.588074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:24:48.588074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:48.588074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:24:48.588074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:48.588584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:24:48.588584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:48.588584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:24:48.588584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:48.588584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:24:48.588584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:24:48.588584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:48.588584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:24:48.588584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:48.588584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:24:48.588584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:24:48.588584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:48.588584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:24:48.589099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:48.589099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:24:48.589725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:48.589725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:24:48.589725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:24:48.589725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:24:48.589725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:24:48.589725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:24:48.589725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:24:48.589725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:24:48.589725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:24:48.589725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:24:48.589725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:24:48.589725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:24:48.589725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:24:48.589725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:24:48.589725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:24:48.589725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:24:48.590345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:24:48.590345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:24:48.590345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:48.590345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:24:48.590345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:48.590345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:24:48.590345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:48.590345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:24:48.590345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:48.590345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:24:48.590345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:48.590345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:24:48.590345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:24:48.590345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:48.590345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:24:48.590345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:48.590345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:24:48.590345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:24:48.590345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:48.590986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:24:48.590986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:48.590986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:24:48.593409 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:24:48.593883 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:24:48.594095 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:24:48.594300 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:24:48.594855 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:24:48.595478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:48.596559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:24:48.596559 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:48.596559 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:24:48.596559 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:24:48.596559 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:48.596559 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:24:48.596559 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:48.596559 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:24:48.598163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:48.598163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:24:48.598163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:24:48.598163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:24:48.598163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:48.598163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:24:48.598163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:48.598163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:24:48.598163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:48.598163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:24:48.598163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:48.598163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:24:48.598163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:48.598163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:24:48.598163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:24:48.598885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:48.598885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:24:48.598885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:48.598885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:24:48.598885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:24:48.598885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:48.598885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:24:48.598885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:48.598885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:24:48.599716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:48.599716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:24:48.599716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:24:48.599716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:24:48.599716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:24:48.599716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:24:48.599716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:24:48.599716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:24:48.599716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:24:48.599716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:24:48.599716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:48.599716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:24:48.600244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:48.600244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:24:48.600244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:48.600244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:24:48.600244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:48.600244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:24:48.600244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:48.600244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:24:48.600244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:24:48.600244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:48.600244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:24:48.600244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:48.600244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:24:48.600244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:24:48.600719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:48.600719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:24:48.600719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:48.600719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:24:48.601572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:48.601572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:24:48.601572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:24:48.601572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:24:48.601572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:24:48.601572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:24:48.601572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:24:48.601572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:24:48.601572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:24:48.601572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:24:48.601572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:24:48.601572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:24:48.601572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:24:48.601572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:24:48.601572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:24:48.601572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:24:48.602234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:24:48.602234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:24:48.602234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:24:48.602234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:24:48.602234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:24:48.602234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:48.602234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:24:48.602234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:48.602234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:24:48.602234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:48.602234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:24:48.602234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:48.602234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:24:48.602234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:48.602234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:24:48.602234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:24:48.602234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:48.602234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:24:48.602908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:48.602908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:24:48.602908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:24:48.602908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:48.602908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:24:48.602908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:48.602908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:24:48.602908 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:24:48.602908 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:24:48.602908 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:24:48.603325 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:24:48.604055 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:24:48.604055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:48.604694 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:24:48.604694 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:48.604694 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:24:48.604694 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:24:48.604694 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:48.604694 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:24:48.604694 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:48.604694 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:24:48.605435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:48.605435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:24:48.605435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:24:48.605435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:24:48.605435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:48.605435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:24:48.605435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:48.605435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:24:48.605435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:48.605435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:24:48.605435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:48.605435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:24:48.605435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:48.605435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:24:48.605435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:24:48.605981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:48.605981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:24:48.605981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:48.605981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:24:48.605981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:24:48.605981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:48.605981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:24:48.605981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:48.605981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:24:48.606394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:48.606394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:24:48.606394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:24:48.606394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:24:48.606394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:24:48.606394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:24:48.606394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:24:48.606394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:24:48.606394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:24:48.606394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:48.606394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:24:48.606818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:48.606818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:24:48.606818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:48.606818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:24:48.606818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:48.606818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:24:48.606818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:48.606818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:24:48.606818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:24:48.606818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:48.606818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:24:48.606818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:48.606818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:24:48.606818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:24:48.607288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:48.607288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:24:48.607288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:48.607288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:24:48.607587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:48.607587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:24:48.607587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:24:48.607587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:24:48.607587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:24:48.607587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:24:48.607587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:24:48.607587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:24:48.607587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:24:48.607587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:24:48.607587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:24:48.607587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:24:48.607587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:24:48.607587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:24:48.607587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:24:48.607587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:24:48.607587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:24:48.608380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:24:48.608380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:24:48.608380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:24:48.608380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:48.608380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:24:48.608380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:48.608380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:24:48.608380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:48.608380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:24:48.608380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:48.608380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:24:48.608380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:48.608380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:24:48.608380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:24:48.608380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:48.608380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:24:48.608380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:48.608380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:24:48.608380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:24:48.609101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:48.609101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:24:48.609101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:48.609101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:24:48.609101 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:24:48.609101 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:24:48.609101 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:24:48.609101 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:24:48.609427 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:24:48.609427 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:24:48.609427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:48.610060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:24:48.610060 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:48.610060 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:24:48.610060 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:24:48.610060 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:48.610060 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:24:48.610060 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:48.610060 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:24:48.611014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:48.611014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:24:48.611014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:24:48.611014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:24:48.611014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:24:48.611014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:24:48.611014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:24:48.611014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:48.611014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:24:48.611014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:48.611014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:24:48.611014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:48.611338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:24:48.611338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:48.611338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:24:48.611338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:48.611338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:24:48.611338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:24:48.611338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:48.611338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:24:48.611338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:48.611338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:24:48.611338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:24:48.611338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:48.611338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:24:48.611670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:48.611670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:24:48.612530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:48.612530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:24:48.612530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:24:48.612530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:24:48.612530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:24:48.612530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:24:48.612530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:24:48.612530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:24:48.612530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:24:48.612530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:24:48.612530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:24:48.612530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:24:48.612530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:24:48.612530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:24:48.612530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:24:48.612530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:24:48.612867 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:24:48.612867 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:24:48.612867 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:48.612867 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:24:48.612867 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:48.612867 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:24:48.612867 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:48.612867 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:24:48.612867 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:48.612867 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:24:48.612867 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:48.612867 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:24:48.612867 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:24:48.612867 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:48.612867 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:24:48.612867 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:48.612867 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:24:48.612867 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:24:48.612867 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:48.613296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:24:48.613296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:48.613296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:24:53.427288 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:24:53.428094 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:24:53.428094 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:24:53.429029 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:24:53.461281 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:24:53.461602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:53.462279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:24:53.462279 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:53.462279 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:24:53.462279 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:24:53.462279 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:53.462279 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:24:53.462279 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:53.462279 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:24:53.463154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:53.463154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:24:53.463154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:24:53.463154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:24:53.463154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:24:53.463154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:24:53.463154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:24:53.463154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:53.463154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:24:53.463154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:53.463154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:24:53.463154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:53.463772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:24:53.463772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:53.463772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:24:53.463772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:53.463772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:24:53.463772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:24:53.463772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:53.463772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:24:53.463772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:53.463772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:24:53.463772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:24:53.463772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:53.463772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:24:53.464438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:53.464438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:24:53.464438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:53.464438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:24:53.464438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:24:53.464438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:24:53.464438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:24:53.464438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:24:53.464438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:24:53.464438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:24:53.464438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:24:53.464438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:24:53.464438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:24:53.464438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:24:53.464438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:24:53.465164 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:24:53.465164 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:24:53.465164 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:24:53.465164 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:24:53.465164 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:24:53.465164 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:53.465164 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:24:53.465164 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:53.465164 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:24:53.465164 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:53.465164 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:24:53.465164 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:53.465164 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:24:53.465164 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:53.465164 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:24:53.465164 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:24:53.465164 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:53.465164 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:24:53.465164 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:53.466103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:24:53.466103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:24:53.466103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:53.466103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:24:53.466103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:53.466103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:24:53.583672 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:24:53.584389 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:24:53.584704 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:24:53.585859 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:24:53.586101 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:24:53.586275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:53.587255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:24:53.587255 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:53.587255 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:24:53.587255 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:24:53.587255 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:53.587255 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:24:53.587255 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:53.587255 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:24:53.588928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:53.588928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:24:53.588928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:24:53.588928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:24:53.588928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:24:53.588928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:24:53.588928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:24:53.588928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:53.588928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:24:53.588928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:53.588928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:24:53.588928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:53.589318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:24:53.589318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:53.589318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:24:53.589318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:53.589318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:24:53.589318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:24:53.589318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:53.589318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:24:53.589318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:53.589318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:24:53.589318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:24:53.589318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:53.589318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:24:53.589660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:53.589660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:24:53.591596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:53.591596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:24:53.591596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:24:53.591596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:24:53.591596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:24:53.591596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:24:53.591596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:24:53.591596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:24:53.591596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:24:53.591596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:24:53.591596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:24:53.591596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:24:53.591596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:24:53.591596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:24:53.591596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:24:53.591596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:24:53.592076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:24:53.592076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:24:53.592076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:53.592076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:24:53.592076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:53.592076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:24:53.592076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:53.592076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:24:53.592076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:53.592076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:24:53.592076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:53.592076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:24:53.592076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:24:53.592076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:53.592076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:24:53.592076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:53.592076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:24:53.592076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:24:53.592076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:53.592537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:24:53.592537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:53.592537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:24:53.593916 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:24:53.595096 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:24:53.595096 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:24:53.595197 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:24:53.595992 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:24:53.596177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:53.597227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:24:53.597227 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:53.597227 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:24:53.597227 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:24:53.597227 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:53.597227 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:24:53.597227 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:53.597227 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:24:53.598102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:53.598102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:24:53.598102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:24:53.598102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:24:53.598102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:53.598102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:24:53.598102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:53.598102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:24:53.598102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:53.598102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:24:53.598102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:53.598102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:24:53.598102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:53.598102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:24:53.598102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:24:53.598492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:53.598492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:24:53.598492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:53.598492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:24:53.598492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:24:53.598492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:53.598492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:24:53.598492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:53.598492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:24:53.599099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:53.599099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:24:53.599099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:24:53.599099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:24:53.599099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:24:53.599099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:24:53.599099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:24:53.599099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:24:53.599099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:24:53.599099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:24:53.599099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:53.599099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:24:53.599386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:53.599386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:24:53.599386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:53.599386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:24:53.599386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:53.599386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:24:53.599386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:53.599386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:24:53.599386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:24:53.599386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:53.599386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:24:53.599386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:53.599386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:24:53.599386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:24:53.599779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:53.599779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:24:53.599779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:53.599779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:24:53.600493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:53.600493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:24:53.600493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:24:53.600493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:24:53.600493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:24:53.600493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:24:53.600493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:24:53.600493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:24:53.600493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:24:53.600493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:24:53.600493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:24:53.600493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:24:53.600493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:24:53.600493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:24:53.600493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:24:53.600493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:24:53.600873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:24:53.600873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:24:53.600873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:24:53.600873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:24:53.600873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:24:53.600873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:53.600873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:24:53.600873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:53.600873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:24:53.600873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:53.600873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:24:53.600873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:53.600873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:24:53.600873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:53.600873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:24:53.600873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:24:53.600873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:53.600873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:24:53.601370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:53.601370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:24:53.601370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:24:53.601370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:53.601370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:24:53.601370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:53.601370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:24:53.602480 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:24:53.602727 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:24:53.602845 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:24:53.602961 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:24:53.603246 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:24:53.603405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:53.603861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:24:53.603861 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:53.603861 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:24:53.603861 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:24:53.603861 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:53.603861 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:24:53.603861 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:53.603861 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:24:53.604611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:53.604611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:24:53.604611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:24:53.604611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:24:53.604611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:53.604611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:24:53.604611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:53.604611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:24:53.604611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:53.604611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:24:53.604611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:53.604611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:24:53.604611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:53.604611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:24:53.604611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:24:53.606077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:53.606077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:24:53.606077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:53.606077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:24:53.606077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:24:53.606077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:53.606077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:24:53.606077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:53.606077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:24:53.606077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:53.606077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:24:53.606077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:24:53.606077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:24:53.606077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:24:53.606636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:24:53.606636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:24:53.606636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:24:53.606636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:24:53.606636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:53.606636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:24:53.606636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:53.606636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:24:53.606636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:53.606636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:24:53.606636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:53.606636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:24:53.606636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:53.607825 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:24:53.607825 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:24:53.607825 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:53.607825 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:24:53.607825 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:53.607825 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:24:53.607825 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:24:53.607825 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:53.607825 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:24:53.607825 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:53.607825 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:24:53.607825 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:53.608334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:24:53.608334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:24:53.608334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:24:53.608334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:24:53.608334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:24:53.608334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:24:53.608334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:24:53.608334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:24:53.608334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:24:53.608334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:24:53.608334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:24:53.608334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:24:53.608334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:24:53.608334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:24:53.608334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:24:53.608334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:24:53.608334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:24:53.608334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:24:53.608334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:24:53.609001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:53.609001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:24:53.609001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:53.609001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:24:53.609001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:53.609001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:24:53.609001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:53.609001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:24:53.609001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:53.609001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:24:53.609001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:24:53.609001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:53.609001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:24:53.609001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:53.609001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:24:53.609001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:24:53.609001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:53.609001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:24:53.609001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:53.609001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:24:53.609857 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:24:53.609857 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:24:53.609857 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:24:53.609857 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:24:53.609857 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:24:53.610213 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:24:53.610213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:53.610876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:24:53.610876 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:53.610876 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:24:53.610876 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:24:53.610876 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:53.610876 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:24:53.610876 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:53.610876 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:24:53.611913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:53.611913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:24:53.611913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:24:53.611913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:24:53.611913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:24:53.611913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:24:53.611913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:24:53.611913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:53.611913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:24:53.611913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:53.611913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:24:53.611913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:53.612273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:24:53.612273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:53.612273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:24:53.612273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:53.612273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:24:53.612273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:24:53.612273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:53.612273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:24:53.612273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:53.612273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:24:53.612273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:24:53.612273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:53.612273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:24:53.612616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:53.612616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:24:53.613108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:53.613108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:24:53.613108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:24:53.613108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:24:53.613108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:24:53.613108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:24:53.613108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:24:53.613108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:24:53.613108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:24:53.613108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:24:53.613108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:24:53.613108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:24:53.613108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:24:53.613108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:24:53.613108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:24:53.613108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:24:53.613565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:24:53.613565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:24:53.613565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:53.613565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:24:53.613565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:53.613565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:24:53.613565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:53.613565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:24:53.613565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:53.613565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:24:53.613565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:53.613565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:24:53.613565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:24:53.613565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:53.613565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:24:53.613565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:53.613565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:24:53.613565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:24:53.613565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:53.614099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:24:53.614099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:53.614099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:24:58.427927 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:24:58.428812 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:24:58.428812 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:24:58.429611 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:24:58.431359 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG dbcounter [-] [106314] Writing DB stats neutron:SELECT=433 {{(pid=106314) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 09 09:24:58.470791 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:24:58.471120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:58.472230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:24:58.472230 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:58.472230 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:24:58.472230 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:24:58.472230 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:58.472230 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:24:58.472230 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:58.472230 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:24:58.473365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.044 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:58.473365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:24:58.473365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:24:58.473365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:24:58.473365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:24:58.473365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:24:58.473365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:24:58.473365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:58.473365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:24:58.473365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:58.473365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:24:58.473365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:58.474277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:24:58.474277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:58.474277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:24:58.474277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:58.474277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:24:58.474277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:24:58.474277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:58.474277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:24:58.474277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:58.474277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:24:58.474277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:24:58.474277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:58.474277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:24:58.474994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:58.474994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:24:58.474994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:58.474994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:24:58.474994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:24:58.474994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:24:58.474994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:24:58.474994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:24:58.474994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:24:58.474994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:24:58.474994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:24:58.474994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:24:58.474994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:24:58.474994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:24:58.474994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:24:58.475977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:24:58.475977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:24:58.475977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:24:58.475977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:24:58.475977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:24:58.475977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:58.475977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:24:58.475977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:58.475977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:24:58.475977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:58.475977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:24:58.475977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:58.475977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:24:58.475977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:58.475977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:24:58.475977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:24:58.475977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:58.475977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:24:58.475977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:58.477118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:24:58.477118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:24:58.477118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:58.477118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:24:58.477118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:58.477118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:24:58.584365 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:24:58.585146 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:24:58.585146 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:24:58.586005 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:24:58.586368 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:24:58.586589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:58.587466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:24:58.587466 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:58.587466 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:24:58.587466 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:24:58.587466 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:58.587466 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:24:58.587466 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:58.587466 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:24:58.588839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:58.588839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:24:58.588839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:24:58.588839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:24:58.588839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:24:58.588839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:24:58.588839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:24:58.588839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:58.588839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:24:58.588839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:58.588839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:24:58.588839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:58.589521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:24:58.589521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:58.589521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:24:58.589521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:58.589521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:24:58.589521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:24:58.589521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:58.589521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:24:58.589521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:58.589521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:24:58.589521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:24:58.589521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:58.589521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:24:58.590173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:58.590173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:24:58.590481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:58.590481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:24:58.590481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:24:58.590481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:24:58.590481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:24:58.590481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:24:58.590481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:24:58.590481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:24:58.590481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:24:58.590481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:24:58.590481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:24:58.590481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:24:58.590481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:24:58.590481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:24:58.590481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:24:58.590481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:24:58.591225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:24:58.591225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:24:58.591225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:58.591225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:24:58.591225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:58.591225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:24:58.591225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:58.591225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:24:58.591225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:58.591225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:24:58.591225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:58.591225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:24:58.591225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:24:58.591225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:58.591225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:24:58.591225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:58.591225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:24:58.591225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:24:58.591225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:58.591764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:24:58.591764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:58.591764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:24:58.594441 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:24:58.594954 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:24:58.595159 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:24:58.595354 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:24:58.596004 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:24:58.596302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:58.597704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:24:58.597704 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:58.597704 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:24:58.597704 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:24:58.597704 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:58.597704 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:24:58.597704 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:58.597704 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:24:58.601355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:58.601355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:24:58.601355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:24:58.601355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:24:58.601355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:58.601355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:24:58.601355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:58.601355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:24:58.601355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:58.601355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:24:58.601355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:58.601355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:24:58.601355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:58.601355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:24:58.601355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:24:58.602537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:58.602537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:24:58.602537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:58.602537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:24:58.602537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:24:58.602537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:58.602537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:24:58.602537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:58.602537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:24:58.603476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.006 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:58.603476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:24:58.603476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:24:58.603476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:24:58.603476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:24:58.603476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:24:58.603476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:24:58.603476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:24:58.603476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:24:58.603476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:24:58.603476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:58.603476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:24:58.604630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:58.604630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:24:58.604630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:58.604630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:24:58.604630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:58.604630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:24:58.604630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:58.604630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:24:58.604630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:24:58.604630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:58.604630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:24:58.604630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:58.604630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:24:58.604630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:24:58.605323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:58.605323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:24:58.605323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:58.605323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:24:58.606326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:58.606326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:24:58.606326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:24:58.606326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:24:58.606326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:24:58.606326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:24:58.606326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:24:58.606326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:24:58.606326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:24:58.606326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:24:58.606326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:24:58.606326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:24:58.606326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:24:58.606326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:24:58.606326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:24:58.606326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:24:58.607271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:24:58.607271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:24:58.607271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:24:58.607271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:24:58.607271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:24:58.607271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:58.607271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:24:58.607271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:58.607271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:24:58.607271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:58.607271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:24:58.607271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:58.607271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:24:58.607271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:58.607271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:24:58.607271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:24:58.607271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:58.607271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:24:58.607820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:58.607820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:24:58.607820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:24:58.607820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:58.607820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:24:58.607820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:58.607820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:24:58.607820 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:24:58.607820 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:24:58.607820 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:24:58.608136 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:24:58.608587 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:24:58.609088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:58.609602 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:24:58.610388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:24:58.610388 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:58.610388 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:24:58.610388 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:24:58.610388 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:58.610388 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:24:58.610388 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:58.610388 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:24:58.611059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:58.611059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:24:58.611059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:24:58.611059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:24:58.611059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:58.611059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:24:58.611059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:58.611059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:24:58.611059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:58.611059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:24:58.611059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:58.611059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:24:58.611059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:58.611059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:24:58.611059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:24:58.611387 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:58.611387 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:24:58.611387 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:58.611387 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:24:58.611387 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:24:58.611387 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:58.611387 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:24:58.611387 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:58.611387 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:24:58.611754 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:58.611754 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:24:58.611754 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:24:58.611754 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:24:58.611754 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:24:58.611754 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:24:58.611754 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:24:58.611754 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:24:58.611754 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:24:58.611754 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:58.611754 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:24:58.612144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:58.612144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:24:58.612144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:58.612144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:24:58.612144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:58.612144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:24:58.612144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:58.612144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:24:58.612144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:24:58.612144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:58.612144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:24:58.612144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:58.612144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:24:58.612144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:24:58.612424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:58.612424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:24:58.612424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:58.612424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:24:58.612610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:58.612610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:24:58.612610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:24:58.612610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:24:58.612610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:24:58.612610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:24:58.612610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:24:58.612610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:24:58.612610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:24:58.612610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:24:58.612610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:24:58.612610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:24:58.612610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:24:58.612610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:24:58.612610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:24:58.612610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:24:58.612610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:24:58.613302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:24:58.613302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:24:58.613302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:24:58.613302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:58.613302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:24:58.613302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:58.613302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:24:58.613302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:58.613302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:24:58.613302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:58.613302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:24:58.613302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:58.613302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:24:58.613302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:24:58.613302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:58.613302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:24:58.613302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:58.613302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:24:58.613302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:24:58.614972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:58.614972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:24:58.614972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:58.614972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:24:58.614972 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:24:58.614972 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:24:58.614972 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:24:58.614972 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:24:58.615259 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:24:58.615259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:58.615259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:24:58.615259 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:58.615259 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:24:58.615259 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:24:58.615259 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:58.615259 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:24:58.615259 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:58.615259 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:24:58.616846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:58.616846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:24:58.616846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:24:58.616846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:24:58.616846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:24:58.616846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:24:58.616846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:24:58.616846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:58.616846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:24:58.616846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:58.616846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:24:58.616846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:58.617215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:24:58.617215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:58.617215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:24:58.617215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:58.617215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:24:58.617215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:24:58.617215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:58.617215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:24:58.617215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:58.617215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:24:58.617215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:24:58.617215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:58.617215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:24:58.617636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:58.617636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:24:58.618317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:58.618317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:24:58.618317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:24:58.618317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:24:58.618317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:24:58.618317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:24:58.618317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:24:58.618317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:24:58.618317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:24:58.618317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:24:58.618317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:24:58.618317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:24:58.618317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:24:58.618317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:24:58.618317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:24:58.618317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:24:58.618831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:24:58.618831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:24:58.618831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:58.618831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:24:58.618831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:24:58.618831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:24:58.618831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:24:58.618831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:24:58.618831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:24:58.618831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:24:58.618831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:24:58.618831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:24:58.618831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:24:58.618831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:24:58.618831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:24:58.618831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:24:58.618831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:24:58.618831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:24:58.618831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:24:58.619472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:24:58.619472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:24:58.619472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:25:03.428635 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:25:03.429410 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:25:03.429602 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:25:03.431000 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:25:03.476692 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:25:03.476692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:03.476971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:25:03.476971 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:03.476971 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:25:03.476971 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:25:03.476971 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:03.476971 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:25:03.476971 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:03.476971 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:25:03.478374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.047 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:03.478374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:25:03.478374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:25:03.478374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:25:03.478374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:25:03.478374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:25:03.478374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:25:03.478374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:03.478374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:25:03.478374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:03.478374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:25:03.478374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:03.478814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:25:03.478814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:03.478814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:25:03.478814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:03.478814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:25:03.478814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:25:03.478814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:03.478814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:25:03.478814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:03.478814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:25:03.478814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:25:03.478814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:03.478814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:25:03.479180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:03.479180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:25:03.479180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:03.479180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:25:03.479180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:25:03.479180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:25:03.479180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:25:03.479180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:25:03.479180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:25:03.479180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:25:03.479180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:25:03.479180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:25:03.479180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:25:03.479180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:25:03.479180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:25:03.479632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:25:03.479632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:25:03.479632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:25:03.479632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:25:03.479632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:25:03.479632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:03.479632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:25:03.479632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:03.479632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:25:03.479632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:03.479632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:25:03.479632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:03.479632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:25:03.479632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:03.479632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:25:03.479632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:25:03.479632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:03.479632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:25:03.479632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:03.480137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:25:03.480137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:25:03.480137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:03.480137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:25:03.480137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:03.480137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:25:03.584939 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:25:03.585634 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:25:03.585931 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:25:03.587846 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:25:03.587846 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:25:03.587846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:03.588233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:25:03.588233 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:03.588233 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:25:03.588233 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:25:03.588233 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:03.588233 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:25:03.588233 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:03.588233 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:25:03.590005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:03.590005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:25:03.590005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:25:03.590005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:25:03.590005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:25:03.590005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:25:03.590005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:25:03.590005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:03.590005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:25:03.590005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:03.590005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:25:03.590005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:03.590471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:25:03.590471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:03.590471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:25:03.590471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:03.590471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:25:03.590471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:25:03.590471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:03.590471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:25:03.590471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:03.590471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:25:03.590471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:25:03.590471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:03.590471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:25:03.590856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:03.590856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:25:03.592260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:03.592260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:25:03.592260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:25:03.592260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:25:03.592260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:25:03.592260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:25:03.592260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:25:03.592260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:25:03.592260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:25:03.592260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:25:03.592260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:25:03.592260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:25:03.592260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:25:03.592260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:25:03.592260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:25:03.592260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:25:03.592942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:25:03.592942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:25:03.592942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:03.592942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:25:03.592942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:03.592942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:25:03.592942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:03.592942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:25:03.592942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:03.592942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:25:03.592942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:03.592942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:25:03.592942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:25:03.592942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:03.592942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:25:03.592942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:03.592942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:25:03.592942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:25:03.592942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:03.593936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:25:03.593936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:03.593936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:25:03.594991 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:25:03.595320 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:25:03.595465 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:25:03.595573 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:25:03.596096 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:25:03.596399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:03.597130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:25:03.597130 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:03.597130 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:25:03.597130 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:25:03.597130 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:03.597130 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:25:03.597130 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:03.597130 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:25:03.597932 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:03.597932 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:25:03.597932 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:25:03.597932 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:25:03.597932 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:03.597932 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:25:03.597932 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:03.597932 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:25:03.597932 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:03.597932 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:25:03.597932 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:03.597932 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:25:03.597932 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:03.597932 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:25:03.597932 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:25:03.598566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:03.598566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:25:03.598566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:03.598566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:25:03.598566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:25:03.598566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:03.598566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:25:03.598566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:03.598566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:25:03.599100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:03.599100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:25:03.599100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:25:03.599100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:25:03.599100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:25:03.599100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:25:03.599100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:25:03.599100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:25:03.599100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:25:03.599100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:25:03.599100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:03.599100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:25:03.599996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:03.599996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:25:03.599996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:03.599996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:25:03.599996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:03.599996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:25:03.599996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:03.599996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:25:03.599996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:25:03.599996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:03.599996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:25:03.599996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:03.599996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:25:03.599996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:25:03.600497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:03.600497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:25:03.600497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:03.600497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:25:03.600497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:03.600497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:25:03.600497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:25:03.600497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:25:03.600497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:25:03.600497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:25:03.600497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:25:03.600497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:25:03.600497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:25:03.600497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:25:03.600990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:25:03.600990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:25:03.600990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:25:03.600990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:25:03.600990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:25:03.600990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:25:03.600990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:25:03.600990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:25:03.600990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:25:03.600990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:25:03.600990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:25:03.600990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:03.600990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:25:03.600990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:03.600990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:25:03.600990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:03.600990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:25:03.600990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:03.600990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:25:03.601547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:03.601547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:25:03.601547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:25:03.601547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:03.601547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:25:03.601547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:03.601547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:25:03.601547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:25:03.601547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:03.601547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:25:03.601547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:03.601547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:25:03.607762 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:25:03.608149 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:25:03.608269 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:25:03.608410 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:25:03.608941 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:25:03.609289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:03.610180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:25:03.610180 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:03.610180 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:25:03.610180 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:25:03.610180 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:03.610180 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:25:03.610180 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:03.610180 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:25:03.611029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:03.611029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:25:03.611029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:25:03.611029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:25:03.611029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:03.611029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:25:03.611029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:03.611029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:25:03.611029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:03.611029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:25:03.611029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:03.611029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:25:03.611029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:03.611029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:25:03.611029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:25:03.611708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:03.611708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:25:03.611708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:03.611708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:25:03.611708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:25:03.611708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:03.611708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:25:03.611708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:03.611708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:25:03.612114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:03.612114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:25:03.612114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:25:03.612114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:25:03.612114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:25:03.612114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:25:03.612114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:25:03.612114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:25:03.612114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:25:03.612114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:03.612114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:25:03.612614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:03.612614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:25:03.612614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:03.612614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:25:03.612614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:03.612614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:25:03.612614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:03.612614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:25:03.612614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:25:03.612614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:03.612614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:25:03.612614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:03.612614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:25:03.612614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:25:03.613227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:03.613227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:25:03.613227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:03.613227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:25:03.613227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:03.613227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:25:03.613227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:25:03.613227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:25:03.613227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:25:03.613227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:25:03.613227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:25:03.613227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:25:03.613227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:25:03.613227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:25:03.614128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:25:03.614128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:25:03.614128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:25:03.614128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:25:03.614128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:25:03.614128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:25:03.614128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:25:03.614128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:25:03.614128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:25:03.614128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:25:03.614128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:03.614128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:25:03.614128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:03.614128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:25:03.614128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:03.614128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:25:03.614128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:03.614128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:25:03.614128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:03.615127 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:25:03.615127 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:25:03.615127 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:03.615127 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:25:03.615127 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:03.615127 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:25:03.615127 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:25:03.615127 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:03.615127 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:25:03.615127 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:03.615127 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:25:03.615127 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:25:03.615127 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:25:03.615850 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:25:03.615850 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:25:03.615850 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:25:03.615850 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:25:03.615850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:03.616094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:25:03.616094 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:03.616094 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:25:03.616094 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:25:03.616094 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:03.616094 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:25:03.616094 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:03.616094 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:25:03.616886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:03.616886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:25:03.616886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:25:03.616886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:25:03.616886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:25:03.616886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:25:03.616886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:25:03.616886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:03.616886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:25:03.616886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:03.616886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:25:03.616886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:03.617536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:25:03.617536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:03.617536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:25:03.617536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:03.617536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:25:03.617536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:25:03.617536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:03.617536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:25:03.617536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:03.617536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:25:03.617536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:25:03.617536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:03.617536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:25:03.618181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:03.618181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:25:03.618181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:03.618181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:25:03.618181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:25:03.618181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:25:03.618181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:25:03.618181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:25:03.618181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:25:03.618181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:25:03.618181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:25:03.618181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:25:03.618181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:25:03.618181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:25:03.618181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:25:03.619044 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:25:03.619044 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:25:03.619044 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:25:03.619044 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:25:03.619044 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:25:03.619044 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:03.619044 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:25:03.619044 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:03.619044 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:25:03.619044 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:03.619044 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:25:03.619044 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:03.619044 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:25:03.619044 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:03.619044 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:25:03.619044 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:25:03.619044 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:03.619044 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:25:03.619044 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:03.619848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:25:03.619848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:25:03.619848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:03.619848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:25:03.619848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:03.619848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:25:08.429456 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:25:08.430339 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:25:08.430463 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:25:08.431338 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:25:08.469553 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:25:08.469908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:08.470584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:25:08.470584 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:08.470584 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:25:08.470584 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:25:08.470584 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:08.470584 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:25:08.470584 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:08.470584 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:25:08.471404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:08.471404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:25:08.471404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:25:08.471404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:25:08.471404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:25:08.471404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:25:08.471404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:25:08.471404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:08.471404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:25:08.471404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:08.471404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:25:08.471404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:08.472040 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:25:08.472040 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:08.472040 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:25:08.472040 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:08.472040 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:25:08.472040 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:25:08.472040 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:08.472040 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:25:08.472040 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:08.472040 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:25:08.472040 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:25:08.472040 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:08.472040 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:25:08.472534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:08.472534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:25:08.472534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:08.472534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:25:08.472534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:25:08.472534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:25:08.472534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:25:08.472534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:25:08.472534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:25:08.472534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:25:08.472534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:25:08.472534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:25:08.472534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:25:08.472534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:25:08.472534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:25:08.473169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:25:08.473169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:25:08.473169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:25:08.473169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:25:08.473169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:25:08.473169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:08.473169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:25:08.473169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:08.473169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:25:08.473169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:08.473169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:25:08.473169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:08.473169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:25:08.473169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:08.473169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:25:08.473169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:25:08.473169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:08.473169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:25:08.473169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:08.473902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:25:08.473902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:25:08.473902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:08.473902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:25:08.473902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:08.473902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:25:08.585474 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:25:08.586023 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:25:08.586309 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:25:08.587173 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:25:08.587394 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:25:08.587524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:08.588299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:25:08.588299 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:08.588299 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:25:08.588299 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:25:08.588299 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:08.588299 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:25:08.588299 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:08.588299 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:25:08.590007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:08.590007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:25:08.590007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:25:08.590007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:25:08.590007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:25:08.590007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:25:08.590007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:25:08.590007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:08.590007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:25:08.590007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:08.590007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:25:08.590007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:08.590608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:25:08.590608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:08.590608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:25:08.590608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:08.590608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:25:08.590608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:25:08.590608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:08.590608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:25:08.590608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:08.590608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:25:08.590608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:25:08.590608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:08.590608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:25:08.591203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:08.591203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:25:08.592458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:08.592458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:25:08.592458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:25:08.592458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:25:08.592458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:25:08.592458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:25:08.592458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:25:08.592458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:25:08.592458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:25:08.592458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:25:08.592458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:25:08.592458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:25:08.592458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:25:08.592458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:25:08.592458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:25:08.592458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:25:08.593280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:25:08.593280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:25:08.593280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:08.593280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:25:08.593280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:08.593280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:25:08.593280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:08.593280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:25:08.593280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:08.593280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:25:08.593280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:08.593280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:25:08.593280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:25:08.593280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:08.593280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:25:08.593280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:08.593280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:25:08.593280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:25:08.593280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:08.594145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:25:08.594145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:08.594145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:25:08.595279 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:25:08.595522 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:25:08.595662 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:25:08.595911 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:25:08.596197 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:25:08.596370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:08.596852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:25:08.596852 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:08.596852 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:25:08.596852 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:25:08.596852 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:08.596852 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:25:08.596852 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:08.596852 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:25:08.597711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:08.597711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:25:08.597711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:25:08.597711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:25:08.597711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:08.597711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:25:08.597711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:08.597711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:25:08.597711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:08.597711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:25:08.597711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:08.597711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:25:08.597711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:08.597711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:25:08.597711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:25:08.598732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:08.598732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:25:08.598732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:08.598732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:25:08.598732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:25:08.598732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:08.598732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:25:08.598732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:08.598732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:25:08.598732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:08.598732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:25:08.598732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:25:08.598732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:25:08.598732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:25:08.599595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:25:08.599595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:25:08.599595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:25:08.599595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:25:08.599595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:25:08.599595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:08.599595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:25:08.599595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:08.599595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:25:08.599595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:08.599595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:25:08.599595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:08.599595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:25:08.600393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:08.600393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:25:08.600393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:25:08.600393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:08.600393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:25:08.600393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:08.600393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:25:08.600393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:25:08.600393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:08.600393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:25:08.600393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:08.600393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:25:08.600999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:08.600999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:25:08.600999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:25:08.600999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:25:08.600999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:25:08.600999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:25:08.600999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:25:08.600999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:25:08.600999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:25:08.600999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:25:08.600999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:25:08.600999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:25:08.600999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:25:08.600999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:25:08.600999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:25:08.600999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:25:08.601847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:25:08.601847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:25:08.601847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:25:08.601847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:25:08.601847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:25:08.601847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:08.601847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:25:08.601847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:08.601847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:25:08.601847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:08.601847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:25:08.601847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:08.601847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:25:08.601847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:08.601847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:25:08.601847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:25:08.601847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:08.601847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:25:08.602994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:08.602994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:25:08.602994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:25:08.602994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:08.602994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:25:08.602994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:08.602994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:25:08.608254 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:25:08.608533 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:25:08.608683 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:25:08.609058 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:25:08.609550 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:25:08.610241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:08.610993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:25:08.610993 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:08.610993 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:25:08.610993 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:25:08.610993 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:08.610993 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:25:08.610993 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:08.610993 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:25:08.611899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:08.611899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:25:08.611899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:25:08.611899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:25:08.611899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:08.611899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:25:08.611899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:08.611899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:25:08.611899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:08.611899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:25:08.611899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:08.611899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:25:08.611899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:08.611899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:25:08.611899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:25:08.612558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:08.612558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:25:08.612558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:08.612558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:25:08.612558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:25:08.612558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:08.612558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:25:08.612558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:08.612558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:25:08.613367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:08.613367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:25:08.613367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:25:08.613367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:25:08.613367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:25:08.613367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:25:08.613367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:25:08.613367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:25:08.613367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:25:08.613367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:08.613367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:25:08.613878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:08.613878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:25:08.613878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:08.613878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:25:08.613878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:08.613878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:25:08.613878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:08.613878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:25:08.613878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:25:08.613878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:08.613878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:25:08.613878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:08.613878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:25:08.613878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:25:08.614350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:08.614350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:25:08.614350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:08.614350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:25:08.615945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:08.615945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:25:08.615945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:25:08.615945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:25:08.615945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:25:08.615945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:25:08.615945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:25:08.615945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:25:08.615945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:25:08.615945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:25:08.615945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:25:08.615945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:25:08.615945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:25:08.615945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:25:08.615945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:25:08.615945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:25:08.615945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:25:08.616664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:25:08.616664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:25:08.616664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:25:08.616664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:08.616664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:25:08.616664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:08.616664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:25:08.616664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:08.616664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:25:08.616664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:08.616664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:25:08.616664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:08.616664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:25:08.616664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:25:08.616664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:08.616664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:25:08.616664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:08.616664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:25:08.616664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:25:08.617326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:08.617326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:25:08.617326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:08.617326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:25:08.617326 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:25:08.617326 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:25:08.617326 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:25:08.617610 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:25:08.618227 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:25:08.618227 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:25:08.618390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:08.619069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:25:08.619069 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:08.619069 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:25:08.619069 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:25:08.619069 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:08.619069 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:25:08.619069 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:08.619069 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:25:08.619702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:08.619702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:25:08.619702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:25:08.619702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:25:08.619702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:25:08.619702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:25:08.619702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:25:08.619702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:08.619702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:25:08.619702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:08.619702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:25:08.619702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:08.620414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:25:08.620414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:08.620414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:25:08.620414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:08.620414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:25:08.620414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:25:08.620414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:08.620414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:25:08.620414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:08.620414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:25:08.620414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:25:08.620414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:08.620414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:25:08.620963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:08.620963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:25:08.620963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:08.620963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:25:08.620963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:25:08.620963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:25:08.620963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:25:08.620963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:25:08.620963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:25:08.620963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:25:08.620963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:25:08.620963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:25:08.620963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:25:08.620963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:25:08.620963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:25:08.621588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:25:08.621588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:25:08.621588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:25:08.621588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:25:08.621588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:25:08.621588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:08.621588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:25:08.621588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:08.621588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:25:08.621588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:08.621588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:25:08.621588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:08.621588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:25:08.621588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:08.621588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:25:08.621588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:25:08.621588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:08.621588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:25:08.621588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:08.622223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:25:08.622223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:25:08.622223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:08.622223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:25:08.622223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:08.622223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:25:13.430804 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:25:13.431653 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:25:13.431653 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:25:13.432370 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:25:13.467356 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:25:13.467583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:13.468138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:25:13.468138 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:13.468138 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:25:13.468138 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:25:13.468138 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:13.468138 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:25:13.468138 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:13.468138 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:25:13.468816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:13.468816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:25:13.468816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:25:13.468816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:25:13.468816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:25:13.468816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:25:13.468816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:25:13.468816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:13.468816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:25:13.468816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:13.468816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:25:13.468816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:13.469277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:25:13.469277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:13.469277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:25:13.469277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:13.469277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:25:13.469277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:25:13.469277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:13.469277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:25:13.469277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:13.469277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:25:13.469277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:25:13.469277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:13.469277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:25:13.469680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:13.469680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:25:13.469680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:13.469680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:25:13.469680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:25:13.469680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:25:13.469680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:25:13.469680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:25:13.469680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:25:13.469680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:25:13.469680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:25:13.469680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:25:13.469680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:25:13.469680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:25:13.469680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:25:13.470171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:25:13.470171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:25:13.470171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:25:13.470171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:25:13.470171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:25:13.470171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:13.470171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:25:13.470171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:13.470171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:25:13.470171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:13.470171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:25:13.470171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:13.470171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:25:13.470171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:13.470171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:25:13.470171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:25:13.470171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:13.470171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:25:13.470171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:13.470683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:25:13.470683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:25:13.470683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:13.470683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:25:13.470683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:13.470683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:25:13.586060 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:25:13.586603 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:25:13.586850 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:25:13.587971 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:25:13.588371 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:25:13.588902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:13.589475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:25:13.589475 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:13.589475 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:25:13.589475 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:25:13.589475 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:13.589475 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:25:13.589475 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:13.589475 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:25:13.590707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:13.590707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:25:13.590707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:25:13.590707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:25:13.590707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:25:13.590707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:25:13.590707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:25:13.590707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:13.590707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:25:13.590707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:13.590707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:25:13.590707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:13.591301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:25:13.591301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:13.591301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:25:13.591301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:13.591301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:25:13.591301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:25:13.591301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:13.591301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:25:13.591301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:13.591301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:25:13.591301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:25:13.591301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:13.591301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:25:13.591633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:13.591633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:25:13.591764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:13.591764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:25:13.591764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:25:13.591764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:25:13.591764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:25:13.591764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:25:13.591764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:25:13.591764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:25:13.591764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:25:13.591764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:25:13.591764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:25:13.591764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:25:13.591764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:25:13.591764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:25:13.591764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:25:13.591764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:25:13.592275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:25:13.592275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:25:13.592275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:13.592275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:25:13.592275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:13.592275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:25:13.592275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:13.592275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:25:13.592275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:13.592275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:25:13.592275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:13.592275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:25:13.592275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:25:13.592275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:13.592275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:25:13.592275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:13.592275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:25:13.592275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:25:13.592275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:13.592796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:25:13.592796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:13.592796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:25:13.595622 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:25:13.595981 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:25:13.596103 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:25:13.596232 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:25:13.596761 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:25:13.597052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:13.597764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:25:13.597764 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:13.597764 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:25:13.597764 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:25:13.597764 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:13.597764 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:25:13.597764 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:13.597764 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:25:13.598644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:13.598644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:25:13.598644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:25:13.598644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:25:13.598644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:13.598644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:25:13.598644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:13.598644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:25:13.598644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:13.598644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:25:13.598644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:13.598644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:25:13.598644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:13.598644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:25:13.598644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:25:13.599151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:13.599151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:25:13.599151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:13.599151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:25:13.599151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:25:13.599151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:13.599151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:25:13.599151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:13.599151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:25:13.599566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:13.599566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:25:13.599566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:25:13.599566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:25:13.599566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:25:13.599566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:25:13.599566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:25:13.599566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:25:13.599566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:25:13.599566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:25:13.599566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:13.599566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:25:13.599962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:13.599962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:25:13.599962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:13.599962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:25:13.599962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:13.599962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:25:13.599962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:13.599962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:25:13.599962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:25:13.599962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:13.599962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:25:13.599962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:13.599962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:25:13.599962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:25:13.600326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:13.600326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:25:13.600326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:13.600326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:25:13.600720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:13.600720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:25:13.600720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:25:13.600720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:25:13.600720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:25:13.600720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:25:13.600720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:25:13.600720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:25:13.600720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:25:13.600720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:25:13.600720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:25:13.600720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:25:13.600720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:25:13.600720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:25:13.600720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:25:13.600720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:25:13.601238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:25:13.601238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:25:13.601238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:25:13.601238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:25:13.601238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:25:13.601238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:13.601238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:25:13.601238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:13.601238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:25:13.601238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:13.601238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:25:13.601238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:13.601238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:25:13.601238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:13.601238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:25:13.601238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:25:13.601238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:13.601238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:25:13.601707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:13.601707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:25:13.601707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:25:13.601707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:13.601707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:25:13.601707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:13.601707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:25:13.608677 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:25:13.609080 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:25:13.609302 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:25:13.609513 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:25:13.610262 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:25:13.610522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:13.611271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:25:13.611271 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:13.611271 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:25:13.611271 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:25:13.611271 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:13.611271 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:25:13.611271 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:13.611271 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:25:13.612510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:13.612510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:25:13.612510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:25:13.612510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:25:13.612510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:13.612510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:25:13.612510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:13.612510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:25:13.612510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:13.612510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:25:13.612510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:13.612510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:25:13.612510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:13.612510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:25:13.612510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:25:13.613280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:13.613280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:25:13.613280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:13.613280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:25:13.613280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:25:13.613280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:13.613280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:25:13.613280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:13.613280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:25:13.614012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:13.614012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:25:13.614012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:25:13.614012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:25:13.614012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:25:13.614012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:25:13.614012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:25:13.614012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:25:13.614012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:25:13.614012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:13.614012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:25:13.614594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:13.614594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:25:13.614594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:13.614594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:25:13.614594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:13.614594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:25:13.614594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:13.614594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:25:13.614594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:25:13.614594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:13.614594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:25:13.614594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:13.614594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:25:13.614594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:25:13.615323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:13.615323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:25:13.615323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:13.615323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:25:13.616141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:13.616141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:25:13.616141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:25:13.616141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:25:13.616141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:25:13.616141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:25:13.616141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:25:13.616141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:25:13.616141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:25:13.616141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:25:13.616141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:25:13.616141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:25:13.616141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:25:13.616141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:25:13.616141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:25:13.616141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:25:13.616141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:25:13.617224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:25:13.617224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:25:13.617224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:25:13.617224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:13.617224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:25:13.617224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:13.617224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:25:13.617224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:13.617224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:25:13.617224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:13.617224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:25:13.617224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:13.617224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:25:13.617224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:25:13.617224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:13.617224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:25:13.617224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:13.617224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:25:13.617224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:25:13.618265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:13.618265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:25:13.618265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:13.618265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:25:13.618265 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:25:13.618265 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:25:13.618265 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:25:13.618265 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:25:13.618888 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:25:13.618888 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:25:13.618888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:13.618888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:25:13.618888 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:13.618888 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:25:13.618888 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:25:13.618888 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:13.618888 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:25:13.618888 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:13.618888 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:25:13.619618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:13.619618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:25:13.619618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:25:13.619618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:25:13.619618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:25:13.619618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:25:13.619618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:25:13.619618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:13.619618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:25:13.619618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:13.619618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:25:13.619618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:13.620352 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:25:13.620352 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:13.620352 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:25:13.620352 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:13.620352 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:25:13.620352 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:25:13.620352 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:13.620352 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:25:13.620352 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:13.620352 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:25:13.620352 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:25:13.620352 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:13.620352 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:25:13.621097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:13.621097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:25:13.621097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:13.621097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:25:13.621097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:25:13.621097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:25:13.621097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:25:13.621097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:25:13.621097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:25:13.621097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:25:13.621097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:25:13.621097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:25:13.621097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:25:13.621097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:25:13.621097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:25:13.622085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:25:13.622085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:25:13.622085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:25:13.622085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:25:13.622085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:25:13.622085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:13.622085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:25:13.622085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:13.622085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:25:13.622085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:13.622085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:25:13.622085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:13.622085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:25:13.622085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:13.622085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:25:13.622085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:25:13.622085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:13.622085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:25:13.622085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:13.622959 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:25:13.622959 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:25:13.622959 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:13.622959 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:25:13.622959 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:13.622959 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:25:18.431120 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:25:18.431835 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:25:18.432001 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:25:18.434012 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:25:18.477508 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:25:18.478074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:18.479992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:25:18.479992 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:18.479992 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:25:18.479992 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:25:18.479992 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:18.479992 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:25:18.479992 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:18.479992 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:25:18.480450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.047 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:18.480450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:25:18.480450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:25:18.480450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:25:18.480450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:25:18.480450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:25:18.480450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:25:18.480450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:18.480450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:25:18.480450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:18.480450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:25:18.480450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:18.480911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:25:18.480911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:18.480911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:25:18.480911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:18.480911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:25:18.480911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:25:18.480911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:18.480911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:25:18.480911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:18.480911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:25:18.480911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:25:18.480911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:18.480911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:25:18.481296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:18.481296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:25:18.481803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:18.481803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:25:18.481803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:25:18.481803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:25:18.481803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:25:18.481803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:25:18.481803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:25:18.481803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:25:18.481803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:25:18.481803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:25:18.481803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:25:18.481803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:25:18.481803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:25:18.481803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:25:18.481803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:25:18.481803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:25:18.482623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:25:18.482623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:25:18.482623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:18.482623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:25:18.482623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:18.482623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:25:18.482623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:18.482623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:25:18.482623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:18.482623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:25:18.482623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:18.482623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:25:18.482623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:25:18.482623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:18.482623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:25:18.482623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:18.482623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:25:18.482623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:25:18.482623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:18.483314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:25:18.483314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:18.483314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:25:18.586686 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:25:18.587439 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:25:18.587631 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:25:18.589262 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:25:18.589715 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:25:18.590098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:18.591263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:25:18.591263 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:18.591263 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:25:18.591263 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:25:18.591263 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:18.591263 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:25:18.591263 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:18.591263 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:25:18.593198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:18.593198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:25:18.593198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:25:18.593198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:25:18.593198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:25:18.593198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:25:18.593198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:25:18.593198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:18.593198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:25:18.593198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:18.593198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:25:18.593198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:18.593896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:25:18.593896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:18.593896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:25:18.593896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:18.593896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:25:18.593896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:25:18.593896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:18.593896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:25:18.593896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:18.593896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:25:18.593896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:25:18.593896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:18.593896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:25:18.594433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:18.594433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:25:18.594628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:18.594628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:25:18.594628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:25:18.594628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:25:18.594628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:25:18.594628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:25:18.594628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:25:18.594628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:25:18.594628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:25:18.594628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:25:18.594628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:25:18.594628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:25:18.594628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:25:18.594628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:25:18.594628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:25:18.594628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:25:18.595495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:25:18.595495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:25:18.595495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:18.595495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:25:18.595495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:18.595495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:25:18.595495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:18.595495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:25:18.595495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:18.595495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:25:18.595495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:18.595495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:25:18.595495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:25:18.595495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:18.595495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:25:18.595495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:18.595495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:25:18.595495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:25:18.595495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:18.596388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:25:18.596388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:18.596388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:25:18.596388 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:25:18.596388 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:25:18.596632 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:25:18.596632 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:25:18.597045 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:25:18.597232 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:18.597777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:25:18.597777 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:18.597777 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:25:18.597777 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:25:18.597777 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:18.597777 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:25:18.597777 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:18.597777 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:25:18.598497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:18.598497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:25:18.598497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:25:18.598497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:25:18.598497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:18.598497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:25:18.598497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:18.598497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:25:18.598497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:18.598497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:25:18.598497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:18.598497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:25:18.598497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:18.598497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:25:18.598497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:25:18.599408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:18.599408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:25:18.599408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:18.599408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:25:18.599408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:25:18.599408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:18.599408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:25:18.599408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:18.599408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:25:18.599408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:18.599408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:25:18.599408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:25:18.599408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:25:18.599408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:25:18.600224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:25:18.600224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:25:18.600224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:25:18.600224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:25:18.600224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:25:18.600224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:18.600224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:25:18.600224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:18.600224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:25:18.600224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:18.600224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:25:18.600224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:18.600224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:25:18.600891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:18.600891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:25:18.600891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:25:18.600891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:18.600891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:25:18.600891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:18.600891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:25:18.600891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:25:18.600891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:18.600891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:25:18.600891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:18.600891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:25:18.601271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:18.601271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:25:18.601271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:25:18.601271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:25:18.601271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:25:18.601271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:25:18.601271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:25:18.601271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:25:18.601271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:25:18.601271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:25:18.601271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:25:18.601271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:25:18.601271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:25:18.601271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:25:18.601271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:25:18.601271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:25:18.601906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:25:18.601906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:25:18.601906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:25:18.601906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:25:18.601906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:25:18.601906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:18.601906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:25:18.601906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:18.601906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:25:18.601906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:18.601906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:25:18.601906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:18.601906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:25:18.601906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:18.601906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:25:18.601906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:25:18.601906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:18.601906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:25:18.602505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:18.602505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:25:18.602505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:25:18.602505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:18.602505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:25:18.602505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:18.602505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:25:18.609102 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:25:18.609320 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:25:18.609422 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:25:18.609606 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:25:18.610156 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:25:18.610483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:18.611197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:25:18.611197 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:18.611197 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:25:18.611197 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:25:18.611197 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:18.611197 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:25:18.611197 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:18.611197 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:25:18.612628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:18.612628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:25:18.612628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:25:18.612628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:25:18.612628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:18.612628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:25:18.612628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:18.612628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:25:18.612628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:18.612628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:25:18.612628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:18.612628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:25:18.612628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:18.612628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:25:18.612628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:25:18.613150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:18.613150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:25:18.613150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:18.613150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:25:18.613150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:25:18.613150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:18.613150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:25:18.613150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:18.613150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:25:18.614153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:18.614153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:25:18.614153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:25:18.614153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:25:18.614153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:25:18.614153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:25:18.614153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:25:18.614153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:25:18.614153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:25:18.614153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:18.614153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:25:18.614540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:18.614540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:25:18.614540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:18.614540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:25:18.614540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:18.614540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:25:18.614540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:18.614540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:25:18.614540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:25:18.614540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:18.614540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:25:18.614540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:18.614540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:25:18.614540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:25:18.614944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:18.614944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:25:18.614944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:18.614944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:25:18.615988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:18.615988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:25:18.615988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:25:18.615988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:25:18.615988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:25:18.615988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:25:18.615988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:25:18.615988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:25:18.615988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:25:18.615988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:25:18.615988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:25:18.615988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:25:18.615988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:25:18.615988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:25:18.615988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:25:18.615988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:25:18.615988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:25:18.616496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:25:18.616496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:25:18.616496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:25:18.616496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:18.616496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:25:18.616496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:18.616496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:25:18.616496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:18.616496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:25:18.616496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:18.616496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:25:18.616496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:18.616496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:25:18.616496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:25:18.616496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:18.616496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:25:18.616496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:18.616496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:25:18.616496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:25:18.616982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:18.616982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:25:18.616982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:18.616982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:25:18.616982 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:25:18.617204 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:25:18.617293 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:25:18.617454 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:25:18.617916 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:25:18.618226 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:25:18.618442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:18.619141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:25:18.619141 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:18.619141 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:25:18.619141 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:25:18.619141 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:18.619141 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:25:18.619141 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:18.619141 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:25:18.620026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:18.620026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:25:18.620026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:25:18.620026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:25:18.620026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:25:18.620026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:25:18.620026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:25:18.620026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:18.620026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:25:18.620026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:18.620026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:25:18.620026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:18.620418 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:25:18.620418 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:18.620418 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:25:18.620418 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:18.620418 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:25:18.620418 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:25:18.620418 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:18.620418 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:25:18.620418 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:18.620418 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:25:18.620418 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:25:18.620418 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:18.620418 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:25:18.620739 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:18.620739 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:25:18.621113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:18.621113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:25:18.621113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:25:18.621113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:25:18.621113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:25:18.621113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:25:18.621113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:25:18.621113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:25:18.621113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:25:18.621113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:25:18.621113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:25:18.621113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:25:18.621113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:25:18.621113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:25:18.621113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:25:18.621113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:25:18.621564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:25:18.621564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:25:18.621564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:18.621564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:25:18.621564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:18.621564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:25:18.621564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:18.621564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:25:18.621564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:18.621564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:25:18.621564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:18.621564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:25:18.621564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:25:18.621564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:18.621564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:25:18.621564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:18.621564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:25:18.621564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:25:18.621564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:18.622103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:25:18.622103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:18.622103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:25:23.431689 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:25:23.432699 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:25:23.432699 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:25:23.433104 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:25:23.467179 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:25:23.467436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:23.468144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:25:23.468144 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:23.468144 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:25:23.468144 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:25:23.468144 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:23.468144 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:25:23.468144 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:23.468144 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:25:23.469363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:23.469363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:25:23.469363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:25:23.469363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:25:23.469363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:25:23.469363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:25:23.469363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:25:23.469363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:23.469363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:25:23.469363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:23.469363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:25:23.469363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:23.470026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:25:23.470026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:23.470026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:25:23.470026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:23.470026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:25:23.470026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:25:23.470026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:23.470026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:25:23.470026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:23.470026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:25:23.470026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:25:23.470026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:23.470026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:25:23.470713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:23.470713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:25:23.471055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:23.471055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:25:23.471055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:25:23.471055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:25:23.471055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:25:23.471055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:25:23.471055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:25:23.471055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:25:23.471055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:25:23.471055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:25:23.471055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:25:23.471055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:25:23.471055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:25:23.471055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:25:23.471055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:25:23.471055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:25:23.471632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:25:23.471632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:25:23.471632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:23.471632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:25:23.471632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:23.471632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:25:23.471632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:23.471632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:25:23.471632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:23.471632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:25:23.471632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:23.471632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:25:23.471632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:25:23.471632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:23.471632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:25:23.471632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:23.471632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:25:23.471632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:25:23.471632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:23.472249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:25:23.472249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:23.472249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:25:23.587373 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:25:23.587937 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:25:23.588221 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:25:23.589349 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:25:23.589771 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:25:23.590123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:23.591313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:25:23.591313 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:23.591313 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:25:23.591313 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:25:23.591313 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:23.591313 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:25:23.591313 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:23.591313 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:25:23.592968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:23.592968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:25:23.592968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:25:23.592968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:25:23.592968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:25:23.592968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:25:23.592968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:25:23.592968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:23.592968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:25:23.592968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:23.592968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:25:23.592968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:23.593593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:25:23.593593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:23.593593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:25:23.593593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:23.593593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:25:23.593593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:25:23.593593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:23.593593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:25:23.593593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:23.593593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:25:23.593593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:25:23.593593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:23.593593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:25:23.594113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:23.594113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:25:23.595234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:23.595234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:25:23.595234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:25:23.595234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:25:23.595234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:25:23.595234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:25:23.595234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:25:23.595234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:25:23.595234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:25:23.595234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:25:23.595234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:25:23.595234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:25:23.595234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:25:23.595234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:25:23.595234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:25:23.595234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:25:23.596045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:25:23.596045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:25:23.596045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:23.596045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:25:23.596045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:23.596045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:25:23.596045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:23.596045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:25:23.596045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:23.596045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:25:23.596045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:23.596045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:25:23.596045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:25:23.596045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:23.596045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:25:23.596045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:23.596045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:25:23.596045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:25:23.596045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:23.597080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:25:23.597080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:23.597080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:25:23.597080 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:25:23.597080 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:25:23.597080 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:25:23.597522 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:25:23.597660 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:25:23.597842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:23.598265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:25:23.598265 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:23.598265 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:25:23.598265 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:25:23.598265 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:23.598265 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:25:23.598265 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:23.598265 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:25:23.599223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:23.599223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:25:23.599223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:25:23.599223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:25:23.599223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:23.599223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:25:23.599223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:23.599223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:25:23.599223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:23.599223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:25:23.599223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:23.599223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:25:23.599223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:23.599223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:25:23.599223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:25:23.600145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:23.600145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:25:23.600145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:23.600145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:25:23.600145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:25:23.600145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:23.600145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:25:23.600145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:23.600145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:25:23.600615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:23.600615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:25:23.600615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:25:23.600615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:25:23.600615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:25:23.600615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:25:23.600615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:25:23.600615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:25:23.600615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:25:23.600615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:25:23.600615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:23.600615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:25:23.601143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:23.601143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:25:23.601143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:23.601143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:25:23.601143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:23.601143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:25:23.601143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:23.601143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:25:23.601143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:25:23.601143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:23.601143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:25:23.601143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:23.601143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:25:23.601143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:25:23.601775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:23.601775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:25:23.601775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:23.601775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:25:23.602061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:23.602061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:25:23.602061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:25:23.602061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:25:23.602061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:25:23.602061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:25:23.602061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:25:23.602061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:25:23.602061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:25:23.602061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:25:23.602061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:25:23.602061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:25:23.602061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:25:23.602061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:25:23.602061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:25:23.602061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:25:23.602592 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:25:23.602592 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:25:23.602592 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:25:23.602592 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:25:23.602592 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:25:23.602592 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:23.602592 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:25:23.602592 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:23.602592 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:25:23.602592 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:23.602592 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:25:23.602592 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:23.602592 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:25:23.602592 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:23.602592 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:25:23.602592 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:25:23.602592 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:23.602592 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:25:23.603209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:23.603209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:25:23.603209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:25:23.603209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:23.603209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:25:23.603209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:23.603209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:25:23.609552 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:25:23.609805 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:25:23.609941 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:25:23.610068 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:25:23.610488 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:25:23.610947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:23.611654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:25:23.611654 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:23.611654 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:25:23.611654 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:25:23.611654 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:23.611654 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:25:23.611654 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:23.611654 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:25:23.612512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:23.612512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:25:23.612512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:25:23.612512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:25:23.612512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:23.612512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:25:23.612512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:23.612512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:25:23.612512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:23.612512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:25:23.612512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:23.612512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:25:23.612512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:23.612512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:25:23.612512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:25:23.613107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:23.613107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:25:23.613107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:23.613107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:25:23.613107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:25:23.613107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:23.613107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:25:23.613107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:23.613107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:25:23.613441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:23.613441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:25:23.613441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:25:23.613441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:25:23.613441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:25:23.613441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:25:23.613441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:25:23.613441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:25:23.613441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:25:23.613441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:23.613441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:25:23.613894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:23.613894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:25:23.613894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:23.613894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:25:23.613894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:23.613894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:25:23.613894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:23.613894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:25:23.613894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:25:23.613894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:23.613894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:25:23.613894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:23.613894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:25:23.613894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:25:23.614358 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:23.614358 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:25:23.614358 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:23.614358 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:25:23.614630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:23.614630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:25:23.614630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:25:23.614630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:25:23.614630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:25:23.614630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:25:23.614630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:25:23.614630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:25:23.614630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:25:23.614630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:25:23.614630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:25:23.614630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:25:23.614630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:25:23.614630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:25:23.614630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:25:23.614630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:25:23.614630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:25:23.615453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:25:23.615453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:25:23.615453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:25:23.615453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:23.615453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:25:23.615453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:23.615453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:25:23.615453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:23.615453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:25:23.615453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:23.615453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:25:23.615453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:23.615453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:25:23.615453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:25:23.615453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:23.615453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:25:23.615453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:23.615453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:25:23.615453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:25:23.616121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:23.616121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:25:23.616121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:23.616121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:25:23.617328 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:25:23.617604 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:25:23.617697 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:25:23.617939 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:25:23.618479 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:25:23.618479 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:25:23.618618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:23.619128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:25:23.619128 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:23.619128 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:25:23.619128 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:25:23.619128 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:23.619128 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:25:23.619128 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:23.619128 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:25:23.620041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:23.620041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:25:23.620041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:25:23.620041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:25:23.620041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:25:23.620041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:25:23.620041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:25:23.620041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:23.620041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:25:23.620041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:23.620041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:25:23.620041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:23.620581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:25:23.620581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:23.620581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:25:23.620581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:23.620581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:25:23.620581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:25:23.620581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:23.620581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:25:23.620581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:23.620581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:25:23.620581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:25:23.620581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:23.620581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:25:23.621070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:23.621070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:25:23.621070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:23.621070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:25:23.621070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:25:23.621070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:25:23.621070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:25:23.621070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:25:23.621070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:25:23.621070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:25:23.621070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:25:23.621070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:25:23.621070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:25:23.621070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:25:23.621070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:25:23.621614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:25:23.621614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:25:23.621614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:25:23.621614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:25:23.621614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:25:23.621614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:23.621614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:25:23.621614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:23.621614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:25:23.621614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:23.621614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:25:23.621614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:23.621614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:25:23.621614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:23.621614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:25:23.621614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:25:23.621614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:23.621614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:25:23.621614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:23.622337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:25:23.622337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:25:23.622337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:23.622337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:25:23.622337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:23.622337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:25:28.432497 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:25:28.433577 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:25:28.433879 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:25:28.435933 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:25:28.472889 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:25:28.473188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:28.473697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:25:28.473697 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:28.473697 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:25:28.473697 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:25:28.473697 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:28.473697 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:25:28.473697 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:28.473697 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:25:28.474540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:28.474540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:25:28.474540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:25:28.474540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:25:28.474540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:25:28.474540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:25:28.474540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:25:28.474540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:28.474540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:25:28.474540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:28.474540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:25:28.474540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:28.475290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:25:28.475290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:28.475290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:25:28.475290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:28.475290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:25:28.475290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:25:28.475290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:28.475290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:25:28.475290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:28.475290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:25:28.475290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:25:28.475290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:28.475290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:25:28.476076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:28.476076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:25:28.476076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:28.476076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:25:28.476076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:25:28.476076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:25:28.476076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:25:28.476076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:25:28.476076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:25:28.476076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:25:28.476076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:25:28.476076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:25:28.476076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:25:28.476076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:25:28.476076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:25:28.476897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:25:28.476897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:25:28.476897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:25:28.476897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:25:28.476897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:25:28.476897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:28.476897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:25:28.476897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:28.476897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:25:28.476897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:28.476897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:25:28.476897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:28.476897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:25:28.476897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:28.476897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:25:28.476897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:25:28.476897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:28.476897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:25:28.476897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:28.478240 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:25:28.478240 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:25:28.478240 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:28.478240 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:25:28.478240 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:28.478240 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:25:28.588622 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:25:28.589072 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:25:28.589279 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:25:28.590770 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:25:28.590984 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:25:28.591148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:28.591886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:25:28.591886 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:28.591886 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:25:28.591886 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:25:28.591886 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:28.591886 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:25:28.591886 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:28.591886 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:25:28.593457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:28.593457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:25:28.593457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:25:28.593457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:25:28.593457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:25:28.593457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:25:28.593457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:25:28.593457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:28.593457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:25:28.593457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:28.593457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:25:28.593457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:28.594108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:25:28.594108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:28.594108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:25:28.594108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:28.594108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:25:28.594108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:25:28.594108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:28.594108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:25:28.594108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:28.594108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:25:28.594108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:25:28.594108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:28.594108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:25:28.594732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:28.594732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:25:28.595835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:28.595835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:25:28.595835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:25:28.595835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:25:28.595835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:25:28.595835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:25:28.595835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:25:28.595835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:25:28.595835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:25:28.595835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:25:28.595835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:25:28.595835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:25:28.595835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:25:28.595835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:25:28.595835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:25:28.595835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:25:28.596853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:25:28.596853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:25:28.596853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:28.596853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:25:28.596853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:28.596853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:25:28.596853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:28.596853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:25:28.596853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:28.596853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:25:28.596853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:28.596853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:25:28.596853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:25:28.596853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:28.596853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:25:28.596853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:28.596853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:25:28.596853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:25:28.596853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:28.598007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:25:28.598007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:28.598007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:25:28.598007 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:25:28.598007 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:25:28.598007 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:25:28.598007 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:25:28.598007 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:25:28.598465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:28.598639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:25:28.598639 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:28.598639 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:25:28.598639 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:25:28.598639 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:28.598639 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:25:28.598639 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:28.598639 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:25:28.599448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:28.599448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:25:28.599448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:25:28.599448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:25:28.599448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:28.599448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:25:28.599448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:28.599448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:25:28.599448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:28.599448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:25:28.599448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:28.599448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:25:28.599448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:28.599448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:25:28.599448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:25:28.600097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:28.600097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:25:28.600097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:28.600097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:25:28.600097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:25:28.600097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:28.600097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:25:28.600097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:28.600097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:25:28.600386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:28.600386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:25:28.600386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:25:28.600386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:25:28.600386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:25:28.600386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:25:28.600386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:25:28.600386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:25:28.600386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:25:28.600386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:25:28.600386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:28.600386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:25:28.600802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:28.600802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:25:28.600802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:28.600802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:25:28.600802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:28.600802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:25:28.600802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:28.600802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:25:28.600802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:25:28.600802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:28.600802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:25:28.600802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:28.600802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:25:28.600802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:25:28.601266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:28.601266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:25:28.601266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:28.601266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:25:28.601466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:28.601466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:25:28.601466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:25:28.601466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:25:28.601466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:25:28.601466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:25:28.601466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:25:28.601466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:25:28.601466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:25:28.601466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:25:28.601466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:25:28.601466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:25:28.601466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:25:28.601466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:25:28.601466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:25:28.601466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:25:28.601994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:25:28.601994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:25:28.601994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:25:28.601994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:25:28.601994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:25:28.601994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:28.601994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:25:28.601994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:28.601994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:25:28.601994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:28.601994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:25:28.601994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:28.601994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:25:28.601994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:28.601994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:25:28.601994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:25:28.601994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:28.601994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:25:28.602512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:28.602512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:25:28.602512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:25:28.602512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:28.602512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:25:28.602512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:28.602512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:25:28.610062 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:25:28.610416 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:25:28.610534 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:25:28.610680 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:25:28.611432 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:25:28.611947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:28.612855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:25:28.612855 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:28.612855 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:25:28.612855 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:25:28.612855 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:28.612855 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:25:28.612855 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:28.612855 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:25:28.617478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:28.617478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:25:28.617478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:25:28.617478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:25:28.617478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:28.617478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:25:28.617478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:28.617478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:25:28.617478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:28.617478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:25:28.617478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:28.617478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:25:28.617478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:28.617478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:25:28.617478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:25:28.617923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:28.617923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:25:28.617923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:28.617923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:25:28.617923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:25:28.617923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:28.617923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:25:28.617923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:28.617923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:25:28.618579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.007 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:28.618579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:25:28.618579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:25:28.618579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:25:28.618579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:25:28.618579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:25:28.618579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:25:28.618579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:25:28.618579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:25:28.618579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:28.618579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:25:28.618953 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:28.618953 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:25:28.618953 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:28.618953 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:25:28.618953 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:28.618953 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:25:28.618953 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:28.618953 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:25:28.618953 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:25:28.618953 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:28.618953 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:25:28.618953 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:28.618953 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:25:28.618953 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:25:28.619335 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:28.619335 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:25:28.619335 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:28.619335 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:25:28.619335 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:25:28.620203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:28.620203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:25:28.620203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:25:28.620203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:25:28.620203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:25:28.620203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:25:28.620203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:25:28.620203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:25:28.620203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:25:28.620203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:25:28.620203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:25:28.620203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:25:28.620203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:25:28.620203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:25:28.620203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:25:28.620203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:25:28.620203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:25:28.620920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:25:28.620920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:25:28.620920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:25:28.620920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:28.620920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:25:28.620920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:28.620920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:25:28.620920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:28.620920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:25:28.620920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:28.620920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:25:28.620920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:28.620920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:25:28.620920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:25:28.620920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:28.620920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:25:28.620920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:28.620920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:25:28.620920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:25:28.621459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:28.621459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:25:28.621459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:28.621459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:25:28.621459 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:25:28.621459 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:25:28.621459 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:25:28.621459 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:25:28.621774 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:25:28.621774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:28.622010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:25:28.622010 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:28.622010 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:25:28.622010 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:25:28.622010 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:28.622010 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:25:28.622010 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:28.622010 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:25:28.623015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:28.623015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:25:28.623015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:25:28.623015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:25:28.623015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:25:28.623015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:25:28.623015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:25:28.623015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:28.623015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:25:28.623015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:28.623015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:25:28.623015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:28.623403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:25:28.623403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:28.623403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:25:28.623403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:28.623403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:25:28.623403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:25:28.623403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:28.623403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:25:28.623403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:28.623403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:25:28.623403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:25:28.623403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:28.623403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:25:28.623806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:28.623806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:25:28.624136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:28.624136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:25:28.624136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:25:28.624136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:25:28.624136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:25:28.624136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:25:28.624136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:25:28.624136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:25:28.624136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:25:28.624136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:25:28.624136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:25:28.624136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:25:28.624136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:25:28.624136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:25:28.624136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:25:28.624136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:25:28.624643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:25:28.624643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:25:28.624643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:28.624643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:25:28.624643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:28.624643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:25:28.624643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:28.624643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:25:28.624643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:28.624643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:25:28.624643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:28.624643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:25:28.624643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:25:28.624643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:28.624643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:25:28.624643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:28.624643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:25:28.624643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:25:28.624643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:28.625291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:25:28.625291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:28.625291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:25:33.433399 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:25:33.434057 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:25:33.434338 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:25:33.435789 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:25:33.471978 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:25:33.472479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:33.473561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:25:33.473561 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:33.473561 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:25:33.473561 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:25:33.473561 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:33.473561 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:25:33.473561 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:33.473561 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:25:33.474395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:33.474395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:25:33.474395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:25:33.474395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:25:33.474395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:25:33.474395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:25:33.474395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:25:33.474395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:33.474395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:25:33.474395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:33.474395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:25:33.474395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:33.474705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:25:33.474705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:33.474705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:25:33.474705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:33.474705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:25:33.474705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:25:33.474705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:33.474705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:25:33.474705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:33.474705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:25:33.474705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:25:33.474705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:33.474705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:25:33.475093 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:33.475093 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:25:33.475262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:33.475262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:25:33.475262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:25:33.475262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:25:33.475262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:25:33.475262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:25:33.475262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:25:33.475262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:25:33.475262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:25:33.475262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:25:33.475262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:25:33.475262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:25:33.475262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:25:33.475262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:25:33.475262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:25:33.475262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:25:33.475630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:25:33.475630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:25:33.475630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:33.475630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:25:33.475630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:33.475630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:25:33.475630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:33.475630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:25:33.475630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:33.475630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:25:33.475630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:33.475630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:25:33.475630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:25:33.475630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:33.475630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:25:33.475630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:33.475630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:25:33.475630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:25:33.475630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:33.476027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:25:33.476027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:33.476027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:25:33.589167 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:25:33.589896 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:25:33.590203 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:25:33.591377 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:25:33.591377 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:25:33.591571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:33.592931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:25:33.592931 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:33.592931 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:25:33.592931 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:25:33.592931 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:33.592931 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:25:33.592931 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:33.592931 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:25:33.594123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:33.594123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:25:33.594123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:25:33.594123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:25:33.594123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:25:33.594123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:25:33.594123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:25:33.594123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:33.594123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:25:33.594123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:33.594123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:25:33.594123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:33.594518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:25:33.594518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:33.594518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:25:33.594518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:33.594518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:25:33.594518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:25:33.594518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:33.594518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:25:33.594518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:33.594518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:25:33.594518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:25:33.594518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:33.594518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:25:33.594866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:33.594866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:25:33.596140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:33.596140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:25:33.596140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:25:33.596140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:25:33.596140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:25:33.596140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:25:33.596140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:25:33.596140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:25:33.596140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:25:33.596140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:25:33.596140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:25:33.596140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:25:33.596140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:25:33.596140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:25:33.596140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:25:33.596140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:25:33.596607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:25:33.596607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:25:33.596607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:33.596607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:25:33.596607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:33.596607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:25:33.596607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:33.596607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:25:33.596607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:33.596607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:25:33.596607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:33.596607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:25:33.596607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:25:33.596607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:33.596607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:25:33.596607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:33.596607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:25:33.596607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:25:33.596607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:33.597133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:25:33.597133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:33.597133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:25:33.597371 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:25:33.597904 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:25:33.598150 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:25:33.598443 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:25:33.599406 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:25:33.599619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:33.600144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:25:33.600144 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:33.600144 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:25:33.600144 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:25:33.600144 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:33.600144 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:25:33.600144 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:33.600144 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:25:33.600926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:33.600926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:25:33.600926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:25:33.600926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:25:33.600926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:33.600926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:25:33.600926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:33.600926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:25:33.600926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:33.600926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:25:33.600926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:33.600926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:25:33.600926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:33.600926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:25:33.600926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:25:33.601551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:33.601551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:25:33.601551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:33.601551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:25:33.601551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:25:33.601551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:33.601551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:25:33.601551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:33.601551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:25:33.601875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:33.601875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:25:33.601875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:25:33.601875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:25:33.601875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:25:33.601875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:25:33.601875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:25:33.601875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:25:33.601875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:25:33.601875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:25:33.601875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:33.601875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:25:33.602391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:33.602391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:25:33.602391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:33.602391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:25:33.602391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:33.602391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:25:33.602391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:33.602391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:25:33.602391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:25:33.602391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:33.602391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:25:33.602391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:33.602391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:25:33.602391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:25:33.602969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:33.602969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:25:33.602969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:33.602969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:25:33.603165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:33.603165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:25:33.603165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:25:33.603165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:25:33.603165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:25:33.603165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:25:33.603165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:25:33.603165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:25:33.603165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:25:33.603165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:25:33.603165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:25:33.603165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:25:33.603165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:25:33.603165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:25:33.603165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:25:33.603165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:25:33.603677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:25:33.603677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:25:33.603677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:25:33.603677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:25:33.603677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:25:33.603677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:33.603677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:25:33.603677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:33.603677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:25:33.603677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:33.603677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:25:33.603677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:33.603677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:25:33.603677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:33.603677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:25:33.603677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:25:33.603677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:33.603677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:25:33.604317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:33.604317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:25:33.604317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:25:33.604317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:33.604317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:25:33.604317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:33.604317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:25:33.610722 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:25:33.611124 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:25:33.611239 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:25:33.611372 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:25:33.611778 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:25:33.612065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:33.612659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:25:33.612659 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:33.612659 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:25:33.612659 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:25:33.612659 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:33.612659 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:25:33.612659 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:33.612659 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:25:33.613508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:33.613508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:25:33.613508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:25:33.613508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:25:33.613508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:33.613508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:25:33.613508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:33.613508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:25:33.613508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:33.613508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:25:33.613508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:33.613508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:25:33.613508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:33.613508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:25:33.613508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:25:33.614032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:33.614032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:25:33.614032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:33.614032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:25:33.614032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:25:33.614032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:33.614032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:25:33.614032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:33.614032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:25:33.614584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:33.614584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:25:33.614584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:25:33.614584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:25:33.614584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:25:33.614584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:25:33.614584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:25:33.614584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:25:33.614584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:25:33.614584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:33.614584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:25:33.615337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:33.615337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:25:33.615337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:33.615337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:25:33.615337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:33.615337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:25:33.615337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:33.615337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:25:33.615337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:25:33.615337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:33.615337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:25:33.615337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:33.615337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:25:33.615337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:25:33.615816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:33.615816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:25:33.615816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:33.615816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:25:33.615816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:33.615816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:25:33.615816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:25:33.615816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:25:33.615816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:25:33.615816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:25:33.615816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:25:33.615816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:25:33.615816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:25:33.615816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:25:33.616295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:25:33.616295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:25:33.616295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:25:33.616295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:25:33.616295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:25:33.616295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:25:33.616295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:25:33.616295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:25:33.616295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:25:33.616295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:25:33.616295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:33.616295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:25:33.616295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:33.616295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:25:33.616295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:33.616295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:25:33.616295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:33.616295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:25:33.616295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:33.617021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:25:33.617021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:25:33.617021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:33.617021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:25:33.617021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:33.617021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:25:33.617021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:25:33.617021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:33.617021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:25:33.617021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:33.617021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:25:33.620630 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:25:33.620984 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:25:33.621091 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:25:33.621363 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:25:33.621814 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:25:33.622098 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:25:33.622247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:33.622809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:25:33.622809 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:33.622809 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:25:33.622809 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:25:33.622809 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:33.622809 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:25:33.622809 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:33.622809 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:25:33.623767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:33.623767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:25:33.623767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:25:33.623767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:25:33.623767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:25:33.623767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:25:33.623767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:25:33.623767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:33.623767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:25:33.623767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:33.623767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:25:33.623767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:33.624141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:25:33.624141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:33.624141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:25:33.624141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:33.624141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:25:33.624141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:25:33.624141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:33.624141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:25:33.624141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:33.624141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:25:33.624141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:25:33.624141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:33.624141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:25:33.624481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:33.624481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:25:33.624843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:33.624843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:25:33.624843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:25:33.624843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:25:33.624843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:25:33.624843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:25:33.624843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:25:33.624843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:25:33.624843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:25:33.624843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:25:33.624843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:25:33.624843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:25:33.624843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:25:33.624843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:25:33.624843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:25:33.624843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:25:33.625365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:25:33.625365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:25:33.625365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:33.625365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:25:33.625365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:33.625365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:25:33.625365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:33.625365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:25:33.625365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:33.625365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:25:33.625365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:33.625365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:25:33.625365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:25:33.625365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:33.625365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:25:33.625365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:33.625365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:25:33.625365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:25:33.625365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:33.626086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:25:33.626086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:33.626086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:25:38.434025 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:25:38.434676 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:25:38.434845 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:25:38.435685 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:25:38.469935 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:25:38.470407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:38.471169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:25:38.471169 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:38.471169 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:25:38.471169 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:25:38.471169 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:38.471169 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:25:38.471169 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:38.471169 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:25:38.471907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:38.471907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:25:38.471907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:25:38.471907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:25:38.471907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:25:38.471907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:25:38.471907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:25:38.471907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:38.471907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:25:38.471907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:38.471907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:25:38.471907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:38.472347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:25:38.472347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:38.472347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:25:38.472347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:38.472347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:25:38.472347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:25:38.472347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:38.472347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:25:38.472347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:38.472347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:25:38.472347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:25:38.472347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:38.472347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:25:38.472806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:38.472806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:25:38.472806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:38.472806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:25:38.472806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:25:38.472806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:25:38.472806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:25:38.472806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:25:38.472806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:25:38.472806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:25:38.472806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:25:38.472806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:25:38.472806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:25:38.472806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:25:38.472806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:25:38.473295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:25:38.473295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:25:38.473295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:25:38.473295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:25:38.473295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:25:38.473295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:38.473295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:25:38.473295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:38.473295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:25:38.473295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:38.473295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:25:38.473295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:38.473295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:25:38.473295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:38.473295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:25:38.473295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:25:38.473295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:38.473295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:25:38.473295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:38.473877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:25:38.473877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:25:38.473877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:38.473877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:25:38.473877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:38.473877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:25:38.589822 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:25:38.590427 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:25:38.590702 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:25:38.591820 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:25:38.592082 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:25:38.592349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:38.593313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:25:38.593313 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:38.593313 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:25:38.593313 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:25:38.593313 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:38.593313 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:25:38.593313 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:38.593313 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:25:38.594914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:38.594914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:25:38.594914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:25:38.594914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:25:38.594914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:25:38.594914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:25:38.594914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:25:38.594914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:38.594914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:25:38.594914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:38.594914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:25:38.594914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:38.595323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:25:38.595323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:38.595323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:25:38.595323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:38.595323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:25:38.595323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:25:38.595323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:38.595323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:25:38.595323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:38.595323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:25:38.595323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:25:38.595323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:38.595323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:25:38.595661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:38.595661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:25:38.597004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:38.597004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:25:38.597004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:25:38.597004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:25:38.597004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:25:38.597004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:25:38.597004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:25:38.597004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:25:38.597004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:25:38.597004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:25:38.597004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:25:38.597004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:25:38.597004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:25:38.597004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:25:38.597004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:25:38.597004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:25:38.598523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:25:38.598523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:25:38.598523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:38.598523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:25:38.598523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:38.598523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:25:38.598523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:38.598523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:25:38.598523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:38.598523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:25:38.598523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:38.598523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:25:38.598523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:25:38.598523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:38.598523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:25:38.598523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:38.598523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:25:38.598523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:25:38.598523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:38.599257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:25:38.599257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:38.599257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:25:38.599257 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:25:38.599257 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:25:38.599257 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:25:38.599257 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:25:38.599527 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:25:38.599957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:38.600440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:25:38.600440 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:38.600440 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:25:38.600440 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:25:38.600440 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:38.600440 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:25:38.600440 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:38.600440 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:25:38.601408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:38.601408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:25:38.601408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:25:38.601408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:25:38.601408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:38.601408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:25:38.601408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:38.601408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:25:38.601408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:38.601408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:25:38.601408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:38.601408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:25:38.601408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:38.601408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:25:38.601408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:25:38.602020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:38.602020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:25:38.602020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:38.602020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:25:38.602020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:25:38.602020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:38.602020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:25:38.602020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:38.602020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:25:38.602501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:38.602501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:25:38.602501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:25:38.602501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:25:38.602501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:25:38.602501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:25:38.602501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:25:38.602501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:25:38.602501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:25:38.602501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:25:38.602501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:38.602501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:25:38.604492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:38.604492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:25:38.604492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:38.604492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:25:38.604492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:38.604492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:25:38.604492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:38.604492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:25:38.604492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:25:38.604492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:38.604492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:25:38.604492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:38.604492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:25:38.604492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:25:38.605906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:38.605906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:25:38.605906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:38.605906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:25:38.605906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:38.605906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:25:38.605906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:25:38.605906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:25:38.605906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:25:38.605906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:25:38.605906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:25:38.605906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:25:38.605906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:25:38.605906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:25:38.606454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:25:38.606454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:25:38.606454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:25:38.606454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:25:38.606454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:25:38.606454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:25:38.606454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:25:38.606454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:25:38.606454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:25:38.606454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:25:38.606454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:25:38.606454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:38.606454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:25:38.606454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:38.606454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:25:38.606454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:38.606454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:25:38.606454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:38.606454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:25:38.607120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:38.607120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:25:38.607120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:25:38.607120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:38.607120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:25:38.607120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:38.607120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:25:38.607120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:25:38.607120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:38.607120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:25:38.607120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:38.607120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:25:38.611259 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:25:38.611590 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:25:38.611725 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:25:38.611938 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:25:38.612409 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:25:38.612766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:38.613475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:25:38.613475 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:38.613475 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:25:38.613475 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:25:38.613475 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:38.613475 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:25:38.613475 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:38.613475 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:25:38.614853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:38.614853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:25:38.614853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:25:38.614853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:25:38.614853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:38.614853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:25:38.614853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:38.614853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:25:38.614853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:38.614853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:25:38.614853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:38.614853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:25:38.614853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:38.614853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:25:38.614853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:25:38.615146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:38.615146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:25:38.615146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:38.615146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:25:38.615146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:25:38.615146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:38.615146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:25:38.615146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:38.615146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:25:38.616281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:38.616281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:25:38.616281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:25:38.616281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:25:38.616281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:25:38.616281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:25:38.616281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:25:38.616281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:25:38.616281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:25:38.616281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:38.616281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:25:38.616516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:38.616516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:25:38.616516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:38.616516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:25:38.616516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:38.616516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:25:38.616516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:38.616516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:25:38.616516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:25:38.616516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:38.616516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:25:38.616516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:38.616516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:25:38.616516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:25:38.616920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:38.616920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:25:38.616920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:38.616920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:25:38.618192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:38.618192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:25:38.618192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:25:38.618192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:25:38.618192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:25:38.618192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:25:38.618192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:25:38.618192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:25:38.618192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:25:38.618192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:25:38.618192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:25:38.618192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:25:38.618192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:25:38.618192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:25:38.618192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:25:38.618192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:25:38.618192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:25:38.619016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:25:38.619016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:25:38.619016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:25:38.619016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:38.619016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:25:38.619016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:38.619016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:25:38.619016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:38.619016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:25:38.619016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:38.619016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:25:38.619016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:38.619016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:25:38.619016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:25:38.619016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:38.619016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:25:38.619016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:38.619016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:25:38.619016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:25:38.620022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:38.620022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:25:38.620022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:38.620022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:25:38.621128 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:25:38.621495 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:25:38.621668 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:25:38.621858 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:25:38.622145 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:25:38.622440 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:25:38.622656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:38.623097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:25:38.623097 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:38.623097 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:25:38.623097 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:25:38.623097 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:38.623097 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:25:38.623097 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:38.623097 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:25:38.624367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:38.624367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:25:38.624367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:25:38.624367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:25:38.624367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:25:38.624367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:25:38.624367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:25:38.624367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:38.624367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:25:38.624367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:38.624367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:25:38.624367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:38.626097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:25:38.626097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:38.626097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:25:38.626097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:38.626097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:25:38.626097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:25:38.626097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:38.626097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:25:38.626097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:38.626097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:25:38.626097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:25:38.626097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:38.626097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:25:38.626983 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:38.626983 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:25:38.626983 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:38.626983 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:25:38.626983 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:25:38.626983 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:25:38.626983 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:25:38.626983 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:25:38.626983 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:25:38.626983 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:25:38.626983 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:25:38.626983 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:25:38.626983 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:25:38.626983 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:25:38.626983 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:25:38.627326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:25:38.627326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:25:38.627326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:25:38.627326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:25:38.627326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:25:38.627326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:38.627326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:25:38.627326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:38.627326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:25:38.627326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:38.627326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:25:38.627326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:38.627326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:25:38.627326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:38.627326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:25:38.627326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:25:38.627326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:38.627326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:25:38.627326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:38.627703 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:25:38.627703 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:25:38.627703 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:38.627703 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:25:38.627703 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:38.627703 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:25:43.434881 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:25:43.435803 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:25:43.435803 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:25:43.437215 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:25:43.473570 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:25:43.474066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:43.474572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:25:43.474572 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:43.474572 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:25:43.474572 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:25:43.474572 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:43.474572 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:25:43.474572 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:43.474572 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:25:43.475459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:43.475459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:25:43.475459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:25:43.475459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:25:43.475459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:25:43.475459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:25:43.475459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:25:43.475459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:43.475459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:25:43.475459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:43.475459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:25:43.475459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:43.476316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:25:43.476316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:43.476316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:25:43.476316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:43.476316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:25:43.476316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:25:43.476316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:43.476316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:25:43.476316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:43.476316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:25:43.476316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:25:43.476316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:43.476316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:25:43.477279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:43.477279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:25:43.477279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:43.477279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:25:43.477279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:25:43.477279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:25:43.477279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:25:43.477279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:25:43.477279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:25:43.477279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:25:43.477279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:25:43.477279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:25:43.477279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:25:43.477279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:25:43.477279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:25:43.478282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:25:43.478282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:25:43.478282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:25:43.478282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:25:43.478282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:25:43.478282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:43.478282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:25:43.478282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:43.478282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:25:43.478282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:43.478282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:25:43.478282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:43.478282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:25:43.478282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:43.478282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:25:43.478282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:25:43.478282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:43.478282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:25:43.478282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:43.480163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:25:43.480163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:25:43.480163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:43.480163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:25:43.480163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:43.480163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:25:43.590565 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:25:43.591028 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:25:43.591151 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:25:43.592100 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:25:43.592480 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:25:43.592731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:43.593725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:25:43.593725 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:43.593725 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:25:43.593725 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:25:43.593725 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:43.593725 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:25:43.593725 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:43.593725 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:25:43.594930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:43.594930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:25:43.594930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:25:43.594930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:25:43.594930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:25:43.594930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:25:43.594930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:25:43.594930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:43.594930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:25:43.594930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:43.594930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:25:43.594930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:43.595548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:25:43.595548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:43.595548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:25:43.595548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:43.595548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:25:43.595548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:25:43.595548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:43.595548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:25:43.595548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:43.595548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:25:43.595548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:25:43.595548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:43.595548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:25:43.595945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:43.595945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:25:43.596267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:43.596267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:25:43.596267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:25:43.596267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:25:43.596267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:25:43.596267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:25:43.596267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:25:43.596267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:25:43.596267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:25:43.596267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:25:43.596267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:25:43.596267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:25:43.596267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:25:43.596267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:25:43.596267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:25:43.596267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:25:43.597397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:25:43.597397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:25:43.597397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:43.597397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:25:43.597397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:43.597397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:25:43.597397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:43.597397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:25:43.597397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:43.597397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:25:43.597397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:43.597397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:25:43.597397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:25:43.597397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:43.597397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:25:43.597397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:43.597397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:25:43.597397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:25:43.597397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:43.597899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:25:43.597899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:43.597899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:25:43.598250 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:25:43.598515 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:25:43.598633 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:25:43.598777 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:25:43.599337 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:25:43.599626 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:43.600358 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:25:43.600358 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:43.600358 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:25:43.600358 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:25:43.600358 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:43.600358 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:25:43.600358 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:43.600358 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:25:43.602186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:43.602186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:25:43.602186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:25:43.602186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:25:43.602186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:43.602186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:25:43.602186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:43.602186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:25:43.602186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:43.602186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:25:43.602186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:43.602186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:25:43.602186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:43.602186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:25:43.602186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:25:43.602645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:43.602645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:25:43.602645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:43.602645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:25:43.602645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:25:43.602645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:43.602645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:25:43.602645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:43.602645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:25:43.603317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:43.603317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:25:43.603317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:25:43.603317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:25:43.603317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:25:43.603317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:25:43.603317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:25:43.603317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:25:43.603317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:25:43.603317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:25:43.603317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:43.603317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:25:43.603724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:43.603724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:25:43.603724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:43.603724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:25:43.603724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:43.603724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:25:43.603724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:43.603724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:25:43.603724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:25:43.603724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:43.603724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:25:43.603724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:43.603724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:25:43.603724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:25:43.604565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:43.604565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:25:43.604565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:43.604565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:25:43.604565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:43.604565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:25:43.604565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:25:43.604565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:25:43.604565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:25:43.604565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:25:43.604565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:25:43.604565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:25:43.604565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:25:43.604565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:25:43.605315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:25:43.605315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:25:43.605315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:25:43.605315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:25:43.605315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:25:43.605315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:25:43.605315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:25:43.605315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:25:43.605315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:25:43.605315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:25:43.605315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:25:43.605315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:43.605315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:25:43.605315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:43.605315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:25:43.605315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:43.605315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:25:43.605315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:43.605315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:25:43.605929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:43.605929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:25:43.605929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:25:43.605929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:43.605929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:25:43.605929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:43.605929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:25:43.605929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:25:43.605929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:43.605929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:25:43.605929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:43.605929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:25:43.611720 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:25:43.612023 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:25:43.612152 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:25:43.612356 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:25:43.612995 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:25:43.613365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:43.614322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:25:43.614322 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:43.614322 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:25:43.614322 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:25:43.614322 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:43.614322 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:25:43.614322 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:43.614322 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:25:43.615130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:43.615130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:25:43.615130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:25:43.615130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:25:43.615130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:43.615130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:25:43.615130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:43.615130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:25:43.615130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:43.615130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:25:43.615130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:43.615130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:25:43.615130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:43.615130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:25:43.615130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:25:43.616051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:43.616051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:25:43.616051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:43.616051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:25:43.616051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:25:43.616051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:43.616051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:25:43.616051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:43.616051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:25:43.616051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:43.616051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:25:43.616051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:25:43.616051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:25:43.616051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:25:43.617673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:25:43.617673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:25:43.617673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:25:43.617673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:25:43.617673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:43.617673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:25:43.617673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:43.617673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:25:43.617673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:43.617673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:25:43.617673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:43.617673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:25:43.617673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:43.618344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:25:43.618344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:25:43.618344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:43.618344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:25:43.618344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:43.618344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:25:43.618344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:25:43.618344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:43.618344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:25:43.618344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:43.618344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:25:43.618344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:43.618835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:25:43.618835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:25:43.618835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:25:43.618835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:25:43.618835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:25:43.618835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:25:43.618835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:25:43.618835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:25:43.618835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:25:43.618835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:25:43.618835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:25:43.618835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:25:43.618835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:25:43.618835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:25:43.618835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:25:43.618835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:25:43.618835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:25:43.618835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:25:43.618835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:25:43.619497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:43.619497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:25:43.619497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:43.619497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:25:43.619497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:43.619497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:25:43.619497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:43.619497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:25:43.619497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:43.619497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:25:43.619497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:25:43.619497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:43.619497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:25:43.619497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:43.619497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:25:43.619497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:25:43.619497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:43.619497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:25:43.619497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:43.619497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:25:43.621425 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:25:43.621815 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:25:43.622032 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:25:43.622283 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:25:43.622943 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:25:43.623426 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:25:43.623668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:43.624670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:25:43.624670 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:43.624670 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:25:43.624670 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:25:43.624670 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:43.624670 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:25:43.624670 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:43.624670 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:25:43.625552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:43.625552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:25:43.625552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:25:43.625552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:25:43.625552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:25:43.625552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:25:43.625552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:25:43.625552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:43.625552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:25:43.625552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:43.625552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:25:43.625552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:43.625971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:25:43.625971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:43.625971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:25:43.625971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:43.625971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:25:43.625971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:25:43.625971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:43.625971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:25:43.625971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:43.625971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:25:43.625971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:25:43.625971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:43.625971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:25:43.626498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:43.626498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:25:43.626498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:43.626498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:25:43.626498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:25:43.626498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:25:43.626498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:25:43.626498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:25:43.626498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:25:43.626498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:25:43.626498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:25:43.626498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:25:43.626498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:25:43.626498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:25:43.626498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:25:43.626987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:25:43.626987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:25:43.626987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:25:43.626987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:25:43.626987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:25:43.626987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:43.626987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:25:43.626987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:43.626987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:25:43.626987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:43.626987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:25:43.626987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:43.626987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:25:43.626987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:43.626987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:25:43.626987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:25:43.626987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:43.626987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:25:43.626987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:43.627572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:25:43.627572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:25:43.627572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:43.627572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:25:43.627572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:43.627572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:25:48.435488 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:25:48.436207 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:25:48.436417 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:25:48.437944 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:25:48.474493 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:25:48.474828 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:48.475660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:25:48.475660 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:48.475660 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:25:48.475660 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:25:48.475660 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:48.475660 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:25:48.475660 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:48.475660 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:25:48.476436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:48.476436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:25:48.476436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:25:48.476436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:25:48.476436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:25:48.476436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:25:48.476436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:25:48.476436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:48.476436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:25:48.476436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:48.476436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:25:48.476436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:48.477098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:25:48.477098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:48.477098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:25:48.477098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:48.477098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:25:48.477098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:25:48.477098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:48.477098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:25:48.477098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:48.477098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:25:48.477098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:25:48.477098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:48.477098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:25:48.477942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:48.477942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:25:48.477942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:48.477942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:25:48.477942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:25:48.477942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:25:48.477942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:25:48.477942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:25:48.477942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:25:48.477942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:25:48.477942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:25:48.477942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:25:48.477942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:25:48.477942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:25:48.477942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:25:48.478997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:25:48.478997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:25:48.478997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:25:48.478997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:25:48.478997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:25:48.478997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:48.478997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:25:48.478997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:48.478997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:25:48.478997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:48.478997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:25:48.478997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:48.478997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:25:48.478997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:48.478997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:25:48.478997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:25:48.478997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:48.478997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:25:48.478997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:48.480258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:25:48.480258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:25:48.480258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:48.480258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:25:48.480258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:48.480258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:25:48.591201 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:25:48.591547 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:25:48.591660 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:25:48.592626 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:25:48.592951 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:25:48.593119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:48.593776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:25:48.593776 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:48.593776 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:25:48.593776 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:25:48.593776 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:48.593776 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:25:48.593776 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:48.593776 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:25:48.594811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:48.594811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:25:48.594811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:25:48.594811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:25:48.594811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:25:48.594811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:25:48.594811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:25:48.594811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:48.594811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:25:48.594811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:48.594811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:25:48.594811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:48.595465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:25:48.595465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:48.595465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:25:48.595465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:48.595465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:25:48.595465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:25:48.595465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:48.595465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:25:48.595465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:48.595465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:25:48.595465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:25:48.595465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:48.595465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:25:48.596167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:48.596167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:25:48.596167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:48.596167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:25:48.596167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:25:48.596167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:25:48.596167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:25:48.596167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:25:48.596167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:25:48.596167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:25:48.596167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:25:48.596167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:25:48.596167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:25:48.596167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:25:48.596167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:25:48.596869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:25:48.596869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:25:48.596869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:25:48.596869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:25:48.596869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:25:48.596869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:48.596869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:25:48.596869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:48.596869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:25:48.596869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:48.596869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:25:48.596869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:48.596869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:25:48.596869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:48.596869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:25:48.596869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:25:48.596869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:48.596869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:25:48.596869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:48.597706 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:25:48.597706 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:25:48.597706 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:48.597706 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:25:48.597706 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:48.597706 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:25:48.598578 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:25:48.598796 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:25:48.598907 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:25:48.599029 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:25:48.599599 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:25:48.599945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:48.600718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:25:48.600718 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:48.600718 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:25:48.600718 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:25:48.600718 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:48.600718 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:25:48.600718 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:48.600718 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:25:48.601477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:48.601477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:25:48.601477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:25:48.601477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:25:48.601477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:48.601477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:25:48.601477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:48.601477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:25:48.601477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:48.601477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:25:48.601477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:48.601477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:25:48.601477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:48.601477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:25:48.601477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:25:48.602025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:48.602025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:25:48.602025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:48.602025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:25:48.602025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:25:48.602025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:48.602025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:25:48.602025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:48.602025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:25:48.602383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:48.602383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:25:48.602383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:25:48.602383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:25:48.602383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:25:48.602383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:25:48.602383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:25:48.602383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:25:48.602383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:25:48.602383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:25:48.602383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:48.602383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:25:48.602786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:48.602786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:25:48.602786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:48.602786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:25:48.602786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:48.602786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:25:48.602786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:48.602786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:25:48.602786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:25:48.602786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:48.602786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:25:48.602786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:48.602786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:25:48.602786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:25:48.603203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:48.603203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:25:48.603203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:48.603203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:25:48.603338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:48.603338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:25:48.603338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:25:48.603338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:25:48.603338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:25:48.603338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:25:48.603338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:25:48.603338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:25:48.603338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:25:48.603338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:25:48.603338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:25:48.603338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:25:48.603338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:25:48.603338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:25:48.603338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:25:48.603338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:25:48.603918 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:25:48.603918 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:25:48.603918 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:25:48.603918 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:25:48.603918 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:25:48.603918 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:48.603918 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:25:48.603918 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:48.603918 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:25:48.603918 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:48.603918 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:25:48.603918 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:48.603918 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:25:48.603918 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:48.603918 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:25:48.603918 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:25:48.603918 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:48.603918 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:25:48.604491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:48.604491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:25:48.604491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:25:48.604491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:48.604491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:25:48.604491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:48.604491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:25:48.612165 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:25:48.612446 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:25:48.612604 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:25:48.612807 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:25:48.613323 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:25:48.613706 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:48.614429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:25:48.614429 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:48.614429 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:25:48.614429 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:25:48.614429 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:48.614429 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:25:48.614429 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:48.614429 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:25:48.615226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:48.615226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:25:48.615226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:25:48.615226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:25:48.615226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:48.615226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:25:48.615226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:48.615226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:25:48.615226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:48.615226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:25:48.615226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:48.615226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:25:48.615226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:48.615226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:25:48.615226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:25:48.615721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:48.615721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:25:48.615721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:48.615721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:25:48.615721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:25:48.615721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:48.615721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:25:48.615721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:48.615721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:25:48.616191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:48.616191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:25:48.616191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:25:48.616191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:25:48.616191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:25:48.616191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:25:48.616191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:25:48.616191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:25:48.616191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:25:48.616191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:48.616191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:25:48.617036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:48.617036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:25:48.617036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:48.617036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:25:48.617036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:48.617036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:25:48.617036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:48.617036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:25:48.617036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:25:48.617036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:48.617036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:25:48.617036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:48.617036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:25:48.617036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:25:48.617693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:48.617693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:25:48.617693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:48.617693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:25:48.617693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:48.617693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:25:48.617693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:25:48.617693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:25:48.617693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:25:48.617693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:25:48.617693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:25:48.617693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:25:48.617693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:25:48.617693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:25:48.618392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:25:48.618392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:25:48.618392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:25:48.618392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:25:48.618392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:25:48.618392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:25:48.618392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:25:48.618392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:25:48.618392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:25:48.618392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:25:48.618392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:48.618392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:25:48.618392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:48.618392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:25:48.618392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:48.618392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:25:48.618392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:48.618392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:25:48.618392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:48.619281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:25:48.619281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:25:48.619281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:48.619281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:25:48.619281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:48.619281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:25:48.619281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:25:48.619281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:48.619281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:25:48.619281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:48.619281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:25:48.621764 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:25:48.622012 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:25:48.622115 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:25:48.622276 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:25:48.622770 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:25:48.623168 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:25:48.623494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:48.624247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:25:48.624247 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:48.624247 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:25:48.624247 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:25:48.624247 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:48.624247 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:25:48.624247 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:48.624247 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:25:48.625184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:48.625184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:25:48.625184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:25:48.625184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:25:48.625184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:25:48.625184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:25:48.625184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:25:48.625184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:48.625184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:25:48.625184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:48.625184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:25:48.625184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:48.625560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:25:48.625560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:48.625560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:25:48.625560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:48.625560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:25:48.625560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:25:48.625560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:48.625560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:25:48.625560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:48.625560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:25:48.625560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:25:48.625560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:48.625560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:25:48.625939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:48.625939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:25:48.626370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:48.626370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:25:48.626370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:25:48.626370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:25:48.626370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:25:48.626370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:25:48.626370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:25:48.626370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:25:48.626370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:25:48.626370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:25:48.626370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:25:48.626370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:25:48.626370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:25:48.626370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:25:48.626370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:25:48.626370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:25:48.626811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:25:48.626811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:25:48.626811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:48.626811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:25:48.626811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:48.626811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:25:48.626811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:48.626811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:25:48.626811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:48.626811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:25:48.626811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:48.626811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:25:48.626811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:25:48.626811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:48.626811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:25:48.626811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:48.626811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:25:48.626811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:25:48.626811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:48.627315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:25:48.627315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:48.627315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:25:53.436319 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:25:53.437078 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:25:53.437078 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:25:53.438155 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:25:53.471312 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:25:53.471512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:53.472035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:25:53.472035 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:53.472035 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:25:53.472035 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:25:53.472035 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:53.472035 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:25:53.472035 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:53.472035 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:25:53.472732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:53.472732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:25:53.472732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:25:53.472732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:25:53.472732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:25:53.472732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:25:53.472732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:25:53.472732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:53.472732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:25:53.472732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:53.472732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:25:53.472732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:53.473167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:25:53.473167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:53.473167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:25:53.473167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:53.473167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:25:53.473167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:25:53.473167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:53.473167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:25:53.473167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:53.473167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:25:53.473167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:25:53.473167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:53.473167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:25:53.473484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:53.473484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:25:53.473548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:53.473548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:25:53.473548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:25:53.473548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:25:53.473548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:25:53.473548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:25:53.473548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:25:53.473548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:25:53.473548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:25:53.473548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:25:53.473548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:25:53.473548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:25:53.473548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:25:53.473548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:25:53.473548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:25:53.473548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:25:53.474020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:25:53.474020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:25:53.474020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:53.474020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:25:53.474020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:53.474020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:25:53.474020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:53.474020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:25:53.474020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:53.474020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:25:53.474020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:53.474020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:25:53.474020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:25:53.474020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:53.474020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:25:53.474020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:53.474020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:25:53.474020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:25:53.474020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:53.474497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:25:53.474497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:53.474497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:25:53.591768 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:25:53.592156 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:25:53.592277 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:25:53.592983 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:25:53.593164 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:25:53.593395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:53.594970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:25:53.594970 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:53.594970 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:25:53.594970 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:25:53.594970 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:53.594970 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:25:53.594970 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:53.594970 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:25:53.596373 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:53.596373 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:25:53.596373 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:25:53.596373 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:25:53.596373 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:25:53.596373 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:25:53.596373 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:25:53.596373 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:53.596373 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:25:53.596373 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:53.596373 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:25:53.596373 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:53.597226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:25:53.597226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:53.597226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:25:53.597226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:53.597226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:25:53.597226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:25:53.597226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:53.597226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:25:53.597226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:53.597226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:25:53.597226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:25:53.597226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:53.597226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:25:53.598093 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:53.598093 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:25:53.600866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:53.600866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:25:53.600866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:25:53.600866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:25:53.600866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:25:53.600866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:25:53.600866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:25:53.600866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:25:53.600866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:25:53.600866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:25:53.600866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:25:53.600866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:25:53.600866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:25:53.600866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:25:53.600866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:25:53.600866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:25:53.601969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:25:53.601969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:25:53.601969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:53.601969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:25:53.601969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:53.601969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:25:53.601969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:53.601969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:25:53.601969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:53.601969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:25:53.601969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:53.601969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:25:53.601969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:25:53.601969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:53.601969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:25:53.601969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:53.601969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:25:53.601969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:25:53.601969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:53.602694 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:25:53.602694 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:53.602694 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:25:53.602694 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:25:53.602694 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:25:53.602694 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:25:53.602694 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:25:53.602694 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:25:53.603015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:53.603815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:25:53.603815 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:53.603815 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:25:53.603815 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:25:53.603815 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:53.603815 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:25:53.603815 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:53.603815 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:25:53.604602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:53.604602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:25:53.604602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:25:53.604602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:25:53.604602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:53.604602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:25:53.604602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:53.604602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:25:53.604602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:53.604602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:25:53.604602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:53.604602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:25:53.604602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:53.604602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:25:53.604602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:25:53.605158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:53.605158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:25:53.605158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:53.605158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:25:53.605158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:25:53.605158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:53.605158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:25:53.605158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:53.605158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:25:53.605519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:53.605519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:25:53.605519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:25:53.605519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:25:53.605519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:25:53.605519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:25:53.605519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:25:53.605519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:25:53.605519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:25:53.605519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:25:53.605519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:53.605519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:25:53.606007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:53.606007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:25:53.606007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:53.606007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:25:53.606007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:53.606007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:25:53.606007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:53.606007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:25:53.606007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:25:53.606007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:53.606007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:25:53.606007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:53.606007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:25:53.606007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:25:53.606495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:53.606495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:25:53.606495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:53.606495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:25:53.606648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:53.606648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:25:53.606648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:25:53.606648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:25:53.606648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:25:53.606648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:25:53.606648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:25:53.606648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:25:53.606648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:25:53.606648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:25:53.606648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:25:53.606648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:25:53.606648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:25:53.606648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:25:53.606648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:25:53.606648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:25:53.607237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:25:53.607237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:25:53.607237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:25:53.607237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:25:53.607237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:25:53.607237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:53.607237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:25:53.607237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:53.607237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:25:53.607237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:53.607237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:25:53.607237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:53.607237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:25:53.607237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:53.607237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:25:53.607237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:25:53.607237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:53.607237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:25:53.608005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:53.608005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:25:53.608005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:25:53.608005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:53.608005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:25:53.608005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:53.608005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:25:53.612496 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:25:53.612758 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:25:53.612846 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:25:53.612945 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:25:53.613294 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:25:53.613579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:53.614148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:25:53.614148 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:53.614148 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:25:53.614148 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:25:53.614148 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:53.614148 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:25:53.614148 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:53.614148 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:25:53.614861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:53.614861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:25:53.614861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:25:53.614861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:25:53.614861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:53.614861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:25:53.614861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:53.614861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:25:53.614861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:53.614861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:25:53.614861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:53.614861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:25:53.614861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:53.614861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:25:53.614861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:25:53.615144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:53.615144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:25:53.615144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:53.615144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:25:53.615144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:25:53.615144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:53.615144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:25:53.615144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:53.615144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:25:53.615302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:53.615302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:25:53.615302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:25:53.615302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:25:53.615302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:25:53.615302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:25:53.615302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:25:53.615302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:25:53.615302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:25:53.615302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:53.615302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:25:53.615528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:53.615528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:25:53.615528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:53.615528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:25:53.615528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:53.615528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:25:53.615528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:53.615528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:25:53.615528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:25:53.615528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:53.615528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:25:53.615528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:53.615528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:25:53.615528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:25:53.615787 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:53.615787 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:25:53.615787 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:53.615787 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:25:53.616426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:53.616426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:25:53.616426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:25:53.616426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:25:53.616426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:25:53.616426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:25:53.616426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:25:53.616426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:25:53.616426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:25:53.616426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:25:53.616426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:25:53.616426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:25:53.616426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:25:53.616426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:25:53.616426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:25:53.616426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:25:53.616426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:25:53.617268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:25:53.617268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:25:53.617268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:25:53.617268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:53.617268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:25:53.617268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:53.617268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:25:53.617268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:53.617268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:25:53.617268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:53.617268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:25:53.617268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:53.617268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:25:53.617268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:25:53.617268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:53.617268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:25:53.617268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:53.617268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:25:53.617268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:25:53.617649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:53.617649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:25:53.617649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:53.617649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:25:53.623787 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:25:53.623787 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:25:53.623787 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:25:53.623787 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:25:53.623787 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:25:53.624031 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:25:53.624194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:53.625073 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:25:53.625073 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:53.625073 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:25:53.625073 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:25:53.625073 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:53.625073 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:25:53.625073 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:53.625073 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:25:53.625975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:53.625975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:25:53.625975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:25:53.625975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:25:53.625975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:25:53.625975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:25:53.625975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:25:53.625975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:53.625975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:25:53.625975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:53.625975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:25:53.625975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:53.626336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:25:53.626336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:53.626336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:25:53.626336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:53.626336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:25:53.626336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:25:53.626336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:53.626336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:25:53.626336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:53.626336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:25:53.626336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:25:53.626336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:53.626336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:25:53.626716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:53.626716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:25:53.627709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:53.627709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:25:53.627709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:25:53.627709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:25:53.627709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:25:53.627709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:25:53.627709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:25:53.627709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:25:53.627709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:25:53.627709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:25:53.627709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:25:53.627709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:25:53.627709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:25:53.627709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:25:53.627709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:25:53.627709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:25:53.628308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:25:53.628308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:25:53.628308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:53.628308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:25:53.628308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:53.628308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:25:53.628308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:53.628308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:25:53.628308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:53.628308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:25:53.628308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:53.628308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:25:53.628308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:25:53.628308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:53.628308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:25:53.628308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:53.628308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:25:53.628308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:25:53.628308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:53.628975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:25:53.628975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:53.628975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:25:58.437091 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:25:58.437955 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:25:58.438013 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:25:58.439237 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:25:58.440637 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG dbcounter [-] [106314] Writing DB stats neutron:SELECT=240 {{(pid=106314) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 09 09:25:58.474007 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:25:58.474404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:58.475306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:25:58.475306 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:58.475306 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:25:58.475306 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:25:58.475306 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:58.475306 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:25:58.475306 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:58.475306 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:25:58.476137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:58.476137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:25:58.476137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:25:58.476137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:25:58.476137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:25:58.476137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:25:58.476137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:25:58.476137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:58.476137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:25:58.476137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:58.476137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:25:58.476137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:58.476845 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:25:58.476845 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:58.476845 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:25:58.476845 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:58.476845 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:25:58.476845 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:25:58.476845 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:58.476845 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:25:58.476845 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:58.476845 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:25:58.476845 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:25:58.476845 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:58.476845 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:25:58.477528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:58.477528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:25:58.477528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:58.477528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:25:58.477528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:25:58.477528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:25:58.477528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:25:58.477528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:25:58.477528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:25:58.477528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:25:58.477528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:25:58.477528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:25:58.477528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:25:58.477528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:25:58.477528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:25:58.478335 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:25:58.478335 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:25:58.478335 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:25:58.478335 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:25:58.478335 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:25:58.478335 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:58.478335 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:25:58.478335 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:58.478335 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:25:58.478335 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:58.478335 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:25:58.478335 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:58.478335 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:25:58.478335 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:58.478335 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:25:58.478335 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:25:58.478335 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:58.478335 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:25:58.478335 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:58.479304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:25:58.479304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:25:58.479304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:58.479304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:25:58.479304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:58.479304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:25:58.592531 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:25:58.593135 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:25:58.593317 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:25:58.594666 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:25:58.595167 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:25:58.595462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:58.596516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:25:58.596516 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:58.596516 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:25:58.596516 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:25:58.596516 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:58.596516 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:25:58.596516 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:58.596516 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:25:58.598303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:58.598303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:25:58.598303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:25:58.598303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:25:58.598303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:25:58.598303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:25:58.598303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:25:58.598303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:58.598303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:25:58.598303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:58.598303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:25:58.598303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:58.599465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:25:58.599465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:58.599465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:25:58.599465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:58.599465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:25:58.599465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:25:58.599465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:58.599465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:25:58.599465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:58.599465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:25:58.599465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:25:58.599465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:58.599465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:25:58.600273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:58.600273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:25:58.600273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:58.600273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:25:58.600273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:25:58.600273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:25:58.600273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:25:58.600273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:25:58.600273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:25:58.600273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:25:58.600273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:25:58.600273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:25:58.600273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:25:58.600273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:25:58.600273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:25:58.601319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:25:58.601319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:25:58.601319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:25:58.601319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:25:58.601319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:25:58.601319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:58.601319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:25:58.601319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:58.601319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:25:58.601319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:58.601319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:25:58.601319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:58.601319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:25:58.601319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:58.601319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:25:58.601319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:25:58.601319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:58.601319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:25:58.601319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:58.602256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:25:58.602256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:25:58.602256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:58.602256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:25:58.602256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:58.602256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:25:58.602256 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:25:58.602657 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:25:58.602859 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:25:58.603052 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:25:58.604282 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:25:58.605221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:58.606180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:25:58.606180 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:58.606180 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:25:58.606180 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:25:58.606180 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:58.606180 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:25:58.606180 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:58.606180 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:25:58.607096 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:58.607096 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:25:58.607096 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:25:58.607096 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:25:58.607096 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:58.607096 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:25:58.607096 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:58.607096 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:25:58.607096 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:58.607096 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:25:58.607096 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:58.607096 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:25:58.607096 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:58.607096 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:25:58.607096 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:25:58.608147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:58.608147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:25:58.608147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:58.608147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:25:58.608147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:25:58.608147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:58.608147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:25:58.608147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:58.608147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:25:58.608147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:58.608147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:25:58.608147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:25:58.608147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:25:58.608147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:25:58.609212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:25:58.609212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:25:58.609212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:25:58.609212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:25:58.609212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:25:58.609212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:58.609212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:25:58.609212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:58.609212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:25:58.609212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:58.609212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:25:58.609212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:58.609212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:25:58.609802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:58.609802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:25:58.609802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:25:58.609802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:58.609802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:25:58.609802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:58.609802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:25:58.609802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:25:58.609802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:58.609802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:25:58.609802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:58.609802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:25:58.610279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:58.610279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:25:58.610279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:25:58.610279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:25:58.610279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:25:58.610279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:25:58.610279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:25:58.610279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:25:58.610279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:25:58.610279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:25:58.610279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:25:58.610279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:25:58.610279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:25:58.610279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:25:58.610279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:25:58.610279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:25:58.610906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:25:58.610906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:25:58.610906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:25:58.610906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:25:58.610906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:25:58.610906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:58.610906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:25:58.610906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:58.610906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:25:58.610906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:58.610906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:25:58.610906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:58.610906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:25:58.610906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:58.610906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:25:58.610906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:25:58.610906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:58.610906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:25:58.611612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:58.611612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:25:58.611612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:25:58.611612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:58.611612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:25:58.611612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:58.611612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:25:58.613095 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:25:58.613411 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:25:58.613522 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:25:58.613691 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:25:58.614139 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:25:58.614394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:58.614937 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:25:58.614937 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:58.614937 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:25:58.614937 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:25:58.614937 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:58.614937 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:25:58.614937 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:58.614937 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:25:58.615760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:58.615760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:25:58.615760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:25:58.615760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:25:58.615760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:58.615760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:25:58.615760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:58.615760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:25:58.615760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:58.615760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:25:58.615760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:58.615760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:25:58.615760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:58.615760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:25:58.615760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:25:58.616377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:58.616377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:25:58.616377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:58.616377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:25:58.616377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:25:58.616377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:58.616377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:25:58.616377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:58.616377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:25:58.616844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:58.616844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:25:58.616844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:25:58.616844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:25:58.616844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:25:58.616844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:25:58.616844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:25:58.616844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:25:58.616844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:25:58.616844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:58.616844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:25:58.617321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:58.617321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:25:58.617321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:58.617321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:25:58.617321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:58.617321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:25:58.617321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:58.617321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:25:58.617321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:25:58.617321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:58.617321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:25:58.617321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:58.617321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:25:58.617321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:25:58.617710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:58.617710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:25:58.617710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:58.617710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:25:58.618069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:58.618069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:25:58.618069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:25:58.618069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:25:58.618069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:25:58.618069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:25:58.618069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:25:58.618069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:25:58.618069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:25:58.618069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:25:58.618069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:25:58.618069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:25:58.618069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:25:58.618069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:25:58.618069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:25:58.618069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:25:58.618069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:25:58.618491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:25:58.618491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:25:58.618491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:25:58.618491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:58.618491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:25:58.618491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:58.618491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:25:58.618491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:58.618491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:25:58.618491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:58.618491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:25:58.618491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:58.618491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:25:58.618491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:25:58.618491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:58.618491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:25:58.618491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:58.618491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:25:58.618491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:25:58.618918 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:58.618918 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:25:58.618918 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:58.618918 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:25:58.622606 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:25:58.622936 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:25:58.623057 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:25:58.623227 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:25:58.623598 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:25:58.623867 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:25:58.624028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:58.624613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:25:58.624613 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:58.624613 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:25:58.624613 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:25:58.624613 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:58.624613 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:25:58.624613 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:58.624613 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:25:58.625566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:58.625566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:25:58.625566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:25:58.625566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:25:58.625566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:25:58.625566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:25:58.625566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:25:58.625566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:58.625566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:25:58.625566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:58.625566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:25:58.625566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:58.625977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:25:58.625977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:58.625977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:25:58.625977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:58.625977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:25:58.625977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:25:58.625977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:58.625977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:25:58.625977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:58.625977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:25:58.625977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:25:58.625977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:58.625977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:25:58.626341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:58.626341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:25:58.626710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:58.626710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:25:58.626710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:25:58.626710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:25:58.626710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:25:58.626710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:25:58.626710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:25:58.626710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:25:58.626710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:25:58.626710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:25:58.626710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:25:58.626710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:25:58.626710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:25:58.626710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:25:58.626710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:25:58.626710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:25:58.627204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:25:58.627204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:25:58.627204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:58.627204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:25:58.627204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:25:58.627204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:25:58.627204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:25:58.627204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:25:58.627204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:25:58.627204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:25:58.627204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:25:58.627204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:25:58.627204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:25:58.627204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:25:58.627204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:25:58.627204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:25:58.627204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:25:58.627204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:25:58.627204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:25:58.627735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:25:58.627735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:25:58.627735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:26:03.438874 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:26:03.439876 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:26:03.440276 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:26:03.441956 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:26:03.479399 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:26:03.480045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:03.481872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:26:03.481872 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:03.481872 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:26:03.481872 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:26:03.481872 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:03.481872 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:26:03.481872 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:03.481872 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:26:03.482504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:03.482504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:26:03.482504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:26:03.482504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:26:03.482504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:26:03.482504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:26:03.482504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:26:03.482504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:03.482504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:26:03.482504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:03.482504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:26:03.482504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:03.483148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:26:03.483148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:03.483148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:26:03.483148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:03.483148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:26:03.483148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:26:03.483148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:03.483148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:26:03.483148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:03.483148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:26:03.483148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:26:03.483148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:03.483148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:26:03.483794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:03.483794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:26:03.483794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:03.483794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:26:03.483794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:26:03.483794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:26:03.483794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:26:03.483794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:26:03.483794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:26:03.483794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:26:03.483794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:26:03.483794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:26:03.483794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:26:03.483794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:26:03.483794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:26:03.484448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:26:03.484448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:26:03.484448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:26:03.484448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:26:03.484448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:26:03.484448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:03.484448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:26:03.484448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:03.484448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:26:03.484448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:03.484448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:26:03.484448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:03.484448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:26:03.484448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:03.484448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:26:03.484448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:26:03.484448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:03.484448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:26:03.484448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:03.485672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:26:03.485672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:26:03.485672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:03.485672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:26:03.485672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:03.485672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:26:03.593246 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:26:03.594129 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:26:03.594344 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:26:03.596075 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:26:03.596621 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:26:03.597024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:03.598211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:26:03.598211 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:03.598211 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:26:03.598211 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:26:03.598211 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:03.598211 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:26:03.598211 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:03.598211 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:26:03.600282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:03.600282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:26:03.600282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:26:03.600282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:26:03.600282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:26:03.600282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:26:03.600282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:26:03.600282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:03.600282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:26:03.600282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:03.600282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:26:03.600282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:03.600889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:26:03.600889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:03.600889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:26:03.600889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:03.600889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:26:03.600889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:26:03.600889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:03.600889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:26:03.600889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:03.600889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:26:03.600889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:26:03.600889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:03.600889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:26:03.601656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:03.601656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:26:03.602719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:03.602719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:26:03.602719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:26:03.602719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:26:03.602719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:26:03.602719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:26:03.602719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:26:03.602719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:26:03.602719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:26:03.602719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:26:03.602719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:26:03.602719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:26:03.602719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:26:03.602719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:26:03.602719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:26:03.602719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:26:03.603670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:26:03.603670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:26:03.603670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:03.603670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:26:03.603670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:03.603670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:26:03.603670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:03.603670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:26:03.603670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:03.603670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:26:03.603670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:03.603670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:26:03.603670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:26:03.603670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:03.603670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:26:03.603670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:03.603670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:26:03.603670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:26:03.603670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:03.604169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:26:03.604169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:03.604169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:26:03.604169 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:26:03.604169 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:26:03.604169 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:26:03.604466 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:26:03.605201 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:26:03.605460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:03.606431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:26:03.606431 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:03.606431 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:26:03.606431 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:26:03.606431 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:03.606431 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:26:03.606431 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:03.606431 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:26:03.607513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:03.607513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:26:03.607513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:26:03.607513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:26:03.607513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:03.607513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:26:03.607513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:03.607513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:26:03.607513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:03.607513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:26:03.607513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:03.607513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:26:03.607513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:03.607513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:26:03.607513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:26:03.607964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:03.607964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:26:03.607964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:03.607964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:26:03.607964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:26:03.607964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:03.607964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:26:03.607964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:03.607964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:26:03.608484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:03.608484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:26:03.608484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:26:03.608484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:26:03.608484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:26:03.608484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:26:03.608484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:26:03.608484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:26:03.608484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:26:03.608484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:26:03.608484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:03.608484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:26:03.608867 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:03.608867 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:26:03.608867 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:03.608867 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:26:03.608867 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:03.608867 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:26:03.608867 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:03.608867 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:26:03.608867 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:26:03.608867 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:03.608867 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:26:03.608867 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:03.608867 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:26:03.608867 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:26:03.609226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:03.609226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:26:03.609226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:03.609226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:26:03.609812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:03.609812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:26:03.609812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:26:03.609812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:26:03.609812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:26:03.609812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:26:03.609812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:26:03.609812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:26:03.609812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:26:03.609812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:26:03.609812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:26:03.609812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:26:03.609812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:26:03.609812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:26:03.609812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:26:03.609812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:26:03.610230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:26:03.610230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:26:03.610230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:26:03.610230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:26:03.610230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:26:03.610230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:03.610230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:26:03.610230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:03.610230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:26:03.610230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:03.610230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:26:03.610230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:03.610230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:26:03.610230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:03.610230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:26:03.610230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:26:03.610230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:03.610230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:26:03.610711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:03.610711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:26:03.610711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:26:03.610711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:03.610711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:26:03.610711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:03.610711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:26:03.613486 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:26:03.613730 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:26:03.613935 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:26:03.614135 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:26:03.614853 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:26:03.615549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:03.616473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:26:03.616473 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:03.616473 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:26:03.616473 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:26:03.616473 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:03.616473 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:26:03.616473 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:03.616473 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:26:03.617351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:03.617351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:26:03.617351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:26:03.617351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:26:03.617351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:03.617351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:26:03.617351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:03.617351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:26:03.617351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:03.617351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:26:03.617351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:03.617351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:26:03.617351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:03.617351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:26:03.617351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:26:03.617956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:03.617956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:26:03.617956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:03.617956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:26:03.617956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:26:03.617956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:03.617956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:26:03.617956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:03.617956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:26:03.618334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:03.618334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:26:03.618334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:26:03.618334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:26:03.618334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:26:03.618334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:26:03.618334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:26:03.618334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:26:03.618334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:26:03.618334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:03.618334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:26:03.618802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:03.618802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:26:03.618802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:03.618802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:26:03.618802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:03.618802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:26:03.618802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:03.618802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:26:03.618802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:26:03.618802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:03.618802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:26:03.618802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:03.618802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:26:03.618802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:26:03.619276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:03.619276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:26:03.619276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:03.619276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:26:03.619914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:03.619914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:26:03.619914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:26:03.619914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:26:03.619914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:26:03.619914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:26:03.619914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:26:03.619914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:26:03.619914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:26:03.619914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:26:03.619914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:26:03.619914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:26:03.619914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:26:03.619914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:26:03.619914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:26:03.619914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:26:03.619914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:26:03.620530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:26:03.620530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:26:03.620530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:26:03.620530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:03.620530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:26:03.620530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:03.620530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:26:03.620530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:03.620530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:26:03.620530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:03.620530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:26:03.620530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:03.620530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:26:03.620530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:26:03.620530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:03.620530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:26:03.620530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:03.620530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:26:03.620530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:26:03.621173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:03.621173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:26:03.621173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:03.621173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:26:03.622962 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:26:03.623261 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:26:03.623367 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:26:03.623549 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:26:03.624784 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:26:03.625313 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:26:03.625587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:03.626607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:26:03.626607 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:03.626607 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:26:03.626607 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:26:03.626607 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:03.626607 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:26:03.626607 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:03.626607 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:26:03.627562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:03.627562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:26:03.627562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:26:03.627562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:26:03.627562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:26:03.627562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:26:03.627562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:26:03.627562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:03.627562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:26:03.627562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:03.627562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:26:03.627562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:03.628135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:26:03.628135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:03.628135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:26:03.628135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:03.628135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:26:03.628135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:26:03.628135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:03.628135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:26:03.628135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:03.628135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:26:03.628135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:26:03.628135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:03.628135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:26:03.628658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:03.628658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:26:03.628931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:03.628931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:26:03.628931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:26:03.628931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:26:03.628931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:26:03.628931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:26:03.628931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:26:03.628931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:26:03.628931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:26:03.628931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:26:03.628931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:26:03.628931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:26:03.628931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:26:03.628931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:26:03.628931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:26:03.628931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:26:03.629558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:26:03.629558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:26:03.629558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:03.629558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:26:03.629558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:03.629558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:26:03.629558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:03.629558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:26:03.629558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:03.629558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:26:03.629558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:03.629558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:26:03.629558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:26:03.629558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:03.629558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:26:03.629558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:03.629558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:26:03.629558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:26:03.629558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:03.630346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:26:03.630346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:03.630346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:26:08.439606 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:26:08.440368 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:26:08.440724 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:26:08.442069 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:26:08.478362 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:26:08.478616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:08.479388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:26:08.479388 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:08.479388 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:26:08.479388 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:26:08.479388 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:08.479388 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:26:08.479388 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:08.479388 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:26:08.480646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:08.480646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:26:08.480646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:26:08.480646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:26:08.480646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:26:08.480646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:26:08.480646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:26:08.480646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:08.480646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:26:08.480646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:08.480646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:26:08.480646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:08.480984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:26:08.480984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:08.480984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:26:08.480984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:08.480984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:26:08.480984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:26:08.480984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:08.480984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:26:08.480984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:08.480984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:26:08.480984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:26:08.480984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:08.480984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:26:08.481231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:08.481231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:26:08.481799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:08.481799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:26:08.481799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:26:08.481799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:26:08.481799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:26:08.481799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:26:08.481799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:26:08.481799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:26:08.481799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:26:08.481799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:26:08.481799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:26:08.481799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:26:08.481799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:26:08.481799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:26:08.481799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:26:08.481799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:26:08.482332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:26:08.482332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:26:08.482332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:08.482332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:26:08.482332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:08.482332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:26:08.482332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:08.482332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:26:08.482332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:08.482332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:26:08.482332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:08.482332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:26:08.482332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:26:08.482332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:08.482332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:26:08.482332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:08.482332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:26:08.482332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:26:08.482332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:08.482916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:26:08.482916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:08.482916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:26:08.595486 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:26:08.596239 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:26:08.596353 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:26:08.597534 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:26:08.597921 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:26:08.598109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:08.599631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:26:08.599631 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:08.599631 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:26:08.599631 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:26:08.599631 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:08.599631 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:26:08.599631 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:08.599631 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:26:08.600261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:08.600261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:26:08.600261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:26:08.600261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:26:08.600261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:26:08.600261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:26:08.600261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:26:08.600261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:08.600261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:26:08.600261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:08.600261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:26:08.600261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:08.601182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:26:08.601182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:08.601182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:26:08.601182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:08.601182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:26:08.601182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:26:08.601182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:08.601182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:26:08.601182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:08.601182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:26:08.601182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:26:08.601182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:08.601182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:26:08.602127 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:08.602127 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:26:08.602127 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:08.602127 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:26:08.602127 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:26:08.602127 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:26:08.602127 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:26:08.602127 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:26:08.602127 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:26:08.602127 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:26:08.602127 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:26:08.602127 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:26:08.602127 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:26:08.602127 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:26:08.602127 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:26:08.603171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:26:08.603171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:26:08.603171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:26:08.603171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:26:08.603171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:26:08.603171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:08.603171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:26:08.603171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:08.603171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:26:08.603171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:08.603171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:26:08.603171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:08.603171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:26:08.603171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:08.603171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:26:08.603171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:26:08.603171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:08.603171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:26:08.603171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:08.604261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:26:08.604261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:26:08.604261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:08.604261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:26:08.604261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:08.604261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:26:08.604261 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:26:08.604786 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:26:08.604786 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:26:08.604786 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:26:08.605217 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:26:08.605661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:08.606725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:26:08.606725 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:08.606725 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:26:08.606725 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:26:08.606725 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:08.606725 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:26:08.606725 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:08.606725 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:26:08.607576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:08.607576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:26:08.607576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:26:08.607576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:26:08.607576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:08.607576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:26:08.607576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:08.607576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:26:08.607576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:08.607576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:26:08.607576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:08.607576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:26:08.607576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:08.607576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:26:08.607576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:26:08.608008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:08.608008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:26:08.608008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:08.608008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:26:08.608008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:26:08.608008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:08.608008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:26:08.608008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:08.608008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:26:08.608518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:08.608518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:26:08.608518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:26:08.608518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:26:08.608518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:26:08.608518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:26:08.608518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:26:08.608518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:26:08.608518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:26:08.608518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:26:08.608518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:08.608518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:26:08.608950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:08.608950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:26:08.608950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:08.608950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:26:08.608950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:08.608950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:26:08.608950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:08.608950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:26:08.608950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:26:08.608950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:08.608950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:26:08.608950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:08.608950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:26:08.608950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:26:08.609399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:08.609399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:26:08.609399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:08.609399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:26:08.609774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:08.609774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:26:08.609774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:26:08.609774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:26:08.609774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:26:08.609774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:26:08.609774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:26:08.609774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:26:08.609774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:26:08.609774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:26:08.609774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:26:08.609774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:26:08.609774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:26:08.609774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:26:08.609774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:26:08.609774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:26:08.610378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:26:08.610378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:26:08.610378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:26:08.610378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:26:08.610378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:26:08.610378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:08.610378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:26:08.610378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:08.610378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:26:08.610378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:08.610378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:26:08.610378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:08.610378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:26:08.610378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:08.610378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:26:08.610378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:26:08.610378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:08.610378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:26:08.610906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:08.610906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:26:08.610906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:26:08.610906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:08.610906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:26:08.610906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:08.610906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:26:08.614014 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:26:08.614387 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:26:08.614529 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:26:08.614723 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:26:08.615435 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:26:08.615810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:08.616416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:26:08.616416 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:08.616416 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:26:08.616416 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:26:08.616416 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:08.616416 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:26:08.616416 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:08.616416 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:26:08.617283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:08.617283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:26:08.617283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:26:08.617283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:26:08.617283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:08.617283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:26:08.617283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:08.617283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:26:08.617283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:08.617283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:26:08.617283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:08.617283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:26:08.617283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:08.617283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:26:08.617283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:26:08.618007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:08.618007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:26:08.618007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:08.618007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:26:08.618007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:26:08.618007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:08.618007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:26:08.618007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:08.618007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:26:08.618552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:08.618552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:26:08.618552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:26:08.618552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:26:08.618552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:26:08.618552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:26:08.618552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:26:08.618552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:26:08.618552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:26:08.618552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:08.618552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:26:08.619022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:08.619022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:26:08.619022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:08.619022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:26:08.619022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:08.619022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:26:08.619022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:08.619022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:26:08.619022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:26:08.619022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:08.619022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:26:08.619022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:08.619022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:26:08.619022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:26:08.619537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:08.619537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:26:08.619537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:08.619537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:26:08.620300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:08.620300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:26:08.620300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:26:08.620300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:26:08.620300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:26:08.620300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:26:08.620300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:26:08.620300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:26:08.620300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:26:08.620300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:26:08.620300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:26:08.620300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:26:08.620300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:26:08.620300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:26:08.620300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:26:08.620300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:26:08.620300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:26:08.620996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:26:08.620996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:26:08.620996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:26:08.620996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:08.620996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:26:08.620996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:08.620996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:26:08.620996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:08.620996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:26:08.620996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:08.620996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:26:08.620996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:08.620996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:26:08.620996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:26:08.620996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:08.620996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:26:08.620996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:08.620996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:26:08.620996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:26:08.621771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:08.621771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:26:08.621771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:08.621771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:26:08.623560 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:26:08.624060 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:26:08.624226 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:26:08.624466 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:26:08.624921 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:26:08.625171 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:26:08.625397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:08.626194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:26:08.626194 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:08.626194 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:26:08.626194 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:26:08.626194 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:08.626194 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:26:08.626194 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:08.626194 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:26:08.627108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:08.627108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:26:08.627108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:26:08.627108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:26:08.627108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:26:08.627108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:26:08.627108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:26:08.627108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:08.627108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:26:08.627108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:08.627108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:26:08.627108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:08.627382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:26:08.627382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:08.627382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:26:08.627382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:08.627382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:26:08.627382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:26:08.627382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:08.627382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:26:08.627382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:08.627382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:26:08.627382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:26:08.627382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:08.627382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:26:08.627669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:08.627669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:26:08.628192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:08.628192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:26:08.628192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:26:08.628192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:26:08.628192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:26:08.628192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:26:08.628192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:26:08.628192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:26:08.628192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:26:08.628192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:26:08.628192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:26:08.628192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:26:08.628192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:26:08.628192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:26:08.628192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:26:08.628192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:26:08.628603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:26:08.628603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:26:08.628603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:08.628603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:26:08.628603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:08.628603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:26:08.628603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:08.628603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:26:08.628603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:08.628603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:26:08.628603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:08.628603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:26:08.628603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:26:08.628603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:08.628603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:26:08.628603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:08.628603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:26:08.628603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:26:08.628603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:08.629102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:26:08.629102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:08.629102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:26:13.440460 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:26:13.441185 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:26:13.441185 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:26:13.441738 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:26:13.476438 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:26:13.476851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:13.477498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:26:13.477498 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:13.477498 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:26:13.477498 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:26:13.477498 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:13.477498 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:26:13.477498 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:13.477498 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:26:13.478205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:13.478205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:26:13.478205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:26:13.478205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:26:13.478205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:26:13.478205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:26:13.478205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:26:13.478205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:13.478205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:26:13.478205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:13.478205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:26:13.478205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:13.479015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:26:13.479015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:13.479015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:26:13.479015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:13.479015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:26:13.479015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:26:13.479015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:13.479015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:26:13.479015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:13.479015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:26:13.479015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:26:13.479015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:13.479015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:26:13.479795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:13.479795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:26:13.479795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:13.479795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:26:13.479795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:26:13.479795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:26:13.479795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:26:13.479795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:26:13.479795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:26:13.479795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:26:13.479795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:26:13.479795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:26:13.479795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:26:13.479795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:26:13.479795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:26:13.480310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:26:13.480310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:26:13.480310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:26:13.480310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:26:13.480310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:26:13.480310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:13.480310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:26:13.480310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:13.480310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:26:13.480310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:13.480310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:26:13.480310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:13.480310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:26:13.480310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:13.480310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:26:13.480310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:26:13.480310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:13.480310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:26:13.480310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:13.480873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:26:13.480873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:26:13.480873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:13.480873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:26:13.480873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:13.480873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:26:13.596230 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:26:13.596620 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:26:13.596712 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:26:13.597551 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:26:13.597871 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:26:13.598123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:13.598874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:26:13.598874 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:13.598874 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:26:13.598874 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:26:13.598874 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:13.598874 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:26:13.598874 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:13.598874 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:26:13.599907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:13.599907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:26:13.599907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:26:13.599907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:26:13.599907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:26:13.599907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:26:13.599907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:26:13.599907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:13.599907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:26:13.599907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:13.599907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:26:13.599907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:13.600560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:26:13.600560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:13.600560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:26:13.600560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:13.600560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:26:13.600560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:26:13.600560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:13.600560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:26:13.600560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:13.600560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:26:13.600560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:26:13.600560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:13.600560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:26:13.601303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:13.601303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:26:13.601679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:13.601679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:26:13.601679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:26:13.601679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:26:13.601679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:26:13.601679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:26:13.601679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:26:13.601679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:26:13.601679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:26:13.601679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:26:13.601679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:26:13.601679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:26:13.601679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:26:13.601679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:26:13.601679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:26:13.601679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:26:13.602575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:26:13.602575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:26:13.602575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:13.602575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:26:13.602575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:13.602575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:26:13.602575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:13.602575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:26:13.602575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:13.602575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:26:13.602575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:13.602575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:26:13.602575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:26:13.602575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:13.602575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:26:13.602575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:13.602575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:26:13.602575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:26:13.602575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:13.604066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:26:13.604066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:13.604066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:26:13.604434 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:26:13.604872 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:26:13.604974 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:26:13.605168 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:26:13.605660 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:26:13.605997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:13.606709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:26:13.606709 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:13.606709 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:26:13.606709 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:26:13.606709 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:13.606709 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:26:13.606709 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:13.606709 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:26:13.607869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:13.607869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:26:13.607869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:26:13.607869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:26:13.607869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:13.607869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:26:13.607869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:13.607869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:26:13.607869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:13.607869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:26:13.607869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:13.607869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:26:13.607869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:13.607869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:26:13.607869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:26:13.608372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:13.608372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:26:13.608372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:13.608372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:26:13.608372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:26:13.608372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:13.608372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:26:13.608372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:13.608372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:26:13.609205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:13.609205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:26:13.609205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:26:13.609205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:26:13.609205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:26:13.609205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:26:13.609205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:26:13.609205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:26:13.609205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:26:13.609205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:26:13.609205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:13.609205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:26:13.610234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:13.610234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:26:13.610234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:13.610234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:26:13.610234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:13.610234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:26:13.610234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:13.610234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:26:13.610234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:26:13.610234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:13.610234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:26:13.610234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:13.610234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:26:13.610234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:26:13.611173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:13.611173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:26:13.611173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:13.611173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:26:13.611173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:13.611173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:26:13.611173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:26:13.611173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:26:13.611173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:26:13.611173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:26:13.611173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:26:13.611173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:26:13.611173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:26:13.611173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:26:13.611595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:26:13.611595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:26:13.611595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:26:13.611595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:26:13.611595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:26:13.611595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:26:13.611595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:26:13.611595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:26:13.611595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:26:13.611595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:26:13.611595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:26:13.611595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:13.611595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:26:13.611595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:13.611595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:26:13.611595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:13.611595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:26:13.611595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:13.611595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:26:13.612228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:13.612228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:26:13.612228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:26:13.612228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:13.612228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:26:13.612228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:13.612228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:26:13.612228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:26:13.612228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:13.612228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:26:13.612228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:13.612228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:26:13.614480 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:26:13.614739 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:26:13.614885 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:26:13.614987 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:26:13.615400 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:26:13.615772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:13.616676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:26:13.616676 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:13.616676 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:26:13.616676 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:26:13.616676 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:13.616676 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:26:13.616676 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:13.616676 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:26:13.617415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:13.617415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:26:13.617415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:26:13.617415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:26:13.617415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:13.617415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:26:13.617415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:13.617415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:26:13.617415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:13.617415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:26:13.617415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:13.617415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:26:13.617415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:13.617415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:26:13.617415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:26:13.618180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:13.618180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:26:13.618180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:13.618180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:26:13.618180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:26:13.618180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:13.618180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:26:13.618180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:13.618180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:26:13.618467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:13.618467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:26:13.618467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:26:13.618467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:26:13.618467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:26:13.618467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:26:13.618467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:26:13.618467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:26:13.618467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:26:13.618467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:13.618467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:26:13.618925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:13.618925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:26:13.618925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:13.618925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:26:13.618925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:13.618925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:26:13.618925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:13.618925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:26:13.618925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:26:13.618925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:13.618925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:26:13.618925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:13.618925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:26:13.618925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:26:13.619382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:13.619382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:26:13.619382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:13.619382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:26:13.619516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:13.619516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:26:13.619516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:26:13.619516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:26:13.619516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:26:13.619516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:26:13.619516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:26:13.619516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:26:13.619516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:26:13.619516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:26:13.619516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:26:13.619516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:26:13.619516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:26:13.619516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:26:13.619516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:26:13.619516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:26:13.619516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:26:13.620209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:26:13.620209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:26:13.620209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:26:13.620209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:13.620209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:26:13.620209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:13.620209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:26:13.620209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:13.620209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:26:13.620209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:13.620209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:26:13.620209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:13.620209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:26:13.620209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:26:13.620209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:13.620209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:26:13.620209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:13.620209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:26:13.620209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:26:13.620726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:13.620726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:26:13.620726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:13.620726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:26:13.624010 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:26:13.624337 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:26:13.624443 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:26:13.624592 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:26:13.625044 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:26:13.625389 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:26:13.625621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:13.626439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:26:13.626439 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:13.626439 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:26:13.626439 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:26:13.626439 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:13.626439 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:26:13.626439 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:13.626439 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:26:13.627378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:13.627378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:26:13.627378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:26:13.627378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:26:13.627378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:26:13.627378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:26:13.627378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:26:13.627378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:13.627378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:26:13.627378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:13.627378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:26:13.627378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:13.628375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:26:13.628375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:13.628375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:26:13.628375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:13.628375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:26:13.628375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:26:13.628375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:13.628375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:26:13.628375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:13.628375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:26:13.628375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:26:13.628375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:13.628375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:26:13.629413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:13.629413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:26:13.629413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:13.629413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:26:13.629413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:26:13.629413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:26:13.629413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:26:13.629413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:26:13.629413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:26:13.629413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:26:13.629413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:26:13.629413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:26:13.629413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:26:13.629413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:26:13.629413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:26:13.630624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:26:13.630624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:26:13.630624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:26:13.630624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:26:13.630624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:26:13.630624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:13.630624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:26:13.630624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:13.630624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:26:13.630624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:13.630624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:26:13.630624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:13.630624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:26:13.630624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:13.630624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:26:13.630624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:26:13.630624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:13.630624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:26:13.630624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:13.632140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:26:13.632140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:26:13.632140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:13.632140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:26:13.632140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:13.632140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:26:18.441458 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:26:18.442738 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:26:18.442944 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:26:18.444183 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:26:18.482465 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:26:18.482890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:18.483596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:26:18.483596 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:18.483596 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:26:18.483596 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:26:18.483596 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:18.483596 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:26:18.483596 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:18.483596 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:26:18.484372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:18.484372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:26:18.484372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:26:18.484372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:26:18.484372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:26:18.484372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:26:18.484372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:26:18.484372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:18.484372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:26:18.484372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:18.484372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:26:18.484372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:18.484741 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:26:18.484741 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:18.484741 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:26:18.484741 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:18.484741 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:26:18.484741 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:26:18.484741 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:18.484741 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:26:18.484741 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:18.484741 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:26:18.484741 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:26:18.484741 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:18.484741 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:26:18.485143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:18.485143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:26:18.485143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:18.485143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:26:18.485143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:26:18.485143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:26:18.485143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:26:18.485143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:26:18.485143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:26:18.485143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:26:18.485143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:26:18.485143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:26:18.485143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:26:18.485143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:26:18.485143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:26:18.488831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:26:18.488831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:26:18.488831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:26:18.488831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:26:18.488831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:26:18.488831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:18.488831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:26:18.488831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:18.488831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:26:18.488831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:18.488831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:26:18.488831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:18.488831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:26:18.488831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:18.488831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:26:18.488831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:26:18.488831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:18.488831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:26:18.488831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:18.489690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:26:18.489690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:26:18.489690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:18.489690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:26:18.489690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:18.489690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:26:18.597094 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:26:18.597709 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:26:18.597963 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:26:18.599361 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:26:18.599596 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:26:18.599767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:18.600505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:26:18.600505 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:18.600505 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:26:18.600505 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:26:18.600505 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:18.600505 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:26:18.600505 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:18.600505 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:26:18.602074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:18.602074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:26:18.602074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:26:18.602074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:26:18.602074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:26:18.602074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:26:18.602074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:26:18.602074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:18.602074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:26:18.602074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:18.602074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:26:18.602074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:18.602430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:26:18.602430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:18.602430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:26:18.602430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:18.602430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:26:18.602430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:26:18.602430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:18.602430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:26:18.602430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:18.602430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:26:18.602430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:26:18.602430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:18.602430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:26:18.602713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:18.602713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:26:18.604132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:18.604132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:26:18.604132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:26:18.604132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:26:18.604132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:26:18.604132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:26:18.604132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:26:18.604132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:26:18.604132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:26:18.604132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:26:18.604132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:26:18.604132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:26:18.604132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:26:18.604132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:26:18.604132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:26:18.604132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:26:18.604569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:26:18.604569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:26:18.604569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:18.604569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:26:18.604569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:18.604569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:26:18.604569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:18.604569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:26:18.604569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:18.604569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:26:18.604569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:18.604569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:26:18.604569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:26:18.604569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:18.604569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:26:18.604569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:18.604569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:26:18.604569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:26:18.604569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:18.605051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:26:18.605051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:18.605051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:26:18.605051 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:26:18.605346 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:26:18.605584 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:26:18.605864 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:26:18.606353 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:26:18.606554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:18.607487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:26:18.607487 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:18.607487 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:26:18.607487 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:26:18.607487 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:18.607487 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:26:18.607487 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:18.607487 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:26:18.608862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:18.608862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:26:18.608862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:26:18.608862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:26:18.608862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:18.608862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:26:18.608862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:18.608862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:26:18.608862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:18.608862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:26:18.608862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:18.608862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:26:18.608862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:18.608862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:26:18.608862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:26:18.609262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:18.609262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:26:18.609262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:18.609262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:26:18.609262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:26:18.609262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:18.609262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:26:18.609262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:18.609262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:26:18.609770 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:18.609770 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:26:18.609770 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:26:18.609770 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:26:18.609770 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:26:18.609770 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:26:18.609770 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:26:18.609770 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:26:18.609770 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:26:18.609770 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:26:18.609770 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:18.609770 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:26:18.610089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:18.610089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:26:18.610089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:18.610089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:26:18.610089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:18.610089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:26:18.610089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:18.610089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:26:18.610089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:26:18.610089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:18.610089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:26:18.610089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:18.610089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:26:18.610089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:26:18.610439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:18.610439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:26:18.610439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:18.610439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:26:18.611006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:18.611006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:26:18.611006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:26:18.611006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:26:18.611006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:26:18.611006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:26:18.611006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:26:18.611006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:26:18.611006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:26:18.611006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:26:18.611006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:26:18.611006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:26:18.611006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:26:18.611006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:26:18.611006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:26:18.611006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:26:18.611464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:26:18.611464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:26:18.611464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:26:18.611464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:26:18.611464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:26:18.611464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:18.611464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:26:18.611464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:18.611464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:26:18.611464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:18.611464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:26:18.611464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:18.611464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:26:18.611464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:18.611464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:26:18.611464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:26:18.611464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:18.611464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:26:18.611941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:18.611941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:26:18.611941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:26:18.611941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:18.611941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:26:18.611941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:18.611941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:26:18.615126 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:26:18.615523 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:26:18.615666 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:26:18.615878 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:26:18.616415 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:26:18.616835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:18.617506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:26:18.617506 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:18.617506 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:26:18.617506 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:26:18.617506 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:18.617506 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:26:18.617506 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:18.617506 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:26:18.618474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:18.618474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:26:18.618474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:26:18.618474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:26:18.618474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:18.618474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:26:18.618474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:18.618474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:26:18.618474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:18.618474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:26:18.618474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:18.618474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:26:18.618474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:18.618474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:26:18.618474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:26:18.618913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:18.618913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:26:18.618913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:18.618913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:26:18.618913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:26:18.618913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:18.618913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:26:18.618913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:18.618913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:26:18.619459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:18.619459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:26:18.619459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:26:18.619459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:26:18.619459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:26:18.619459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:26:18.619459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:26:18.619459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:26:18.619459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:26:18.619459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:18.619459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:26:18.619796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:18.619796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:26:18.619796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:18.619796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:26:18.619796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:18.619796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:26:18.619796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:18.619796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:26:18.619796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:26:18.619796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:18.619796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:26:18.619796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:18.619796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:26:18.619796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:26:18.620123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:18.620123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:26:18.620123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:18.620123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:26:18.620649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:18.620649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:26:18.620649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:26:18.620649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:26:18.620649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:26:18.620649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:26:18.620649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:26:18.620649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:26:18.620649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:26:18.620649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:26:18.620649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:26:18.620649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:26:18.620649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:26:18.620649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:26:18.620649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:26:18.620649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:26:18.620649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:26:18.621103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:26:18.621103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:26:18.621103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:26:18.621103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:18.621103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:26:18.621103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:18.621103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:26:18.621103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:18.621103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:26:18.621103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:18.621103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:26:18.621103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:18.621103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:26:18.621103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:26:18.621103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:18.621103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:26:18.621103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:18.621103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:26:18.621103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:26:18.621558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:18.621558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:26:18.621558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:18.621558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:26:18.624650 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:26:18.625044 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:26:18.625159 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:26:18.625323 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:26:18.626224 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:26:18.626761 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:26:18.627154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:18.628119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:26:18.628119 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:18.628119 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:26:18.628119 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:26:18.628119 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:18.628119 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:26:18.628119 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:18.628119 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:26:18.629026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:18.629026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:26:18.629026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:26:18.629026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:26:18.629026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:26:18.629026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:26:18.629026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:26:18.629026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:18.629026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:26:18.629026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:18.629026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:26:18.629026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:18.629393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:26:18.629393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:18.629393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:26:18.629393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:18.629393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:26:18.629393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:26:18.629393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:18.629393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:26:18.629393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:18.629393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:26:18.629393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:26:18.629393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:18.629393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:26:18.629711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:18.629711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:26:18.630051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:18.630051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:26:18.630051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:26:18.630051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:26:18.630051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:26:18.630051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:26:18.630051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:26:18.630051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:26:18.630051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:26:18.630051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:26:18.630051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:26:18.630051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:26:18.630051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:26:18.630051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:26:18.630051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:26:18.630051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:26:18.630464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:26:18.630464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:26:18.630464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:18.630464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:26:18.630464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:18.630464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:26:18.630464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:18.630464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:26:18.630464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:18.630464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:26:18.630464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:18.630464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:26:18.630464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:26:18.630464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:18.630464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:26:18.630464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:18.630464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:26:18.630464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:26:18.630464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:18.630974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:26:18.630974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:18.630974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:26:23.442769 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:26:23.443450 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:26:23.443501 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:26:23.444672 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:26:23.477851 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:26:23.478036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:23.478429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:26:23.478429 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:23.478429 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:26:23.478429 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:26:23.478429 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:23.478429 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:26:23.478429 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:23.478429 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:26:23.479087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:23.479087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:26:23.479087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:26:23.479087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:26:23.479087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:26:23.479087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:26:23.479087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:26:23.479087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:23.479087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:26:23.479087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:23.479087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:26:23.479087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:23.479482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:26:23.479482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:23.479482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:26:23.479482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:23.479482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:26:23.479482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:26:23.479482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:23.479482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:26:23.479482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:23.479482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:26:23.479482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:26:23.479482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:23.479482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:26:23.479818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:23.479818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:26:23.479896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:23.479896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:26:23.479896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:26:23.479896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:26:23.479896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:26:23.479896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:26:23.479896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:26:23.479896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:26:23.479896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:26:23.479896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:26:23.479896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:26:23.479896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:26:23.479896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:26:23.479896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:26:23.479896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:26:23.479896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:26:23.480315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:26:23.480315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:26:23.480315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:23.480315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:26:23.480315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:23.480315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:26:23.480315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:23.480315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:26:23.480315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:23.480315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:26:23.480315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:23.480315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:26:23.480315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:26:23.480315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:23.480315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:26:23.480315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:23.480315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:26:23.480315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:26:23.480315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:23.480743 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:26:23.480743 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:23.480743 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:26:23.597882 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:26:23.598451 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:26:23.598573 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:26:23.599482 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:26:23.599844 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:26:23.600099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:23.600846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:26:23.600846 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:23.600846 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:26:23.600846 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:26:23.600846 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:23.600846 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:26:23.600846 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:23.600846 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:26:23.601803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:23.601803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:26:23.601803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:26:23.601803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:26:23.601803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:26:23.601803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:26:23.601803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:26:23.601803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:23.601803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:26:23.601803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:23.601803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:26:23.601803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:23.602410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:26:23.602410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:23.602410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:26:23.602410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:23.602410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:26:23.602410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:26:23.602410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:23.602410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:26:23.602410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:23.602410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:26:23.602410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:26:23.602410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:23.602410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:26:23.603016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:23.603016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:26:23.603016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:23.603016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:26:23.603016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:26:23.603016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:26:23.603016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:26:23.603016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:26:23.603016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:26:23.603016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:26:23.603016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:26:23.603016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:26:23.603016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:26:23.603016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:26:23.603016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:26:23.603841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:26:23.603841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:26:23.603841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:26:23.603841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:26:23.603841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:26:23.603841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:23.603841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:26:23.603841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:23.603841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:26:23.603841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:23.603841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:26:23.603841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:23.603841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:26:23.603841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:23.603841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:26:23.603841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:26:23.603841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:23.603841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:26:23.603841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:23.604669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:26:23.604669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:26:23.604669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:23.604669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:26:23.604669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:23.604669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:26:23.605451 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:26:23.605772 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:26:23.605879 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:26:23.606011 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:26:23.606463 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:26:23.606868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:23.607395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:26:23.607395 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:23.607395 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:26:23.607395 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:26:23.607395 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:23.607395 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:26:23.607395 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:23.607395 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:26:23.608188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:23.608188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:26:23.608188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:26:23.608188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:26:23.608188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:23.608188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:26:23.608188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:23.608188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:26:23.608188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:23.608188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:26:23.608188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:23.608188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:26:23.608188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:23.608188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:26:23.608188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:26:23.609041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:23.609041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:26:23.609041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:23.609041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:26:23.609041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:26:23.609041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:23.609041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:26:23.609041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:23.609041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:26:23.609041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:23.609041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:26:23.609041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:26:23.609041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:26:23.609041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:26:23.609817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:26:23.609817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:26:23.609817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:26:23.609817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:26:23.609817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:26:23.609817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:23.609817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:26:23.609817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:23.609817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:26:23.609817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:23.609817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:26:23.609817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:23.609817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:26:23.610529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:23.610529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:26:23.610529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:26:23.610529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:23.610529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:26:23.610529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:23.610529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:26:23.610529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:26:23.610529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:23.610529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:26:23.610529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:23.610529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:26:23.611223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:23.611223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:26:23.611223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:26:23.611223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:26:23.611223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:26:23.611223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:26:23.611223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:26:23.611223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:26:23.611223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:26:23.611223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:26:23.611223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:26:23.611223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:26:23.611223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:26:23.611223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:26:23.611223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:26:23.611223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:26:23.612150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:26:23.612150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:26:23.612150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:26:23.612150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:26:23.612150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:26:23.612150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:23.612150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:26:23.612150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:23.612150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:26:23.612150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:23.612150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:26:23.612150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:23.612150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:26:23.612150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:23.612150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:26:23.612150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:26:23.612150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:23.612150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:26:23.613079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:23.613079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:26:23.613079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:26:23.613079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:23.613079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:26:23.613079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:23.613079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:26:23.615619 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:26:23.615916 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:26:23.616002 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:26:23.616150 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:26:23.616603 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:26:23.616989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:23.617703 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:26:23.617703 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:23.617703 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:26:23.617703 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:26:23.617703 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:23.617703 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:26:23.617703 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:23.617703 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:26:23.618764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:23.618764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:26:23.618764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:26:23.618764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:26:23.618764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:23.618764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:26:23.618764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:23.618764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:26:23.618764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:23.618764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:26:23.618764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:23.618764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:26:23.618764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:23.618764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:26:23.618764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:26:23.619176 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:23.619176 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:26:23.619176 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:23.619176 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:26:23.619176 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:26:23.619176 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:23.619176 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:26:23.619176 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:23.619176 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:26:23.619762 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:23.619762 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:26:23.619762 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:26:23.619762 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:26:23.619762 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:26:23.619762 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:26:23.619762 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:26:23.619762 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:26:23.619762 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:26:23.619762 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:23.619762 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:26:23.620135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:23.620135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:26:23.620135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:23.620135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:26:23.620135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:23.620135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:26:23.620135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:23.620135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:26:23.620135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:26:23.620135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:23.620135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:26:23.620135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:23.620135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:26:23.620135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:26:23.620516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:23.620516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:26:23.620516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:23.620516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:26:23.620942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:23.620942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:26:23.620942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:26:23.620942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:26:23.620942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:26:23.620942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:26:23.620942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:26:23.620942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:26:23.620942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:26:23.620942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:26:23.620942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:26:23.620942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:26:23.620942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:26:23.620942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:26:23.620942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:26:23.620942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:26:23.620942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:26:23.621439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:26:23.621439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:26:23.621439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:26:23.621439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:23.621439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:26:23.621439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:23.621439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:26:23.621439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:23.621439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:26:23.621439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:23.621439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:26:23.621439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:23.621439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:26:23.621439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:26:23.621439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:23.621439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:26:23.621439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:23.621439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:26:23.621439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:26:23.622148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:23.622148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:26:23.622148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:23.622148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:26:23.625238 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:26:23.625625 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:26:23.625841 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:26:23.626143 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:26:23.626594 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:26:23.627012 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:26:23.627304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:23.628207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:26:23.628207 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:23.628207 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:26:23.628207 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:26:23.628207 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:23.628207 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:26:23.628207 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:23.628207 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:26:23.629631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:23.629631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:26:23.629631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:26:23.629631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:26:23.629631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:26:23.629631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:26:23.629631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:26:23.629631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:23.629631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:26:23.629631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:23.629631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:26:23.629631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:23.630083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:26:23.630083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:23.630083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:26:23.630083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:23.630083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:26:23.630083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:26:23.630083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:23.630083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:26:23.630083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:23.630083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:26:23.630083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:26:23.630083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:23.630083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:26:23.630392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:23.630392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:26:23.631572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:23.631572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:26:23.631572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:26:23.631572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:26:23.631572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:26:23.631572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:26:23.631572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:26:23.631572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:26:23.631572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:26:23.631572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:26:23.631572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:26:23.631572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:26:23.631572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:26:23.631572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:26:23.631572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:26:23.631572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:26:23.632076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:26:23.632076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:26:23.632076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:23.632076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:26:23.632076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:23.632076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:26:23.632076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:23.632076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:26:23.632076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:23.632076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:26:23.632076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:23.632076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:26:23.632076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:26:23.632076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:23.632076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:26:23.632076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:23.632076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:26:23.632076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:26:23.632076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:23.632612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:26:23.632612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:23.632612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:26:28.443573 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:26:28.444674 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:26:28.444929 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:26:28.446275 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:26:28.479224 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:26:28.480108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:28.480468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:26:28.480468 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:28.480468 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:26:28.480468 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:26:28.480468 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:28.480468 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:26:28.480468 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:28.480468 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:26:28.481821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:28.481821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:26:28.481821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:26:28.481821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:26:28.481821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:26:28.481821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:26:28.481821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:26:28.481821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:28.481821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:26:28.481821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:28.481821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:26:28.481821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:28.486214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:26:28.486214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:28.486214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:26:28.486214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:28.486214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:26:28.486214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:26:28.486214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:28.486214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:26:28.486214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:28.486214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:26:28.486214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:26:28.486214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:28.486214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:26:28.486623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:28.486623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:26:28.486623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:28.486623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:26:28.486623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:26:28.486623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:26:28.486623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:26:28.486623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:26:28.486623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:26:28.486623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:26:28.486623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:26:28.486623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:26:28.486623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:26:28.486623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:26:28.486623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:26:28.487093 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:26:28.487093 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:26:28.487093 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:26:28.487093 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:26:28.487093 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:26:28.487093 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:28.487093 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:26:28.487093 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:28.487093 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:26:28.487093 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:28.487093 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:26:28.487093 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:28.487093 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:26:28.487093 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:28.487093 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:26:28.487093 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:26:28.487093 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:28.487093 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:26:28.487093 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:28.488058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:26:28.488058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:26:28.488058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:28.488058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:26:28.488058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:28.488058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:26:28.598661 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:26:28.599331 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:26:28.599537 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:26:28.600768 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:26:28.601136 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:26:28.601388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:28.602569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:26:28.602569 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:28.602569 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:26:28.602569 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:26:28.602569 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:28.602569 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:26:28.602569 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:28.602569 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:26:28.604206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:28.604206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:26:28.604206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:26:28.604206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:26:28.604206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:26:28.604206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:26:28.604206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:26:28.604206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:28.604206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:26:28.604206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:28.604206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:26:28.604206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:28.604729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:26:28.604729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:28.604729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:26:28.604729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:28.604729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:26:28.604729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:26:28.604729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:28.604729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:26:28.604729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:28.604729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:26:28.604729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:26:28.604729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:28.604729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:26:28.605081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:28.605081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:26:28.606253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:28.606253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:26:28.606253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:26:28.606253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:26:28.606253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:26:28.606253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:26:28.606253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:26:28.606253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:26:28.606253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:26:28.606253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:26:28.606253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:26:28.606253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:26:28.606253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:26:28.606253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:26:28.606253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:26:28.606253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:26:28.606737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:26:28.606737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:26:28.606737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:28.606737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:26:28.606737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:28.606737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:26:28.606737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:28.606737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:26:28.606737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:28.606737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:26:28.606737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:28.606737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:26:28.606737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:26:28.606737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:28.606737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:26:28.606737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:28.606737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:26:28.606737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:26:28.606737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:28.607240 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:26:28.607240 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:28.607240 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:26:28.607240 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:26:28.607240 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:26:28.607240 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:26:28.607609 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:26:28.608043 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:26:28.608387 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:28.609006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:26:28.609006 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:28.609006 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:26:28.609006 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:26:28.609006 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:28.609006 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:26:28.609006 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:28.609006 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:26:28.609852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:28.609852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:26:28.609852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:26:28.609852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:26:28.609852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:28.609852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:26:28.609852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:28.609852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:26:28.609852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:28.609852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:26:28.609852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:28.609852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:26:28.609852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:28.609852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:26:28.609852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:26:28.611036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:28.611036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:26:28.611036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:28.611036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:26:28.611036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:26:28.611036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:28.611036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:26:28.611036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:28.611036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:26:28.611036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:28.611036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:26:28.611036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:26:28.611036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:26:28.611036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:26:28.611791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:26:28.611791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:26:28.611791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:26:28.611791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:26:28.611791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:26:28.611791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:28.611791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:26:28.611791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:28.611791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:26:28.611791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:28.611791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:26:28.611791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:28.611791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:26:28.612523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:28.612523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:26:28.612523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:26:28.612523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:28.612523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:26:28.612523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:28.612523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:26:28.612523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:26:28.612523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:28.612523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:26:28.612523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:28.612523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:26:28.612857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:28.612857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:26:28.612857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:26:28.612857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:26:28.612857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:26:28.612857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:26:28.612857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:26:28.612857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:26:28.612857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:26:28.612857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:26:28.612857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:26:28.612857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:26:28.612857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:26:28.612857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:26:28.612857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:26:28.612857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:26:28.613254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:26:28.613254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:26:28.613254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:26:28.613254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:26:28.613254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:26:28.613254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:28.613254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:26:28.613254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:28.613254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:26:28.613254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:28.613254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:26:28.613254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:28.613254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:26:28.613254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:28.613254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:26:28.613254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:26:28.613254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:28.613254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:26:28.613706 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:28.613706 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:26:28.613706 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:26:28.613706 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:28.613706 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:26:28.613706 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:28.613706 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:26:28.616168 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:26:28.616517 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:26:28.616639 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:26:28.616809 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:26:28.617270 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:26:28.617669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:28.618386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:26:28.618386 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:28.618386 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:26:28.618386 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:26:28.618386 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:28.618386 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:26:28.618386 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:28.618386 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:26:28.619193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:28.619193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:26:28.619193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:26:28.619193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:26:28.619193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:28.619193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:26:28.619193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:28.619193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:26:28.619193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:28.619193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:26:28.619193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:28.619193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:26:28.619193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:28.619193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:26:28.619193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:26:28.620382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:28.620382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:26:28.620382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:28.620382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:26:28.620382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:26:28.620382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:28.620382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:26:28.620382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:28.620382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:26:28.620382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:28.620382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:26:28.620382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:26:28.620382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:26:28.620382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:26:28.621184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:26:28.621184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:26:28.621184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:26:28.621184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:26:28.621184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:28.621184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:26:28.621184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:28.621184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:26:28.621184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:28.621184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:26:28.621184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:28.621184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:26:28.621184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:28.621651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:26:28.621651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:26:28.621651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:28.621651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:26:28.621651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:28.621651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:26:28.621651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:26:28.621651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:28.621651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:26:28.621651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:28.621651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:26:28.621651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:28.622055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:26:28.622055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:26:28.622055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:26:28.622055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:26:28.622055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:26:28.622055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:26:28.622055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:26:28.622055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:26:28.622055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:26:28.622055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:26:28.622055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:26:28.622055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:26:28.622055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:26:28.622055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:26:28.622055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:26:28.622055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:26:28.622055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:26:28.622055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:26:28.622055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:26:28.622585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:28.622585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:26:28.622585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:28.622585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:26:28.622585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:28.622585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:26:28.622585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:28.622585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:26:28.622585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:28.622585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:26:28.622585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:26:28.622585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:28.622585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:26:28.622585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:28.622585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:26:28.622585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:26:28.622585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:28.622585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:26:28.622585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:28.622585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:26:28.625766 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:26:28.626192 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:26:28.626307 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:26:28.626481 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:26:28.627075 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:26:28.627418 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:26:28.627669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:28.628512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:26:28.628512 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:28.628512 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:26:28.628512 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:26:28.628512 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:28.628512 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:26:28.628512 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:28.628512 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:26:28.629439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:28.629439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:26:28.629439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:26:28.629439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:26:28.629439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:26:28.629439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:26:28.629439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:26:28.629439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:28.629439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:26:28.629439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:28.629439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:26:28.629439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:28.629871 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:26:28.629871 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:28.629871 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:26:28.629871 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:28.629871 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:26:28.629871 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:26:28.629871 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:28.629871 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:26:28.629871 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:28.629871 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:26:28.629871 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:26:28.629871 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:28.629871 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:26:28.630199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:28.630199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:26:28.630562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:28.630562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:26:28.630562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:26:28.630562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:26:28.630562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:26:28.630562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:26:28.630562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:26:28.630562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:26:28.630562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:26:28.630562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:26:28.630562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:26:28.630562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:26:28.630562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:26:28.630562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:26:28.630562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:26:28.630562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:26:28.631943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:26:28.631943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:26:28.631943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:28.631943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:26:28.631943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:28.631943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:26:28.631943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:28.631943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:26:28.631943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:28.631943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:26:28.631943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:28.631943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:26:28.631943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:26:28.631943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:28.631943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:26:28.631943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:28.631943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:26:28.631943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:26:28.631943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:28.632542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:26:28.632542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:28.632542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:26:33.444646 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:26:33.445453 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:26:33.445714 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:26:33.447424 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:26:33.481071 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:26:33.481326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:33.482096 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:26:33.482096 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:33.482096 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:26:33.482096 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:26:33.482096 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:33.482096 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:26:33.482096 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:33.482096 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:26:33.483293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:33.483293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:26:33.483293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:26:33.483293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:26:33.483293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:26:33.483293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:26:33.483293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:26:33.483293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:33.483293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:26:33.483293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:33.483293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:26:33.483293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:33.483762 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:26:33.483762 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:33.483762 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:26:33.483762 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:33.483762 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:26:33.483762 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:26:33.483762 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:33.483762 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:26:33.483762 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:33.483762 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:26:33.483762 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:26:33.483762 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:33.483762 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:26:33.484090 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:33.484090 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:26:33.484990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:33.484990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:26:33.484990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:26:33.484990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:26:33.484990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:26:33.484990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:26:33.484990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:26:33.484990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:26:33.484990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:26:33.484990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:26:33.484990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:26:33.484990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:26:33.484990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:26:33.484990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:26:33.484990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:26:33.484990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:26:33.485345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:26:33.485345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:26:33.485345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:33.485345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:26:33.485345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:33.485345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:26:33.485345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:33.485345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:26:33.485345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:33.485345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:26:33.485345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:33.485345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:26:33.485345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:26:33.485345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:33.485345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:26:33.485345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:33.485345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:26:33.485345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:26:33.485345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:33.485719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:26:33.485719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:33.485719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:26:33.599260 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:26:33.599578 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:26:33.599707 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:26:33.600927 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:26:33.601465 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:26:33.601694 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:33.602536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:26:33.602536 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:33.602536 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:26:33.602536 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:26:33.602536 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:33.602536 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:26:33.602536 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:33.602536 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:26:33.603776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:33.603776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:26:33.603776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:26:33.603776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:26:33.603776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:26:33.603776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:26:33.603776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:26:33.603776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:33.603776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:26:33.603776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:33.603776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:26:33.603776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:33.604355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:26:33.604355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:33.604355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:26:33.604355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:33.604355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:26:33.604355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:26:33.604355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:33.604355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:26:33.604355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:33.604355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:26:33.604355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:26:33.604355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:33.604355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:26:33.604802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:33.604802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:26:33.604974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:33.604974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:26:33.604974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:26:33.604974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:26:33.604974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:26:33.604974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:26:33.604974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:26:33.604974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:26:33.604974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:26:33.604974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:26:33.604974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:26:33.604974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:26:33.604974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:26:33.604974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:26:33.604974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:26:33.604974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:26:33.605432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:26:33.605432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:26:33.605432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:33.605432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:26:33.605432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:33.605432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:26:33.605432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:33.605432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:26:33.605432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:33.605432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:26:33.605432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:33.605432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:26:33.605432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:26:33.605432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:33.605432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:26:33.605432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:33.605432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:26:33.605432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:26:33.605432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:33.605991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:26:33.605991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:33.605991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:26:33.607137 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:26:33.607360 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:26:33.607480 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:26:33.607602 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:26:33.608304 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:26:33.608715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:33.609654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:26:33.609654 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:33.609654 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:26:33.609654 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:26:33.609654 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:33.609654 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:26:33.609654 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:33.609654 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:26:33.610500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:33.610500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:26:33.610500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:26:33.610500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:26:33.610500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:33.610500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:26:33.610500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:33.610500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:26:33.610500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:33.610500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:26:33.610500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:33.610500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:26:33.610500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:33.610500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:26:33.610500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:26:33.610958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:33.610958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:26:33.610958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:33.610958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:26:33.610958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:26:33.610958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:33.610958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:26:33.610958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:33.610958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:26:33.611399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:33.611399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:26:33.611399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:26:33.611399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:26:33.611399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:26:33.611399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:26:33.611399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:26:33.611399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:26:33.611399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:26:33.611399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:26:33.611399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:33.611399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:26:33.611730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:33.611730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:26:33.611730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:33.611730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:26:33.611730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:33.611730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:26:33.611730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:33.611730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:26:33.611730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:26:33.611730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:33.611730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:26:33.611730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:33.611730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:26:33.611730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:26:33.612256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:33.612256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:26:33.612256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:33.612256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:26:33.612532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:33.612532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:26:33.612532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:26:33.612532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:26:33.612532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:26:33.612532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:26:33.612532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:26:33.612532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:26:33.612532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:26:33.612532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:26:33.612532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:26:33.612532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:26:33.612532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:26:33.612532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:26:33.612532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:26:33.612532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:26:33.613112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:26:33.613112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:26:33.613112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:26:33.613112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:26:33.613112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:26:33.613112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:33.613112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:26:33.613112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:33.613112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:26:33.613112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:33.613112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:26:33.613112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:33.613112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:26:33.613112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:33.613112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:26:33.613112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:26:33.613112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:33.613112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:26:33.613802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:33.613802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:26:33.613802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:26:33.613802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:33.613802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:26:33.613802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:33.613802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:26:33.616670 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:26:33.616948 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:26:33.617089 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:26:33.617270 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:26:33.617827 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:26:33.618148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:33.618858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:26:33.618858 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:33.618858 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:26:33.618858 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:26:33.618858 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:33.618858 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:26:33.618858 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:33.618858 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:26:33.619666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:33.619666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:26:33.619666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:26:33.619666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:26:33.619666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:33.619666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:26:33.619666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:33.619666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:26:33.619666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:33.619666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:26:33.619666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:33.619666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:26:33.619666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:33.619666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:26:33.619666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:26:33.620328 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:33.620328 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:26:33.620328 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:33.620328 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:26:33.620328 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:26:33.620328 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:33.620328 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:26:33.620328 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:33.620328 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:26:33.620766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:33.620766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:26:33.620766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:26:33.620766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:26:33.620766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:26:33.620766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:26:33.620766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:26:33.620766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:26:33.620766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:26:33.620766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:33.620766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:26:33.621070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:33.621070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:26:33.621070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:33.621070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:26:33.621070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:33.621070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:26:33.621070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:33.621070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:26:33.621070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:26:33.621070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:33.621070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:26:33.621070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:33.621070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:26:33.621070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:26:33.621350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:33.621350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:26:33.621350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:33.621350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:26:33.622035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:33.622035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:26:33.622035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:26:33.622035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:26:33.622035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:26:33.622035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:26:33.622035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:26:33.622035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:26:33.622035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:26:33.622035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:26:33.622035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:26:33.622035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:26:33.622035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:26:33.622035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:26:33.622035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:26:33.622035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:26:33.622035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:26:33.622404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:26:33.622404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:26:33.622404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:26:33.622404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:33.622404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:26:33.622404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:33.622404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:26:33.622404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:33.622404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:26:33.622404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:33.622404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:26:33.622404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:33.622404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:26:33.622404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:26:33.622404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:33.622404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:26:33.622404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:33.622404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:26:33.622404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:26:33.622825 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:33.622825 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:26:33.622825 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:33.622825 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:26:33.626375 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:26:33.626762 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:26:33.626829 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:26:33.627097 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:26:33.627962 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:26:33.628187 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:26:33.628474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:33.629179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:26:33.629179 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:33.629179 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:26:33.629179 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:26:33.629179 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:33.629179 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:26:33.629179 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:33.629179 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:26:33.630265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:33.630265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:26:33.630265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:26:33.630265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:26:33.630265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:26:33.630265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:26:33.630265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:26:33.630265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:33.630265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:26:33.630265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:33.630265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:26:33.630265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:33.630865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:26:33.630865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:33.630865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:26:33.630865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:33.630865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:26:33.630865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:26:33.630865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:33.630865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:26:33.630865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:33.630865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:26:33.630865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:26:33.630865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:33.630865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:26:33.631421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:33.631421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:26:33.631421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:33.631421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:26:33.631421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:26:33.631421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:26:33.631421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:26:33.631421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:26:33.631421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:26:33.631421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:26:33.631421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:26:33.631421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:26:33.631421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:26:33.631421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:26:33.631421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:26:33.632053 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:26:33.632053 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:26:33.632053 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:26:33.632053 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:26:33.632053 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:26:33.632053 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:33.632053 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:26:33.632053 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:33.632053 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:26:33.632053 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:33.632053 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:26:33.632053 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:33.632053 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:26:33.632053 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:33.632053 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:26:33.632053 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:26:33.632053 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:33.632053 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:26:33.632053 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:33.632855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:26:33.632855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:26:33.632855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:33.632855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:26:33.632855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:33.632855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:26:38.447098 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:26:38.447650 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:26:38.447806 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:26:38.448681 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:26:38.482256 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:26:38.482538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:38.483803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:26:38.483803 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:38.483803 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:26:38.483803 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:26:38.483803 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:38.483803 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:26:38.483803 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:38.483803 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:26:38.484228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:38.484228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:26:38.484228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:26:38.484228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:26:38.484228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:26:38.484228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:26:38.484228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:26:38.484228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:38.484228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:26:38.484228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:38.484228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:26:38.484228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:38.484709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:26:38.484709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:38.484709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:26:38.484709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:38.484709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:26:38.484709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:26:38.484709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:38.484709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:26:38.484709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:38.484709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:26:38.484709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:26:38.484709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:38.484709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:26:38.485162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:38.485162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:26:38.485162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:38.485162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:26:38.485162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:26:38.485162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:26:38.485162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:26:38.485162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:26:38.485162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:26:38.485162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:26:38.485162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:26:38.485162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:26:38.485162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:26:38.485162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:26:38.485162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:26:38.485672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:26:38.485672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:26:38.485672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:26:38.485672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:26:38.485672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:26:38.485672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:38.485672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:26:38.485672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:38.485672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:26:38.485672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:38.485672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:26:38.485672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:38.485672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:26:38.485672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:38.485672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:26:38.485672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:26:38.485672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:38.485672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:26:38.485672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:38.486298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:26:38.486298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:26:38.486298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:38.486298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:26:38.486298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:38.486298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:26:38.599711 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:26:38.600516 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:26:38.600814 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:26:38.602904 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:26:38.603413 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:26:38.603413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:38.604374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:26:38.604374 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:38.604374 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:26:38.604374 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:26:38.604374 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:38.604374 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:26:38.604374 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:38.604374 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:26:38.606419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:38.606419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:26:38.606419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:26:38.606419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:26:38.606419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:26:38.606419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:26:38.606419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:26:38.606419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:38.606419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:26:38.606419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:38.606419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:26:38.606419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:38.607398 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:26:38.607398 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:38.607398 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:26:38.607398 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:38.607398 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:26:38.607398 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:26:38.607398 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:38.607398 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:26:38.607398 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:38.607398 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:26:38.607398 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:26:38.607398 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:38.607398 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:26:38.608186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:38.608186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:26:38.610901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:38.610901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:26:38.610901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:26:38.610901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:26:38.610901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:26:38.610901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:26:38.610901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:26:38.610901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:26:38.610901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:26:38.610901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:26:38.610901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:26:38.610901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:26:38.610901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:26:38.610901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:26:38.610901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:26:38.610901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:26:38.611480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:26:38.611480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:26:38.611480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:38.611480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:26:38.611480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:38.611480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:26:38.611480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:38.611480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:26:38.611480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:38.611480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:26:38.611480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:38.611480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:26:38.611480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:26:38.611480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:38.611480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:26:38.611480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:38.611480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:26:38.611480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:26:38.611480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:38.611999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:26:38.611999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:38.611999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:26:38.611999 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:26:38.612290 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:26:38.612409 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:26:38.612548 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:26:38.613520 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:26:38.613520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:38.614185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:26:38.614185 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:38.614185 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:26:38.614185 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:26:38.614185 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:38.614185 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:26:38.614185 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:38.614185 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:26:38.614845 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:38.614845 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:26:38.614845 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:26:38.614845 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:26:38.614845 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:38.614845 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:26:38.614845 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:38.614845 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:26:38.614845 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:38.614845 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:26:38.614845 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:38.614845 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:26:38.614845 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:38.614845 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:26:38.614845 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:26:38.615367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:38.615367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:26:38.615367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:38.615367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:26:38.615367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:26:38.615367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:38.615367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:26:38.615367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:38.615367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:26:38.615678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:38.615678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:26:38.615678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:26:38.615678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:26:38.615678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:26:38.615678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:26:38.615678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:26:38.615678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:26:38.615678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:26:38.615678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:26:38.615678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:38.615678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:26:38.616068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:38.616068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:26:38.616068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:38.616068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:26:38.616068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:38.616068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:26:38.616068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:38.616068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:26:38.616068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:26:38.616068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:38.616068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:26:38.616068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:38.616068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:26:38.616068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:26:38.616473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:38.616473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:26:38.616473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:38.616473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:26:38.616788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:38.616788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:26:38.616788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:26:38.616788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:26:38.616788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:26:38.616788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:26:38.616788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:26:38.616788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:26:38.616788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:26:38.616788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:26:38.616788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:26:38.616788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:26:38.616788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:26:38.616788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:26:38.616788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:26:38.616788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:26:38.617456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:26:38.617456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:26:38.617456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:26:38.617456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:26:38.617456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:26:38.617456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:38.617456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:26:38.617456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:38.617456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:26:38.617456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:38.617456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:26:38.617456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:38.617456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:26:38.617456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:38.617456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:26:38.617456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:26:38.617456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:38.617456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:26:38.618157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:38.618157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:26:38.618157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:26:38.618157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:38.618157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:26:38.618157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:38.618157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:26:38.618157 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:26:38.618157 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:26:38.618157 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:26:38.618541 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:26:38.618541 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:26:38.618541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:38.619167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:26:38.619167 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:38.619167 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:26:38.619167 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:26:38.619167 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:38.619167 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:26:38.619167 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:38.619167 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:26:38.620059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:38.620059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:26:38.620059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:26:38.620059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:26:38.620059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:38.620059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:26:38.620059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:38.620059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:26:38.620059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:38.620059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:26:38.620059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:38.620059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:26:38.620059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:38.620059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:26:38.620059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:26:38.620716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:38.620716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:26:38.620716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:38.620716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:26:38.620716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:26:38.620716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:38.620716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:26:38.620716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:38.620716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:26:38.621314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:38.621314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:26:38.621314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:26:38.621314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:26:38.621314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:26:38.621314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:26:38.621314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:26:38.621314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:26:38.621314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:26:38.621314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:38.621314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:26:38.621839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:38.621839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:26:38.621839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:38.621839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:26:38.621839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:38.621839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:26:38.621839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:38.621839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:26:38.621839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:26:38.621839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:38.621839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:26:38.621839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:38.621839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:26:38.621839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:26:38.622371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:38.622371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:26:38.622371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:38.622371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:26:38.622810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:38.622810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:26:38.622810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:26:38.622810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:26:38.622810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:26:38.622810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:26:38.622810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:26:38.622810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:26:38.622810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:26:38.622810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:26:38.622810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:26:38.622810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:26:38.622810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:26:38.622810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:26:38.622810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:26:38.622810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:26:38.622810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:26:38.623325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:26:38.623325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:26:38.623325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:26:38.623325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:38.623325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:26:38.623325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:38.623325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:26:38.623325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:38.623325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:26:38.623325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:38.623325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:26:38.623325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:38.623325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:26:38.623325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:26:38.623325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:38.623325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:26:38.623325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:38.623325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:26:38.623325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:26:38.624081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:38.624081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:26:38.624081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:38.624081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:26:38.626724 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:26:38.627032 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:26:38.627186 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:26:38.627384 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:26:38.627776 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:26:38.628205 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:26:38.628488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:38.629274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:26:38.629274 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:38.629274 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:26:38.629274 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:26:38.629274 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:38.629274 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:26:38.629274 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:38.629274 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:26:38.630603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:38.630603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:26:38.630603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:26:38.630603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:26:38.630603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:26:38.630603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:26:38.630603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:26:38.630603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:38.630603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:26:38.630603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:38.630603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:26:38.630603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:38.631284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:26:38.631284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:38.631284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:26:38.631284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:38.631284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:26:38.631284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:26:38.631284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:38.631284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:26:38.631284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:38.631284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:26:38.631284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:26:38.631284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:38.631284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:26:38.631915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:38.631915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:26:38.632026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:38.632026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:26:38.632026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:26:38.632026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:26:38.632026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:26:38.632026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:26:38.632026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:26:38.632026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:26:38.632026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:26:38.632026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:26:38.632026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:26:38.632026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:26:38.632026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:26:38.632026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:26:38.632026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:26:38.632026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:26:38.632817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:26:38.632817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:26:38.632817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:38.632817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:26:38.632817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:38.632817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:26:38.632817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:38.632817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:26:38.632817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:38.632817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:26:38.632817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:38.632817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:26:38.632817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:26:38.632817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:38.632817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:26:38.632817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:38.632817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:26:38.632817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:26:38.632817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:38.633705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:26:38.633705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:38.633705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:26:43.448024 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:26:43.448583 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:26:43.448631 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:26:43.449517 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:26:43.498101 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:26:43.498101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:43.498933 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:26:43.498933 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:43.498933 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:26:43.498933 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:26:43.498933 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:43.498933 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:26:43.498933 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:43.498933 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:26:43.500080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.050 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:43.500080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:26:43.500080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:26:43.500080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:26:43.500080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:26:43.500080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:26:43.500080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:26:43.500080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:43.500080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:26:43.500080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:43.500080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:26:43.500080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:43.501108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:26:43.501108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:43.501108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:26:43.501108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:43.501108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:26:43.501108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:26:43.501108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:43.501108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:26:43.501108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:43.501108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:26:43.501108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:26:43.501108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:43.501108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:26:43.502194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:43.502194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:26:43.502194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:43.502194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:26:43.502194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:26:43.502194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:26:43.502194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:26:43.502194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:26:43.502194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:26:43.502194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:26:43.502194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:26:43.502194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:26:43.502194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:26:43.502194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:26:43.502194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:26:43.503350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:26:43.503350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:26:43.503350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:26:43.503350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:26:43.503350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:26:43.503350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:43.503350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:26:43.503350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:43.503350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:26:43.503350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:43.503350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:26:43.503350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:43.503350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:26:43.503350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:43.503350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:26:43.503350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:26:43.503350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:43.503350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:26:43.503350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:43.504775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:26:43.504775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:26:43.504775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:43.504775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:26:43.504775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:43.504775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:26:43.601026 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:26:43.601623 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:26:43.601932 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:26:43.603149 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:26:43.603420 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:26:43.603564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:43.604343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:26:43.604343 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:43.604343 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:26:43.604343 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:26:43.604343 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:43.604343 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:26:43.604343 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:43.604343 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:26:43.605876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:43.605876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:26:43.605876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:26:43.605876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:26:43.605876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:26:43.605876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:26:43.605876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:26:43.605876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:43.605876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:26:43.605876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:43.605876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:26:43.605876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:43.606490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:26:43.606490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:43.606490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:26:43.606490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:43.606490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:26:43.606490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:26:43.606490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:43.606490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:26:43.606490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:43.606490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:26:43.606490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:26:43.606490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:43.606490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:26:43.607114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:43.607114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:26:43.607464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:43.607464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:26:43.607464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:26:43.607464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:26:43.607464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:26:43.607464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:26:43.607464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:26:43.607464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:26:43.607464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:26:43.607464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:26:43.607464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:26:43.607464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:26:43.607464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:26:43.607464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:26:43.607464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:26:43.607464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:26:43.608250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:26:43.608250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:26:43.608250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:43.608250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:26:43.608250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:43.608250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:26:43.608250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:43.608250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:26:43.608250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:43.608250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:26:43.608250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:43.608250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:26:43.608250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:26:43.608250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:43.608250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:26:43.608250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:43.608250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:26:43.608250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:26:43.608250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:43.608790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:26:43.608790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:43.608790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:26:43.612370 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:26:43.612635 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:26:43.612760 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:26:43.612907 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:26:43.613303 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:26:43.613521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:43.614164 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:26:43.614164 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:43.614164 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:26:43.614164 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:26:43.614164 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:43.614164 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:26:43.614164 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:43.614164 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:26:43.614949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:43.614949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:26:43.614949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:26:43.614949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:26:43.614949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:43.614949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:26:43.614949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:43.614949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:26:43.614949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:43.614949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:26:43.614949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:43.614949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:26:43.614949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:43.614949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:26:43.614949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:26:43.615507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:43.615507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:26:43.615507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:43.615507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:26:43.615507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:26:43.615507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:43.615507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:26:43.615507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:43.615507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:26:43.615968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:43.615968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:26:43.615968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:26:43.615968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:26:43.615968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:26:43.615968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:26:43.615968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:26:43.615968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:26:43.615968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:26:43.615968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:26:43.615968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:43.615968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:26:43.616393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:43.616393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:26:43.616393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:43.616393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:26:43.616393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:43.616393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:26:43.616393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:43.616393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:26:43.616393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:26:43.616393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:43.616393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:26:43.616393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:43.616393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:26:43.616393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:26:43.616853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:43.616853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:26:43.616853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:43.616853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:26:43.617038 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:43.617038 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:26:43.617038 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:26:43.617038 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:26:43.617038 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:26:43.617038 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:26:43.617038 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:26:43.617038 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:26:43.617038 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:26:43.617038 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:26:43.617038 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:26:43.617038 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:26:43.617038 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:26:43.617038 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:26:43.617038 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:26:43.617038 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:26:43.617588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:26:43.617588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:26:43.617588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:26:43.617588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:26:43.617588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:26:43.617588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:43.617588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:26:43.617588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:43.617588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:26:43.617588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:43.617588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:26:43.617588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:43.617588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:26:43.617588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:43.617588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:26:43.617588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:26:43.617588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:43.617588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:26:43.618253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:43.618253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:26:43.618253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:26:43.618253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:43.618253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:26:43.618253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:43.618253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:26:43.618253 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:26:43.618253 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:26:43.618253 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:26:43.618562 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:26:43.619426 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:26:43.619688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:43.620337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:26:43.620337 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:43.620337 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:26:43.620337 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:26:43.620337 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:43.620337 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:26:43.620337 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:43.620337 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:26:43.621198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:43.621198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:26:43.621198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:26:43.621198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:26:43.621198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:43.621198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:26:43.621198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:43.621198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:26:43.621198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:43.621198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:26:43.621198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:43.621198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:26:43.621198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:43.621198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:26:43.621198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:26:43.621643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:43.621643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:26:43.621643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:43.621643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:26:43.621643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:26:43.621643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:43.621643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:26:43.621643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:43.621643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:26:43.622277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:43.622277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:26:43.622277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:26:43.622277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:26:43.622277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:26:43.622277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:26:43.622277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:26:43.622277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:26:43.622277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:26:43.622277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:43.622277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:26:43.622761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:43.622761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:26:43.622761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:43.622761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:26:43.622761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:43.622761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:26:43.622761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:43.622761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:26:43.622761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:26:43.622761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:43.622761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:26:43.622761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:43.622761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:26:43.622761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:26:43.623273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:43.623273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:26:43.623273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:43.623273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:26:43.623731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:43.623731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:26:43.623731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:26:43.623731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:26:43.623731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:26:43.623731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:26:43.623731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:26:43.623731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:26:43.623731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:26:43.623731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:26:43.623731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:26:43.623731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:26:43.623731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:26:43.623731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:26:43.623731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:26:43.623731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:26:43.623731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:26:43.624465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:26:43.624465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:26:43.624465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:26:43.624465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:43.624465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:26:43.624465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:43.624465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:26:43.624465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:43.624465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:26:43.624465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:43.624465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:26:43.624465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:43.624465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:26:43.624465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:26:43.624465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:43.624465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:26:43.624465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:43.624465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:26:43.624465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:26:43.625192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:43.625192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:26:43.625192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:43.625192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:26:43.627121 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:26:43.627426 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:26:43.627550 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:26:43.627716 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:26:43.628280 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:26:43.628548 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:26:43.628783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:43.629913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:26:43.629913 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:43.629913 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:26:43.629913 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:26:43.629913 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:43.629913 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:26:43.629913 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:43.629913 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:26:43.630850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:43.630850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:26:43.630850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:26:43.630850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:26:43.630850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:26:43.630850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:26:43.630850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:26:43.630850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:43.630850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:26:43.630850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:43.630850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:26:43.630850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:43.631476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:26:43.631476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:43.631476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:26:43.631476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:43.631476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:26:43.631476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:26:43.631476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:43.631476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:26:43.631476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:43.631476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:26:43.631476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:26:43.631476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:43.631476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:26:43.632019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:43.632019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:26:43.632019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:43.632019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:26:43.632019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:26:43.632019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:26:43.632019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:26:43.632019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:26:43.632019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:26:43.632019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:26:43.632019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:26:43.632019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:26:43.632019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:26:43.632019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:26:43.632019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:26:43.632513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:26:43.632513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:26:43.632513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:26:43.632513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:26:43.632513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:26:43.632513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:43.632513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:26:43.632513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:43.632513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:26:43.632513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:43.632513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:26:43.632513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:43.632513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:26:43.632513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:43.632513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:26:43.632513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:26:43.632513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:43.632513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:26:43.632513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:43.633039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:26:43.633039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:26:43.633039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:43.633039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:26:43.633039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:43.633039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:26:48.448568 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:26:48.449334 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:26:48.449334 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:26:48.450103 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:26:48.482323 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:26:48.482638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:48.483591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:26:48.483591 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:48.483591 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:26:48.483591 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:26:48.483591 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:48.483591 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:26:48.483591 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:48.483591 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:26:48.484726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:48.484726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:26:48.484726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:26:48.484726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:26:48.484726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:26:48.484726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:26:48.484726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:26:48.484726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:48.484726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:26:48.484726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:48.484726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:26:48.484726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:48.485647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:26:48.485647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:48.485647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:26:48.485647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:48.485647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:26:48.485647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:26:48.485647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:48.485647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:26:48.485647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:48.485647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:26:48.485647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:26:48.485647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:48.485647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:26:48.486627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:48.486627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:26:48.486627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:48.486627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:26:48.486627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:26:48.486627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:26:48.486627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:26:48.486627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:26:48.486627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:26:48.486627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:26:48.486627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:26:48.486627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:26:48.486627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:26:48.486627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:26:48.486627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:26:48.487721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:26:48.487721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:26:48.487721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:26:48.487721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:26:48.487721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:26:48.487721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:48.487721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:26:48.487721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:48.487721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:26:48.487721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:48.487721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:26:48.487721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:48.487721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:26:48.487721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:48.487721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:26:48.487721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:26:48.487721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:48.487721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:26:48.487721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:48.488863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:26:48.488863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:26:48.488863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:48.488863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:26:48.488863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:48.488863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:26:48.602415 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:26:48.602999 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:26:48.603099 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:26:48.604377 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:26:48.604808 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:26:48.605088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:48.606130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:26:48.606130 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:48.606130 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:26:48.606130 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:26:48.606130 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:48.606130 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:26:48.606130 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:48.606130 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:26:48.608030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:48.608030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:26:48.608030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:26:48.608030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:26:48.608030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:26:48.608030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:26:48.608030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:26:48.608030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:48.608030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:26:48.608030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:48.608030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:26:48.608030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:48.608931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:26:48.608931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:48.608931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:26:48.608931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:48.608931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:26:48.608931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:26:48.608931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:48.608931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:26:48.608931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:48.608931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:26:48.608931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:26:48.608931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:48.608931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:26:48.609817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:48.609817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:26:48.609817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:48.609817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:26:48.609817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:26:48.609817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:26:48.609817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:26:48.609817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:26:48.609817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:26:48.609817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:26:48.609817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:26:48.609817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:26:48.609817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:26:48.609817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:26:48.609817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:26:48.610733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:26:48.610733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:26:48.610733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:26:48.610733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:26:48.610733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:26:48.610733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:48.610733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:26:48.610733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:48.610733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:26:48.610733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:48.610733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:26:48.610733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:48.610733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:26:48.610733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:48.610733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:26:48.610733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:26:48.610733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:48.610733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:26:48.610733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:48.611623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:26:48.611623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:26:48.611623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:48.611623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:26:48.611623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:48.611623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:26:48.612886 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:26:48.613360 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:26:48.613566 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:26:48.613804 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:26:48.614443 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:26:48.614751 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:48.615605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:26:48.615605 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:48.615605 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:26:48.615605 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:26:48.615605 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:48.615605 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:26:48.615605 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:48.615605 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:26:48.616975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:48.616975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:26:48.616975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:26:48.616975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:26:48.616975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:48.616975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:26:48.616975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:48.616975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:26:48.616975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:48.616975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:26:48.616975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:48.616975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:26:48.616975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:48.616975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:26:48.616975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:26:48.617853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:48.617853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:26:48.617853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:48.617853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:26:48.617853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:26:48.617853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:48.617853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:26:48.617853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:48.617853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:26:48.618611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:48.618611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:26:48.618611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:26:48.618611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:26:48.618611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:26:48.618611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:26:48.618611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:26:48.618611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:26:48.618611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:26:48.618611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:26:48.618611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:48.618611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:26:48.619193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:48.619193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:26:48.619193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:48.619193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:26:48.619193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:48.619193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:26:48.619193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:48.619193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:26:48.619193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:26:48.619193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:48.619193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:26:48.619193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:48.619193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:26:48.619193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:26:48.619725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:48.619725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:26:48.619725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:48.619725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:26:48.620154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:48.620154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:26:48.620154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:26:48.620154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:26:48.620154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:26:48.620154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:26:48.620154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:26:48.620154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:26:48.620154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:26:48.620154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:26:48.620154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:26:48.620154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:26:48.620154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:26:48.620154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:26:48.620154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:26:48.620154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:26:48.620858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:26:48.620858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:26:48.620858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:26:48.620858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:26:48.620858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:26:48.620858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:48.620858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:26:48.620858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:48.620858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:26:48.620858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:48.620858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:26:48.620858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:48.620858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:26:48.620858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:48.620858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:26:48.620858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:26:48.620858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:48.620858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:26:48.621582 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:48.621582 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:26:48.621582 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:26:48.621582 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:48.621582 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:26:48.621582 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:48.621582 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:26:48.621582 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:26:48.621582 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:26:48.621582 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:26:48.622044 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:26:48.622044 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:26:48.622044 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:48.622474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:26:48.622474 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:48.622474 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:26:48.622474 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:26:48.622474 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:48.622474 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:26:48.622474 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:48.622474 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:26:48.623329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:48.623329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:26:48.623329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:26:48.623329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:26:48.623329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:48.623329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:26:48.623329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:48.623329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:26:48.623329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:48.623329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:26:48.623329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:48.623329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:26:48.623329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:48.623329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:26:48.623329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:26:48.623956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:48.623956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:26:48.623956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:48.623956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:26:48.623956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:26:48.623956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:48.623956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:26:48.623956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:48.623956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:26:48.624326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:48.624326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:26:48.624326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:26:48.624326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:26:48.624326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:26:48.624326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:26:48.624326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:26:48.624326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:26:48.624326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:26:48.624326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:48.624326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:26:48.624834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:48.624834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:26:48.624834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:48.624834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:26:48.624834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:48.624834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:26:48.624834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:48.624834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:26:48.624834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:26:48.624834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:48.624834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:26:48.624834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:48.624834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:26:48.624834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:26:48.625366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:48.625366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:26:48.625366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:48.625366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:26:48.625554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:48.625554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:26:48.625554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:26:48.625554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:26:48.625554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:26:48.625554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:26:48.625554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:26:48.625554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:26:48.625554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:26:48.625554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:26:48.625554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:26:48.625554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:26:48.625554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:26:48.625554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:26:48.625554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:26:48.625554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:26:48.625554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:26:48.626215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:26:48.626215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:26:48.626215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:26:48.626215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:48.626215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:26:48.626215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:48.626215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:26:48.626215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:48.626215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:26:48.626215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:48.626215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:26:48.626215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:48.626215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:26:48.626215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:26:48.626215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:48.626215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:26:48.626215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:48.626215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:26:48.626215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:26:48.627008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:48.627008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:26:48.627008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:48.627008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:26:48.627732 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:26:48.628115 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:26:48.628229 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:26:48.628415 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:26:48.628759 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:26:48.629010 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:26:48.629174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:48.629806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:26:48.629806 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:48.629806 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:26:48.629806 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:26:48.629806 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:48.629806 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:26:48.629806 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:48.629806 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:26:48.631315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:48.631315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:26:48.631315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:26:48.631315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:26:48.631315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:26:48.631315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:26:48.631315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:26:48.631315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:48.631315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:26:48.631315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:48.631315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:26:48.631315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:48.631655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:26:48.631655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:48.631655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:26:48.631655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:48.631655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:26:48.631655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:26:48.631655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:48.631655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:26:48.631655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:48.631655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:26:48.631655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:26:48.631655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:48.631655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:26:48.632018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:48.632018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:26:48.632334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:48.632334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:26:48.632334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:26:48.632334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:26:48.632334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:26:48.632334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:26:48.632334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:26:48.632334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:26:48.632334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:26:48.632334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:26:48.632334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:26:48.632334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:26:48.632334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:26:48.632334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:26:48.632334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:26:48.632334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:26:48.632815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:26:48.632815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:26:48.632815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:48.632815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:26:48.632815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:48.632815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:26:48.632815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:48.632815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:26:48.632815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:48.632815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:26:48.632815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:48.632815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:26:48.632815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:26:48.632815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:48.632815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:26:48.632815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:48.632815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:26:48.632815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:26:48.632815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:48.633334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:26:48.633334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:48.633334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:26:53.450059 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:26:53.450059 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:26:53.450059 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:26:53.452004 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:26:53.483227 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:26:53.483502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:53.486432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:26:53.486432 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:53.486432 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:26:53.486432 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:26:53.486432 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:53.486432 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:26:53.486432 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:53.486432 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:26:53.487401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:53.487401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:26:53.487401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:26:53.487401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:26:53.487401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:26:53.487401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:26:53.487401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:26:53.487401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:53.487401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:26:53.487401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:53.487401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:26:53.487401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:53.488117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:26:53.488117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:53.488117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:26:53.488117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:53.488117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:26:53.488117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:26:53.488117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:53.488117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:26:53.488117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:53.488117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:26:53.488117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:26:53.488117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:53.488117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:26:53.488832 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:53.488832 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:26:53.488832 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:53.488832 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:26:53.488832 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:26:53.488832 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:26:53.488832 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:26:53.488832 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:26:53.488832 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:26:53.488832 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:26:53.488832 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:26:53.488832 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:26:53.488832 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:26:53.488832 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:26:53.488832 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:26:53.489622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:26:53.489622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:26:53.489622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:26:53.489622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:26:53.489622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:26:53.489622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:53.489622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:26:53.489622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:53.489622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:26:53.489622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:53.489622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:26:53.489622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:53.489622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:26:53.489622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:53.489622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:26:53.489622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:26:53.489622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:53.489622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:26:53.489622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:53.490586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:26:53.490586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:26:53.490586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:53.490586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:26:53.490586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:53.490586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:26:53.603101 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:26:53.603627 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:26:53.603784 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:26:53.604710 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:26:53.604982 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:26:53.605153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:53.605996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:26:53.605996 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:53.605996 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:26:53.605996 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:26:53.605996 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:53.605996 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:26:53.605996 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:53.605996 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:26:53.608167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:53.608167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:26:53.608167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:26:53.608167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:26:53.608167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:26:53.608167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:26:53.608167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:26:53.608167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:53.608167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:26:53.608167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:53.608167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:26:53.608167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:53.608675 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:26:53.608675 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:53.608675 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:26:53.608675 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:53.608675 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:26:53.608675 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:26:53.608675 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:53.608675 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:26:53.608675 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:53.608675 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:26:53.608675 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:26:53.608675 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:53.608675 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:26:53.609121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:53.609121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:26:53.609514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:53.609514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:26:53.609514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:26:53.609514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:26:53.609514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:26:53.609514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:26:53.609514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:26:53.609514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:26:53.609514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:26:53.609514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:26:53.609514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:26:53.609514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:26:53.609514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:26:53.609514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:26:53.609514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:26:53.609514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:26:53.610006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:26:53.610006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:26:53.610006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:53.610006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:26:53.610006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:53.610006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:26:53.610006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:53.610006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:26:53.610006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:53.610006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:26:53.610006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:53.610006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:26:53.610006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:26:53.610006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:53.610006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:26:53.610006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:53.610006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:26:53.610006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:26:53.610006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:53.610573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:26:53.610573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:53.610573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:26:53.613931 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:26:53.613931 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:26:53.614076 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:26:53.614224 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:26:53.614766 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:26:53.615112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:53.615957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:26:53.615957 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:53.615957 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:26:53.615957 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:26:53.615957 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:53.615957 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:26:53.615957 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:53.615957 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:26:53.616776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:53.616776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:26:53.616776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:26:53.616776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:26:53.616776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:53.616776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:26:53.616776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:53.616776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:26:53.616776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:53.616776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:26:53.616776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:53.616776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:26:53.616776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:53.616776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:26:53.616776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:26:53.617423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:53.617423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:26:53.617423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:53.617423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:26:53.617423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:26:53.617423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:53.617423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:26:53.617423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:53.617423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:26:53.617734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:53.617734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:26:53.617734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:26:53.617734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:26:53.617734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:26:53.617734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:26:53.617734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:26:53.617734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:26:53.617734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:26:53.617734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:26:53.617734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:53.617734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:26:53.618222 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:53.618222 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:26:53.618222 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:53.618222 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:26:53.618222 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:53.618222 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:26:53.618222 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:53.618222 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:26:53.618222 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:26:53.618222 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:53.618222 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:26:53.618222 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:53.618222 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:26:53.618222 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:26:53.618625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:53.618625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:26:53.618625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:53.618625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:26:53.618625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:53.618625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:26:53.618625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:26:53.618625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:26:53.618625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:26:53.618625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:26:53.618625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:26:53.618625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:26:53.618625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:26:53.618625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:26:53.619091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:26:53.619091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:26:53.619091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:26:53.619091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:26:53.619091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:26:53.619091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:26:53.619091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:26:53.619091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:26:53.619091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:26:53.619091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:26:53.619091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:26:53.619091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:53.619091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:26:53.619091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:53.619091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:26:53.619091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:53.619091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:26:53.619091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:53.619091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:26:53.619656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:53.619656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:26:53.619656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:26:53.619656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:53.619656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:26:53.619656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:53.619656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:26:53.619656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:26:53.619656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:53.619656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:26:53.619656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:53.619656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:26:53.623223 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:26:53.623223 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:26:53.623223 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:26:53.623223 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:26:53.623223 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:26:53.623223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:53.623458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:26:53.623458 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:53.623458 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:26:53.623458 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:26:53.623458 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:53.623458 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:26:53.623458 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:53.623458 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:26:53.624117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:53.624117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:26:53.624117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:26:53.624117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:26:53.624117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:53.624117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:26:53.624117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:53.624117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:26:53.624117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:53.624117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:26:53.624117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:53.624117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:26:53.624117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:53.624117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:26:53.624117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:26:53.624549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:53.624549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:26:53.624549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:53.624549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:26:53.624549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:26:53.624549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:53.624549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:26:53.624549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:53.624549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:26:53.625010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:53.625010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:26:53.625010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:26:53.625010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:26:53.625010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:26:53.625010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:26:53.625010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:26:53.625010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:26:53.625010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:26:53.625010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:53.625010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:26:53.625787 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:53.625787 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:26:53.625787 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:53.625787 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:26:53.625787 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:53.625787 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:26:53.625787 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:53.625787 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:26:53.625787 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:26:53.625787 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:53.625787 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:26:53.625787 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:53.625787 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:26:53.625787 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:26:53.626043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:53.626043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:26:53.626043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:53.626043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:26:53.626325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:53.626325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:26:53.626325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:26:53.626325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:26:53.626325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:26:53.626325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:26:53.626325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:26:53.626325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:26:53.626325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:26:53.626325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:26:53.626325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:26:53.626325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:26:53.626325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:26:53.626325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:26:53.626325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:26:53.626325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:26:53.626325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:26:53.626658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:26:53.626658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:26:53.626658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:26:53.626658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:53.626658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:26:53.626658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:53.626658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:26:53.626658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:53.626658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:26:53.626658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:53.626658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:26:53.626658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:53.626658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:26:53.626658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:26:53.626658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:53.626658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:26:53.626658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:53.626658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:26:53.626658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:26:53.627017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:53.627017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:26:53.627017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:53.627017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:26:53.628359 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:26:53.628972 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:26:53.628972 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:26:53.628972 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:26:53.629412 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:26:53.629961 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:26:53.630273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:53.631595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:26:53.631595 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:53.631595 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:26:53.631595 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:26:53.631595 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:53.631595 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:26:53.631595 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:53.631595 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:26:53.632035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:53.632035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:26:53.632035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:26:53.632035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:26:53.632035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:26:53.632035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:26:53.632035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:26:53.632035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:53.632035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:26:53.632035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:53.632035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:26:53.632035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:53.632286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:26:53.632286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:53.632286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:26:53.632286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:53.632286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:26:53.632286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:26:53.632286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:53.632286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:26:53.632286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:53.632286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:26:53.632286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:26:53.632286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:53.632286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:26:53.632593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:53.632593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:26:53.633061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:53.633061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:26:53.633061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:26:53.633061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:26:53.633061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:26:53.633061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:26:53.633061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:26:53.633061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:26:53.633061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:26:53.633061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:26:53.633061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:26:53.633061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:26:53.633061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:26:53.633061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:26:53.633061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:26:53.633061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:26:53.633546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:26:53.633546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:26:53.633546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:53.633546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:26:53.633546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:53.633546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:26:53.633546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:53.633546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:26:53.633546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:53.633546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:26:53.633546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:53.633546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:26:53.633546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:26:53.633546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:53.633546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:26:53.633546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:53.633546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:26:53.633546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:26:53.633546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:53.634095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:26:53.634095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:53.634095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:26:58.449907 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:26:58.450666 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:26:58.450666 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:26:58.451393 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:26:58.453021 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG dbcounter [-] [106314] Writing DB stats neutron:SELECT=240 {{(pid=106314) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 09 09:26:58.490768 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:26:58.491238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:58.492372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:26:58.492372 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:58.492372 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:26:58.492372 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:26:58.492372 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:58.492372 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:26:58.492372 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:58.492372 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:26:58.493388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:58.493388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:26:58.493388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:26:58.493388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:26:58.493388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:26:58.493388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:26:58.493388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:26:58.493388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:58.493388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:26:58.493388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:58.493388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:26:58.493388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:58.494092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:26:58.494092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:58.494092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:26:58.494092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:58.494092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:26:58.494092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:26:58.494092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:58.494092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:26:58.494092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:58.494092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:26:58.494092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:26:58.494092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:58.494092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:26:58.494730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:58.494730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:26:58.494730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:58.494730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:26:58.494730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:26:58.494730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:26:58.494730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:26:58.494730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:26:58.494730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:26:58.494730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:26:58.494730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:26:58.494730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:26:58.494730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:26:58.494730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:26:58.494730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:26:58.495533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:26:58.495533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:26:58.495533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:26:58.495533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:26:58.495533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:26:58.495533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:58.495533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:26:58.495533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:58.495533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:26:58.495533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:58.495533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:26:58.495533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:58.495533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:26:58.495533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:58.495533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:26:58.495533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:26:58.495533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:58.495533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:26:58.495533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:58.496479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:26:58.496479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:26:58.496479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:58.496479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:26:58.496479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:58.496479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:26:58.603929 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:26:58.604581 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:26:58.604762 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:26:58.605831 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:26:58.606116 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:26:58.606354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:58.607192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:26:58.607192 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:58.607192 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:26:58.607192 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:26:58.607192 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:58.607192 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:26:58.607192 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:58.607192 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:26:58.608628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:58.608628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:26:58.608628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:26:58.608628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:26:58.608628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:26:58.608628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:26:58.608628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:26:58.608628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:58.608628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:26:58.608628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:58.608628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:26:58.608628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:58.609573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:26:58.609573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:58.609573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:26:58.609573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:58.609573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:26:58.609573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:26:58.609573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:58.609573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:26:58.609573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:58.609573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:26:58.609573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:26:58.609573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:58.609573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:26:58.610148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:58.610148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:26:58.610429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:58.610429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:26:58.610429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:26:58.610429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:26:58.610429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:26:58.610429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:26:58.610429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:26:58.610429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:26:58.610429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:26:58.610429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:26:58.610429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:26:58.610429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:26:58.610429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:26:58.610429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:26:58.610429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:26:58.610429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:26:58.611363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:26:58.611363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:26:58.611363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:58.611363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:26:58.611363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:58.611363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:26:58.611363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:58.611363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:26:58.611363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:58.611363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:26:58.611363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:58.611363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:26:58.611363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:26:58.611363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:58.611363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:26:58.611363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:58.611363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:26:58.611363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:26:58.611363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:58.612331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:26:58.612331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:58.612331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:26:58.613844 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:26:58.614148 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:26:58.614343 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:26:58.614343 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:26:58.614934 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:26:58.615246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:58.615987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:26:58.615987 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:58.615987 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:26:58.615987 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:26:58.615987 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:58.615987 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:26:58.615987 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:58.615987 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:26:58.616711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:58.616711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:26:58.616711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:26:58.616711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:26:58.616711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:58.616711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:26:58.616711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:58.616711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:26:58.616711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:58.616711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:26:58.616711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:58.616711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:26:58.616711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:58.616711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:26:58.616711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:26:58.617321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:58.617321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:26:58.617321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:58.617321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:26:58.617321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:26:58.617321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:58.617321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:26:58.617321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:58.617321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:26:58.617714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:58.617714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:26:58.617714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:26:58.617714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:26:58.617714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:26:58.617714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:26:58.617714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:26:58.617714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:26:58.617714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:26:58.617714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:26:58.617714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:58.617714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:26:58.618112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:58.618112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:26:58.618112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:58.618112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:26:58.618112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:58.618112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:26:58.618112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:58.618112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:26:58.618112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:26:58.618112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:58.618112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:26:58.618112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:58.618112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:26:58.618112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:26:58.618468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:58.618468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:26:58.618468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:58.618468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:26:58.618635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:58.618635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:26:58.618635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:26:58.618635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:26:58.618635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:26:58.618635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:26:58.618635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:26:58.618635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:26:58.618635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:26:58.618635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:26:58.618635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:26:58.618635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:26:58.618635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:26:58.618635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:26:58.618635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:26:58.618635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:26:58.619273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:26:58.619273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:26:58.619273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:26:58.619273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:26:58.619273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:26:58.619273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:58.619273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:26:58.619273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:58.619273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:26:58.619273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:58.619273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:26:58.619273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:58.619273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:26:58.619273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:58.619273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:26:58.619273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:26:58.619273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:58.619273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:26:58.619797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:58.619797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:26:58.619797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:26:58.619797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:58.619797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:26:58.619797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:58.619797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:26:58.621553 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:26:58.621854 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:26:58.621988 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:26:58.622162 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:26:58.622906 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:26:58.623140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:58.623602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:26:58.623602 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:58.623602 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:26:58.623602 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:26:58.623602 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:58.623602 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:26:58.623602 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:58.623602 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:26:58.624311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:58.624311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:26:58.624311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:26:58.624311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:26:58.624311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:58.624311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:26:58.624311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:58.624311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:26:58.624311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:58.624311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:26:58.624311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:58.624311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:26:58.624311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:58.624311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:26:58.624311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:26:58.624732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:58.624732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:26:58.624732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:58.624732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:26:58.624732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:26:58.624732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:58.624732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:26:58.624732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:58.624732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:26:58.625202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:58.625202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:26:58.625202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:26:58.625202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:26:58.625202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:26:58.625202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:26:58.625202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:26:58.625202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:26:58.625202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:26:58.625202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:58.625202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:26:58.625541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:58.625541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:26:58.625541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:58.625541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:26:58.625541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:58.625541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:26:58.625541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:58.625541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:26:58.625541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:26:58.625541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:58.625541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:26:58.625541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:58.625541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:26:58.625541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:26:58.625926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:58.625926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:26:58.625926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:58.625926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:26:58.626242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:58.626242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:26:58.626242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:26:58.626242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:26:58.626242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:26:58.626242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:26:58.626242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:26:58.626242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:26:58.626242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:26:58.626242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:26:58.626242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:26:58.626242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:26:58.626242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:26:58.626242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:26:58.626242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:26:58.626242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:26:58.626242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:26:58.626826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:26:58.626826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:26:58.626826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:26:58.626826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:58.626826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:26:58.626826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:58.626826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:26:58.626826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:58.626826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:26:58.626826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:58.626826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:26:58.626826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:58.626826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:26:58.626826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:26:58.626826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:58.626826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:26:58.626826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:58.626826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:26:58.626826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:26:58.627388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:58.627388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:26:58.627388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:58.627388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:26:58.628758 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:26:58.629110 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:26:58.629223 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:26:58.629368 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:26:58.629674 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:26:58.630092 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:26:58.630395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:58.631129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:26:58.631129 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:58.631129 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:26:58.631129 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:26:58.631129 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:58.631129 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:26:58.631129 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:58.631129 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:26:58.631945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:58.631945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:26:58.631945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:26:58.631945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:26:58.631945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:26:58.631945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:26:58.631945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:26:58.631945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:58.631945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:26:58.631945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:58.631945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:26:58.631945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:58.632383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:26:58.632383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:58.632383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:26:58.632383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:58.632383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:26:58.632383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:26:58.632383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:58.632383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:26:58.632383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:58.632383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:26:58.632383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:26:58.632383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:58.632383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:26:58.632725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:58.632725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:26:58.633054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:58.633054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:26:58.633054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:26:58.633054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:26:58.633054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:26:58.633054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:26:58.633054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:26:58.633054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:26:58.633054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:26:58.633054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:26:58.633054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:26:58.633054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:26:58.633054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:26:58.633054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:26:58.633054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:26:58.633054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:26:58.633840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:26:58.633840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:26:58.633840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:58.633840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:26:58.633840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:26:58.633840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:26:58.633840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:26:58.633840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:26:58.633840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:26:58.633840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:26:58.633840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:26:58.633840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:26:58.633840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:26:58.633840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:26:58.633840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:26:58.633840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:26:58.633840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:26:58.633840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:26:58.633840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:26:58.634418 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:26:58.634418 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:26:58.634418 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:27:03.451105 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:27:03.452663 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:27:03.452829 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:27:03.453478 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:27:03.483084 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:27:03.483329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:03.484399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:27:03.484399 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:03.484399 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:27:03.484399 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:27:03.484399 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:03.484399 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:27:03.484399 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:03.484399 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:27:03.486063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.032 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:03.486063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:27:03.486063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:27:03.486063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:27:03.486063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:27:03.486063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:27:03.486063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:27:03.486063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:03.486063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:27:03.486063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:03.486063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:27:03.486063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:03.486642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:27:03.486642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:03.486642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:27:03.486642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:03.486642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:27:03.486642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:27:03.486642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:03.486642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:27:03.486642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:03.486642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:27:03.486642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:27:03.486642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:03.486642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:27:03.487462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:03.487462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:27:03.488085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:03.488085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:27:03.488085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:27:03.488085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:27:03.488085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:27:03.488085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:27:03.488085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:27:03.488085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:27:03.488085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:27:03.488085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:27:03.488085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:27:03.488085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:27:03.488085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:27:03.488085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:27:03.488085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:27:03.488085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:27:03.490697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:27:03.490697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:27:03.490697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:03.490697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:27:03.490697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:03.490697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:27:03.490697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:03.490697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:27:03.490697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:03.490697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:27:03.490697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:03.490697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:27:03.490697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:27:03.490697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:03.490697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:27:03.490697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:03.490697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:27:03.490697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:27:03.490697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:03.491721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:27:03.491721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:03.491721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:27:03.604583 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:27:03.605382 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:27:03.605382 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:27:03.607137 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:27:03.607613 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:27:03.608014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:03.608982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:27:03.608982 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:03.608982 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:27:03.608982 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:27:03.608982 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:03.608982 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:27:03.608982 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:03.608982 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:27:03.610671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:03.610671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:27:03.610671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:27:03.610671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:27:03.610671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:27:03.610671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:27:03.610671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:27:03.610671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:03.610671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:27:03.610671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:03.610671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:27:03.610671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:03.611484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:27:03.611484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:03.611484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:27:03.611484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:03.611484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:27:03.611484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:27:03.611484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:03.611484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:27:03.611484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:03.611484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:27:03.611484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:27:03.611484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:03.611484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:27:03.612249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:03.612249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:27:03.612404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:03.612404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:27:03.612404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:27:03.612404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:27:03.612404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:27:03.612404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:27:03.612404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:27:03.612404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:27:03.612404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:27:03.612404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:27:03.612404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:27:03.612404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:27:03.612404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:27:03.612404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:27:03.612404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:27:03.612404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:27:03.613436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:27:03.613436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:27:03.613436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:03.613436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:27:03.613436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:03.613436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:27:03.613436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:03.613436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:27:03.613436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:03.613436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:27:03.613436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:03.613436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:27:03.613436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:27:03.613436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:03.613436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:27:03.613436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:03.613436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:27:03.613436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:27:03.613436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:03.614995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:27:03.614995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:03.614995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:27:03.614995 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:27:03.614995 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:27:03.614995 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:27:03.614995 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:27:03.615931 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:27:03.615931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:03.616279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:27:03.616279 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:03.616279 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:27:03.616279 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:27:03.616279 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:03.616279 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:27:03.616279 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:03.616279 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:27:03.617126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:03.617126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:27:03.617126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:27:03.617126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:27:03.617126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:03.617126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:27:03.617126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:03.617126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:27:03.617126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:03.617126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:27:03.617126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:03.617126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:27:03.617126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:03.617126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:27:03.617126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:27:03.617625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:03.617625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:27:03.617625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:03.617625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:27:03.617625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:27:03.617625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:03.617625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:27:03.617625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:03.617625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:27:03.618057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:03.618057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:27:03.618057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:27:03.618057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:27:03.618057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:27:03.618057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:27:03.618057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:27:03.618057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:27:03.618057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:27:03.618057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:27:03.618057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:03.618057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:27:03.618342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:03.618342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:27:03.618342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:03.618342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:27:03.618342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:03.618342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:27:03.618342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:03.618342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:27:03.618342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:27:03.618342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:03.618342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:27:03.618342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:03.618342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:27:03.618342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:27:03.618675 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:03.618675 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:27:03.618675 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:03.618675 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:27:03.619251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:03.619251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:27:03.619251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:27:03.619251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:27:03.619251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:27:03.619251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:27:03.619251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:27:03.619251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:27:03.619251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:27:03.619251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:27:03.619251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:27:03.619251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:27:03.619251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:27:03.619251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:27:03.619251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:27:03.619251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:27:03.619667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:27:03.619667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:27:03.619667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:27:03.619667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:27:03.619667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:27:03.619667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:03.619667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:27:03.619667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:03.619667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:27:03.619667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:03.619667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:27:03.619667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:03.619667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:27:03.619667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:03.619667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:27:03.619667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:27:03.619667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:03.619667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:27:03.620114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:03.620114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:27:03.620114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:27:03.620114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:03.620114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:27:03.620114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:03.620114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:27:03.622083 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:27:03.622421 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:27:03.622533 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:27:03.622671 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:27:03.623209 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:27:03.623602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:03.624165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:27:03.624165 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:03.624165 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:27:03.624165 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:27:03.624165 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:03.624165 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:27:03.624165 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:03.624165 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:27:03.624676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:03.624676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:27:03.624676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:27:03.624676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:27:03.624676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:03.624676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:27:03.624676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:03.624676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:27:03.624676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:03.624676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:27:03.624676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:03.624676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:27:03.624676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:03.624676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:27:03.624676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:27:03.625291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:03.625291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:27:03.625291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:03.625291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:27:03.625291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:27:03.625291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:03.625291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:27:03.625291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:03.625291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:27:03.625631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:03.625631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:27:03.625631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:27:03.625631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:27:03.625631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:27:03.625631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:27:03.625631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:27:03.625631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:27:03.625631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:27:03.625631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:03.625631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:27:03.626059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:03.626059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:27:03.626059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:03.626059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:27:03.626059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:03.626059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:27:03.626059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:03.626059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:27:03.626059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:27:03.626059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:03.626059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:27:03.626059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:03.626059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:27:03.626059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:27:03.626576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:03.626576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:27:03.626576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:03.626576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:27:03.626576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:03.626576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:27:03.626576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:27:03.626576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:27:03.626576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:27:03.626576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:27:03.626576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:27:03.626576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:27:03.626576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:27:03.626576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:27:03.627096 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:27:03.627096 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:27:03.627096 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:27:03.627096 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:27:03.627096 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:27:03.627096 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:27:03.627096 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:27:03.627096 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:27:03.627096 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:27:03.627096 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:27:03.627096 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:03.627096 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:27:03.627096 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:03.627096 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:27:03.627096 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:03.627096 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:27:03.627096 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:03.627096 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:27:03.627096 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:03.627953 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:27:03.627953 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:27:03.627953 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:03.627953 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:27:03.627953 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:03.627953 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:27:03.627953 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:27:03.627953 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:03.627953 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:27:03.627953 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:03.627953 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:27:03.629119 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:27:03.629394 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:27:03.629508 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:27:03.629664 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:27:03.629971 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:27:03.630198 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:27:03.630355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:03.630841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:27:03.630841 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:03.630841 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:27:03.630841 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:27:03.630841 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:03.630841 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:27:03.630841 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:03.630841 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:27:03.632031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:03.632031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:27:03.632031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:27:03.632031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:27:03.632031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:27:03.632031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:27:03.632031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:27:03.632031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:03.632031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:27:03.632031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:03.632031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:27:03.632031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:03.632493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:27:03.632493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:03.632493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:27:03.632493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:03.632493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:27:03.632493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:27:03.632493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:03.632493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:27:03.632493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:03.632493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:27:03.632493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:27:03.632493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:03.632493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:27:03.632947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:03.632947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:27:03.633351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:03.633351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:27:03.633351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:27:03.633351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:27:03.633351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:27:03.633351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:27:03.633351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:27:03.633351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:27:03.633351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:27:03.633351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:27:03.633351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:27:03.633351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:27:03.633351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:27:03.633351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:27:03.633351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:27:03.633351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:27:03.633939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:27:03.633939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:27:03.633939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:03.633939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:27:03.633939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:03.633939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:27:03.633939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:03.633939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:27:03.633939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:03.633939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:27:03.633939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:03.633939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:27:03.633939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:27:03.633939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:03.633939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:27:03.633939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:03.633939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:27:03.633939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:27:03.633939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:03.634562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:27:03.634562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:03.634562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:27:08.453571 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:27:08.454906 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:27:08.454906 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:27:08.456151 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:27:08.489604 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:27:08.490963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:08.490963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:27:08.490963 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:08.490963 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:27:08.490963 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:27:08.490963 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:08.490963 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:27:08.490963 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:08.490963 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:27:08.491640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:08.491640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:27:08.491640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:27:08.491640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:27:08.491640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:27:08.491640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:27:08.491640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:27:08.491640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:08.491640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:27:08.491640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:08.491640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:27:08.491640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:08.492100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:27:08.492100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:08.492100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:27:08.492100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:08.492100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:27:08.492100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:27:08.492100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:08.492100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:27:08.492100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:08.492100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:27:08.492100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:27:08.492100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:08.492100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:27:08.492483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:08.492483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:27:08.492483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:08.492483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:27:08.492483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:27:08.492483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:27:08.492483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:27:08.492483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:27:08.492483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:27:08.492483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:27:08.492483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:27:08.492483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:27:08.492483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:27:08.492483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:27:08.492483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:27:08.493012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:27:08.493012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:27:08.493012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:27:08.493012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:27:08.493012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:27:08.493012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:08.493012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:27:08.493012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:08.493012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:27:08.493012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:08.493012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:27:08.493012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:08.493012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:27:08.493012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:08.493012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:27:08.493012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:27:08.493012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:08.493012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:27:08.493012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:08.493580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:27:08.493580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:27:08.493580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:08.493580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:27:08.493580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:08.493580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:27:08.605382 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:27:08.605921 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:27:08.606033 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:27:08.607194 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:27:08.607564 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:27:08.607858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:08.608768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:27:08.608768 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:08.608768 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:27:08.608768 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:27:08.608768 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:08.608768 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:27:08.608768 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:08.608768 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:27:08.609883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:08.609883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:27:08.609883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:27:08.609883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:27:08.609883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:27:08.609883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:27:08.609883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:27:08.609883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:08.609883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:27:08.609883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:08.609883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:27:08.609883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:08.610260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:27:08.610260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:08.610260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:27:08.610260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:08.610260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:27:08.610260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:27:08.610260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:08.610260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:27:08.610260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:08.610260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:27:08.610260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:27:08.610260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:08.610260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:27:08.610611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:08.610611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:27:08.610985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:08.610985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:27:08.610985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:27:08.610985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:27:08.610985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:27:08.610985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:27:08.610985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:27:08.610985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:27:08.610985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:27:08.610985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:27:08.610985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:27:08.610985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:27:08.610985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:27:08.610985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:27:08.610985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:27:08.610985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:27:08.611424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:27:08.611424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:27:08.611424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:08.611424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:27:08.611424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:08.611424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:27:08.611424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:08.611424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:27:08.611424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:08.611424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:27:08.611424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:08.611424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:27:08.611424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:27:08.611424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:08.611424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:27:08.611424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:08.611424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:27:08.611424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:27:08.611424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:08.611947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:27:08.611947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:08.611947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:27:08.614757 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:27:08.615048 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:27:08.615151 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:27:08.615294 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:27:08.615674 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:27:08.615917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:08.616452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:27:08.616452 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:08.616452 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:27:08.616452 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:27:08.616452 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:08.616452 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:27:08.616452 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:08.616452 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:27:08.617254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:08.617254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:27:08.617254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:27:08.617254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:27:08.617254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:08.617254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:27:08.617254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:08.617254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:27:08.617254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:08.617254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:27:08.617254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:08.617254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:27:08.617254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:08.617254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:27:08.617254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:27:08.618010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:08.618010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:27:08.618010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:08.618010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:27:08.618010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:27:08.618010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:08.618010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:27:08.618010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:08.618010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:27:08.618450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:08.618450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:27:08.618450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:27:08.618450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:27:08.618450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:27:08.618450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:27:08.618450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:27:08.618450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:27:08.618450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:27:08.618450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:27:08.618450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:08.618450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:27:08.618974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:08.618974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:27:08.618974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:08.618974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:27:08.618974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:08.618974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:27:08.618974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:08.618974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:27:08.618974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:27:08.618974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:08.618974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:27:08.618974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:08.618974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:27:08.618974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:27:08.619452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:08.619452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:27:08.619452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:08.619452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:27:08.619678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:08.619678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:27:08.619678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:27:08.619678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:27:08.619678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:27:08.619678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:27:08.619678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:27:08.619678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:27:08.619678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:27:08.619678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:27:08.619678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:27:08.619678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:27:08.619678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:27:08.619678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:27:08.619678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:27:08.619678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:27:08.620207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:27:08.620207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:27:08.620207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:27:08.620207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:27:08.620207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:27:08.620207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:08.620207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:27:08.620207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:08.620207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:27:08.620207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:08.620207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:27:08.620207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:08.620207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:27:08.620207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:08.620207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:27:08.620207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:27:08.620207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:08.620207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:27:08.620686 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:08.620686 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:27:08.620686 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:27:08.620686 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:08.620686 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:27:08.620686 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:08.620686 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:27:08.622592 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:27:08.622940 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:27:08.623053 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:27:08.623182 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:27:08.623682 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:27:08.624128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:08.624968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:27:08.624968 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:08.624968 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:27:08.624968 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:27:08.624968 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:08.624968 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:27:08.624968 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:08.624968 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:27:08.625835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:08.625835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:27:08.625835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:27:08.625835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:27:08.625835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:08.625835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:27:08.625835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:08.625835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:27:08.625835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:08.625835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:27:08.625835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:08.625835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:27:08.625835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:08.625835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:27:08.625835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:27:08.626247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:08.626247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:27:08.626247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:08.626247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:27:08.626247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:27:08.626247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:08.626247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:27:08.626247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:08.626247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:27:08.626722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:08.626722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:27:08.626722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:27:08.626722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:27:08.626722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:27:08.626722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:27:08.626722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:27:08.626722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:27:08.626722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:27:08.626722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:08.626722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:27:08.627099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:08.627099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:27:08.627099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:08.627099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:27:08.627099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:08.627099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:27:08.627099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:08.627099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:27:08.627099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:27:08.627099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:08.627099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:27:08.627099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:08.627099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:27:08.627099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:27:08.627471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:08.627471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:27:08.627471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:08.627471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:27:08.627862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:08.627862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:27:08.627862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:27:08.627862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:27:08.627862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:27:08.627862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:27:08.627862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:27:08.627862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:27:08.627862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:27:08.627862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:27:08.627862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:27:08.627862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:27:08.627862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:27:08.627862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:27:08.627862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:27:08.627862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:27:08.627862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:27:08.628450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:27:08.628450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:27:08.628450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:27:08.628450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:08.628450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:27:08.628450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:08.628450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:27:08.628450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:08.628450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:27:08.628450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:08.628450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:27:08.628450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:08.628450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:27:08.628450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:27:08.628450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:08.628450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:27:08.628450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:08.628450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:27:08.628450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:27:08.628988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:08.628988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:27:08.628988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:08.628988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:27:08.629772 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:27:08.630198 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:27:08.630257 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:27:08.630392 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:27:08.630875 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:27:08.631283 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:27:08.631598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:08.632387 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:27:08.632387 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:08.632387 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:27:08.632387 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:27:08.632387 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:08.632387 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:27:08.632387 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:08.632387 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:27:08.633258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:08.633258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:27:08.633258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:27:08.633258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:27:08.633258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:27:08.633258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:27:08.633258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:27:08.633258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:08.633258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:27:08.633258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:08.633258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:27:08.633258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:08.633670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:27:08.633670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:08.633670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:27:08.633670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:08.633670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:27:08.633670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:27:08.633670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:08.633670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:27:08.633670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:08.633670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:27:08.633670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:27:08.633670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:08.633670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:27:08.634021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:08.634021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:27:08.634307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:08.634307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:27:08.634307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:27:08.634307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:27:08.634307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:27:08.634307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:27:08.634307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:27:08.634307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:27:08.634307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:27:08.634307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:27:08.634307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:27:08.634307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:27:08.634307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:27:08.634307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:27:08.634307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:27:08.634307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:27:08.634757 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:27:08.634757 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:27:08.634757 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:08.634757 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:27:08.634757 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:08.634757 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:27:08.634757 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:08.634757 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:27:08.634757 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:08.634757 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:27:08.634757 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:08.634757 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:27:08.634757 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:27:08.634757 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:08.634757 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:27:08.634757 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:08.634757 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:27:08.634757 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:27:08.634757 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:08.635261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:27:08.635261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:08.635261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:27:13.454734 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:27:13.455722 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:27:13.455722 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:27:13.456648 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:27:13.493937 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:27:13.494472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:13.495569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:27:13.495569 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:13.495569 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:27:13.495569 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:27:13.495569 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:13.495569 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:27:13.495569 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:13.495569 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:27:13.496660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:13.496660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:27:13.496660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:27:13.496660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:27:13.496660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:27:13.496660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:27:13.496660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:27:13.496660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:13.496660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:27:13.496660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:13.496660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:27:13.496660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:13.497581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:27:13.497581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:13.497581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:27:13.497581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:13.497581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:27:13.497581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:27:13.497581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:13.497581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:27:13.497581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:13.497581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:27:13.497581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:27:13.497581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:13.497581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:27:13.498490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:13.498490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:27:13.498490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:13.498490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:27:13.498490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:27:13.498490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:27:13.498490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:27:13.498490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:27:13.498490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:27:13.498490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:27:13.498490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:27:13.498490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:27:13.498490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:27:13.498490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:27:13.498490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:27:13.500000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:27:13.500000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:27:13.500000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:27:13.500000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:27:13.500000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:27:13.500000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:13.500000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:27:13.500000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:13.500000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:27:13.500000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:13.500000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:27:13.500000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:13.500000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:27:13.500000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:13.500000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:27:13.500000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:27:13.500000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:13.500000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:27:13.500000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:13.501194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:27:13.501194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:27:13.501194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:13.501194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:27:13.501194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:13.501194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:27:13.606098 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:27:13.607118 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:27:13.607310 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:27:13.608557 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:27:13.608985 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:27:13.609225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:13.610125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:27:13.610125 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:13.610125 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:27:13.610125 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:27:13.610125 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:13.610125 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:27:13.610125 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:13.610125 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:27:13.611632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:13.611632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:27:13.611632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:27:13.611632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:27:13.611632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:27:13.611632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:27:13.611632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:27:13.611632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:13.611632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:27:13.611632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:13.611632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:27:13.611632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:13.612510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:27:13.612510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:13.612510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:27:13.612510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:13.612510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:27:13.612510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:27:13.612510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:13.612510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:27:13.612510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:13.612510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:27:13.612510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:27:13.612510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:13.612510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:27:13.613278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:13.613278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:27:13.614245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:13.614245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:27:13.614245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:27:13.614245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:27:13.614245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:27:13.614245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:27:13.614245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:27:13.614245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:27:13.614245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:27:13.614245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:27:13.614245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:27:13.614245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:27:13.614245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:27:13.614245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:27:13.614245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:27:13.614245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:27:13.615083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:27:13.615083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:27:13.615083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:13.615083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:27:13.615083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:13.615083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:27:13.615083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:13.615083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:27:13.615083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:13.615083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:27:13.615083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:13.615083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:27:13.615083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:27:13.615083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:13.615083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:27:13.615083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:13.615083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:27:13.615083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:27:13.615083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:13.616564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:27:13.616564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:13.616564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:27:13.616564 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:27:13.616564 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:27:13.616564 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:27:13.616564 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:27:13.617063 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:27:13.617063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:13.617857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:27:13.617857 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:13.617857 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:27:13.617857 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:27:13.617857 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:13.617857 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:27:13.617857 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:13.617857 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:27:13.618872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:13.618872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:27:13.618872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:27:13.618872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:27:13.618872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:13.618872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:27:13.618872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:13.618872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:27:13.618872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:13.618872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:27:13.618872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:13.618872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:27:13.618872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:13.618872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:27:13.618872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:27:13.619436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:13.619436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:27:13.619436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:13.619436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:27:13.619436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:27:13.619436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:13.619436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:27:13.619436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:13.619436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:27:13.620189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:13.620189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:27:13.620189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:27:13.620189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:27:13.620189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:27:13.620189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:27:13.620189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:27:13.620189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:27:13.620189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:27:13.620189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:27:13.620189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:13.620189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:27:13.621396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:13.621396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:27:13.621396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:13.621396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:27:13.621396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:13.621396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:27:13.621396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:13.621396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:27:13.621396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:27:13.621396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:13.621396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:27:13.621396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:13.621396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:27:13.621396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:27:13.621828 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:13.621828 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:27:13.621828 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:13.621828 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:27:13.621828 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:13.621828 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:27:13.621828 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:27:13.621828 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:27:13.621828 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:27:13.621828 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:27:13.621828 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:27:13.621828 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:27:13.621828 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:27:13.621828 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:27:13.622179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:27:13.622179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:27:13.622179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:27:13.622179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:27:13.622179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:27:13.622179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:27:13.622179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:27:13.622179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:27:13.622179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:27:13.622179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:27:13.622179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:27:13.622179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:13.622179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:27:13.622179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:13.622179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:27:13.622179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:13.622179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:27:13.622179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:13.622179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:27:13.622820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:13.622820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:27:13.622820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:27:13.622820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:13.622820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:27:13.622820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:13.622820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:27:13.622820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:27:13.622820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:13.622820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:27:13.622820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:13.622820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:27:13.623260 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:27:13.623351 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:27:13.623507 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:27:13.623686 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:27:13.624327 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:27:13.624823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:13.625783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:27:13.625783 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:13.625783 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:27:13.625783 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:27:13.625783 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:13.625783 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:27:13.625783 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:13.625783 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:27:13.626709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:13.626709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:27:13.626709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:27:13.626709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:27:13.626709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:13.626709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:27:13.626709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:13.626709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:27:13.626709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:13.626709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:27:13.626709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:13.626709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:27:13.626709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:13.626709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:27:13.626709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:27:13.627292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:13.627292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:27:13.627292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:13.627292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:27:13.627292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:27:13.627292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:13.627292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:27:13.627292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:13.627292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:27:13.627931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:13.627931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:27:13.627931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:27:13.627931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:27:13.627931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:27:13.627931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:27:13.627931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:27:13.627931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:27:13.627931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:27:13.627931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:13.627931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:27:13.628372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:13.628372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:27:13.628372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:13.628372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:27:13.628372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:13.628372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:27:13.628372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:13.628372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:27:13.628372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:27:13.628372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:13.628372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:27:13.628372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:13.628372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:27:13.628372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:27:13.628933 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:13.628933 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:27:13.628933 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:13.628933 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:27:13.629397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:13.629397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:27:13.629397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:27:13.629397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:27:13.629397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:27:13.629397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:27:13.629397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:27:13.629397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:27:13.629397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:27:13.629397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:27:13.629397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:27:13.629397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:27:13.629397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:27:13.629397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:27:13.629397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:27:13.629397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:27:13.629397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:27:13.630047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:27:13.630047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:27:13.630047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:27:13.630047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:13.630047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:27:13.630047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:13.630047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:27:13.630047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:13.630047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:27:13.630047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:13.630047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:27:13.630047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:13.630047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:27:13.630047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:27:13.630047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:13.630047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:27:13.630047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:13.630047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:27:13.630047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:27:13.630860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:13.630860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:27:13.630860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:13.630860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:27:13.630860 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:27:13.630860 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:27:13.630860 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:27:13.630860 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:27:13.631578 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:27:13.631937 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:27:13.632166 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:13.632896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:27:13.632896 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:13.632896 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:27:13.632896 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:27:13.632896 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:13.632896 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:27:13.632896 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:13.632896 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:27:13.633904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:13.633904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:27:13.633904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:27:13.633904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:27:13.633904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:27:13.633904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:27:13.633904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:27:13.633904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:13.633904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:27:13.633904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:13.633904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:27:13.633904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:13.634460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:27:13.634460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:13.634460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:27:13.634460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:13.634460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:27:13.634460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:27:13.634460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:13.634460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:27:13.634460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:13.634460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:27:13.634460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:27:13.634460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:13.634460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:27:13.634958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:13.634958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:27:13.635311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:13.635311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:27:13.635311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:27:13.635311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:27:13.635311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:27:13.635311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:27:13.635311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:27:13.635311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:27:13.635311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:27:13.635311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:27:13.635311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:27:13.635311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:27:13.635311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:27:13.635311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:27:13.635311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:27:13.635311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:27:13.636028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:27:13.636028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:27:13.636028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:13.636028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:27:13.636028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:13.636028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:27:13.636028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:13.636028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:27:13.636028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:13.636028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:27:13.636028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:13.636028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:27:13.636028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:27:13.636028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:13.636028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:27:13.636028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:13.636028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:27:13.636028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:27:13.636028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:13.636530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:27:13.636530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:13.636530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:27:18.455468 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:27:18.456353 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:27:18.456353 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:27:18.457539 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:27:18.487539 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:27:18.487963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:18.488903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:27:18.488903 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:18.488903 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:27:18.488903 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:27:18.488903 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:18.488903 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:27:18.488903 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:18.488903 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:27:18.490092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.033 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:18.490092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:27:18.490092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:27:18.490092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:27:18.490092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:27:18.490092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:27:18.490092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:27:18.490092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:18.490092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:27:18.490092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:18.490092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:27:18.490092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:18.490655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:27:18.490655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:18.490655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:27:18.490655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:18.490655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:27:18.490655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:27:18.490655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:18.490655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:27:18.490655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:18.490655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:27:18.490655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:27:18.490655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:18.490655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:27:18.491291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:18.491291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:27:18.491574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:18.491574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:27:18.491574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:27:18.491574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:27:18.491574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:27:18.491574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:27:18.491574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:27:18.491574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:27:18.491574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:27:18.491574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:27:18.491574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:27:18.491574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:27:18.491574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:27:18.491574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:27:18.491574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:27:18.491574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:27:18.492161 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:27:18.492161 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:27:18.492161 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:18.492161 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:27:18.492161 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:18.492161 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:27:18.492161 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:18.492161 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:27:18.492161 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:18.492161 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:27:18.492161 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:18.492161 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:27:18.492161 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:27:18.492161 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:18.492161 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:27:18.492161 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:18.492161 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:27:18.492161 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:27:18.492161 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:18.492593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:27:18.492593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:18.492593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:27:18.607262 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:27:18.607581 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:27:18.607768 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:27:18.608936 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:27:18.609308 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:27:18.609609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:18.610574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:27:18.610574 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:18.610574 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:27:18.610574 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:27:18.610574 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:18.610574 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:27:18.610574 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:18.610574 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:27:18.612323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:18.612323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:27:18.612323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:27:18.612323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:27:18.612323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:27:18.612323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:27:18.612323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:27:18.612323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:18.612323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:27:18.612323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:18.612323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:27:18.612323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:18.612946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:27:18.612946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:18.612946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:27:18.612946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:18.612946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:27:18.612946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:27:18.612946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:18.612946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:27:18.612946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:18.612946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:27:18.612946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:27:18.612946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:18.612946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:27:18.613477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:18.613477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:27:18.614939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:18.614939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:27:18.614939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:27:18.614939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:27:18.614939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:27:18.614939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:27:18.614939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:27:18.614939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:27:18.614939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:27:18.614939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:27:18.614939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:27:18.614939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:27:18.614939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:27:18.614939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:27:18.614939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:27:18.614939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:27:18.615735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:27:18.615735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:27:18.615735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:18.615735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:27:18.615735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:18.615735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:27:18.615735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:18.615735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:27:18.615735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:18.615735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:27:18.615735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:18.615735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:27:18.615735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:27:18.615735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:18.615735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:27:18.615735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:18.615735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:27:18.615735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:27:18.615735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:18.616640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:27:18.616640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:18.616640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:27:18.616640 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:27:18.616640 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:27:18.616640 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:27:18.616991 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:27:18.617255 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:27:18.617479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:18.618239 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:27:18.618239 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:18.618239 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:27:18.618239 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:27:18.618239 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:18.618239 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:27:18.618239 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:18.618239 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:27:18.619076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:18.619076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:27:18.619076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:27:18.619076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:27:18.619076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:18.619076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:27:18.619076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:18.619076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:27:18.619076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:18.619076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:27:18.619076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:18.619076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:27:18.619076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:18.619076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:27:18.619076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:27:18.619951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:18.619951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:27:18.619951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:18.619951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:27:18.619951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:27:18.619951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:18.619951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:27:18.619951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:18.619951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:27:18.619951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:18.619951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:27:18.619951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:27:18.619951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:27:18.619951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:27:18.620725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:27:18.620725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:27:18.620725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:27:18.620725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:27:18.620725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:27:18.620725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:18.620725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:27:18.620725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:18.620725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:27:18.620725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:18.620725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:27:18.620725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:18.620725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:27:18.621414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:18.621414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:27:18.621414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:27:18.621414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:18.621414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:27:18.621414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:18.621414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:27:18.621414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:27:18.621414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:18.621414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:27:18.621414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:18.621414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:27:18.622139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:18.622139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:27:18.622139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:27:18.622139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:27:18.622139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:27:18.622139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:27:18.622139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:27:18.622139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:27:18.622139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:27:18.622139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:27:18.622139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:27:18.622139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:27:18.622139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:27:18.622139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:27:18.622139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:27:18.622139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:27:18.622627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:27:18.622627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:27:18.622627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:27:18.622627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:27:18.622627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:27:18.622627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:18.622627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:27:18.622627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:18.622627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:27:18.622627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:18.622627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:27:18.622627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:18.622627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:27:18.622627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:18.622627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:27:18.622627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:27:18.622627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:18.622627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:27:18.623166 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:18.623166 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:27:18.623166 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:27:18.623166 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:18.623166 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:27:18.623166 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:18.623166 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:27:18.623500 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:27:18.623828 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:27:18.623979 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:27:18.624114 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:27:18.624596 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:27:18.625006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:18.625914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:27:18.625914 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:18.625914 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:27:18.625914 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:27:18.625914 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:18.625914 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:27:18.625914 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:18.625914 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:27:18.626847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:18.626847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:27:18.626847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:27:18.626847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:27:18.626847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:18.626847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:27:18.626847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:18.626847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:27:18.626847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:18.626847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:27:18.626847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:18.626847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:27:18.626847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:18.626847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:27:18.626847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:27:18.627343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:18.627343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:27:18.627343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:18.627343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:27:18.627343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:27:18.627343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:18.627343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:27:18.627343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:18.627343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:27:18.627910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:18.627910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:27:18.627910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:27:18.627910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:27:18.627910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:27:18.627910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:27:18.627910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:27:18.627910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:27:18.627910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:27:18.627910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:18.627910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:27:18.628225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:18.628225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:27:18.628225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:18.628225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:27:18.628225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:18.628225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:27:18.628225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:18.628225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:27:18.628225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:27:18.628225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:18.628225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:27:18.628225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:18.628225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:27:18.628225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:27:18.628559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:18.628559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:27:18.628559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:18.628559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:27:18.629007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:18.629007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:27:18.629007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:27:18.629007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:27:18.629007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:27:18.629007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:27:18.629007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:27:18.629007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:27:18.629007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:27:18.629007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:27:18.629007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:27:18.629007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:27:18.629007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:27:18.629007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:27:18.629007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:27:18.629007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:27:18.629007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:27:18.629546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:27:18.629546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:27:18.629546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:27:18.629546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:18.629546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:27:18.629546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:18.629546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:27:18.629546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:18.629546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:27:18.629546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:18.629546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:27:18.629546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:18.629546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:27:18.629546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:27:18.629546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:18.629546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:27:18.629546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:18.629546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:27:18.629546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:27:18.630114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:18.630114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:27:18.630114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:18.630114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:27:18.630635 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:27:18.630989 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:27:18.631136 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:27:18.631332 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:27:18.631918 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:27:18.632386 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:27:18.632637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:18.633417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:27:18.633417 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:18.633417 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:27:18.633417 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:27:18.633417 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:18.633417 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:27:18.633417 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:18.633417 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:27:18.634345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:18.634345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:27:18.634345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:27:18.634345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:27:18.634345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:27:18.634345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:27:18.634345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:27:18.634345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:18.634345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:27:18.634345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:18.634345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:27:18.634345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:18.634711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:27:18.634711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:18.634711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:27:18.634711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:18.634711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:27:18.634711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:27:18.634711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:18.634711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:27:18.634711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:18.634711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:27:18.634711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:27:18.634711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:18.634711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:27:18.635073 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:18.635073 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:27:18.635445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:18.635445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:27:18.635445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:27:18.635445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:27:18.635445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:27:18.635445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:27:18.635445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:27:18.635445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:27:18.635445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:27:18.635445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:27:18.635445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:27:18.635445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:27:18.635445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:27:18.635445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:27:18.635445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:27:18.635445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:27:18.635916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:27:18.635916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:27:18.635916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:18.635916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:27:18.635916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:18.635916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:27:18.635916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:18.635916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:27:18.635916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:18.635916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:27:18.635916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:18.635916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:27:18.635916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:27:18.635916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:18.635916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:27:18.635916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:18.635916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:27:18.635916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:27:18.635916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:18.636414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:27:18.636414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:18.636414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:27:23.456261 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:27:23.457558 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:27:23.457558 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:27:23.458732 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:27:23.499258 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:27:23.499503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:23.500632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:27:23.500632 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:23.500632 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:27:23.500632 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:27:23.500632 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:23.500632 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:27:23.500632 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:23.500632 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:27:23.501934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.044 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:23.501934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:27:23.501934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:27:23.501934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:27:23.501934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:27:23.501934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:27:23.501934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:27:23.501934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:23.501934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:27:23.501934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:23.501934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:27:23.501934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:23.502524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:27:23.502524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:23.502524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:27:23.502524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:23.502524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:27:23.502524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:27:23.502524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:23.502524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:27:23.502524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:23.502524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:27:23.502524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:27:23.502524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:23.502524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:27:23.503223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:23.503223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:27:23.503460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:23.503460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:27:23.503460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:27:23.503460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:27:23.503460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:27:23.503460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:27:23.503460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:27:23.503460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:27:23.503460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:27:23.503460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:27:23.503460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:27:23.503460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:27:23.503460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:27:23.503460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:27:23.503460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:27:23.503460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:27:23.504098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:27:23.504098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:27:23.504098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:23.504098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:27:23.504098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:23.504098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:27:23.504098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:23.504098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:27:23.504098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:23.504098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:27:23.504098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:23.504098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:27:23.504098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:27:23.504098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:23.504098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:27:23.504098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:23.504098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:27:23.504098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:27:23.504098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:23.504651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:27:23.504651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:23.504651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:27:23.607826 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:27:23.608356 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:27:23.608623 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:27:23.609839 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:27:23.610291 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:27:23.610291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:23.610891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:27:23.610891 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:23.610891 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:27:23.610891 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:27:23.610891 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:23.610891 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:27:23.610891 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:23.610891 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:27:23.612247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:23.612247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:27:23.612247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:27:23.612247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:27:23.612247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:27:23.612247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:27:23.612247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:27:23.612247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:23.612247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:27:23.612247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:23.612247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:27:23.612247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:23.612829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:27:23.612829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:23.612829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:27:23.612829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:23.612829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:27:23.612829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:27:23.612829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:23.612829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:27:23.612829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:23.612829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:27:23.612829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:27:23.612829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:23.612829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:27:23.613333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:23.613333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:27:23.613816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:23.613816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:27:23.613816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:27:23.613816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:27:23.613816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:27:23.613816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:27:23.613816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:27:23.613816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:27:23.613816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:27:23.613816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:27:23.613816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:27:23.613816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:27:23.613816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:27:23.613816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:27:23.613816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:27:23.613816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:27:23.614485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:27:23.614485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:27:23.614485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:23.614485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:27:23.614485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:23.614485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:27:23.614485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:23.614485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:27:23.614485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:23.614485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:27:23.614485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:23.614485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:27:23.614485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:27:23.614485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:23.614485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:27:23.614485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:23.614485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:27:23.614485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:27:23.614485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:23.615141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:27:23.615141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:23.615141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:27:23.616354 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:27:23.616620 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:27:23.616839 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:27:23.616944 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:27:23.617473 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:27:23.617736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:23.618407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:27:23.618407 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:23.618407 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:27:23.618407 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:27:23.618407 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:23.618407 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:27:23.618407 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:23.618407 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:27:23.619317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:23.619317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:27:23.619317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:27:23.619317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:27:23.619317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:23.619317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:27:23.619317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:23.619317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:27:23.619317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:23.619317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:27:23.619317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:23.619317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:27:23.619317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:23.619317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:27:23.619317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:27:23.620028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:23.620028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:27:23.620028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:23.620028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:27:23.620028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:27:23.620028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:23.620028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:27:23.620028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:23.620028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:27:23.620502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:23.620502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:27:23.620502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:27:23.620502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:27:23.620502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:27:23.620502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:27:23.620502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:27:23.620502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:27:23.620502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:27:23.620502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:27:23.620502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:23.620502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:27:23.621076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:23.621076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:27:23.621076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:23.621076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:27:23.621076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:23.621076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:27:23.621076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:23.621076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:27:23.621076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:27:23.621076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:23.621076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:27:23.621076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:23.621076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:27:23.621076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:27:23.621643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:23.621643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:27:23.621643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:23.621643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:27:23.622016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:23.622016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:27:23.622016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:27:23.622016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:27:23.622016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:27:23.622016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:27:23.622016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:27:23.622016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:27:23.622016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:27:23.622016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:27:23.622016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:27:23.622016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:27:23.622016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:27:23.622016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:27:23.622016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:27:23.622016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:27:23.622494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:27:23.622494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:27:23.622494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:27:23.622494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:27:23.622494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:27:23.622494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:23.622494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:27:23.622494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:23.622494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:27:23.622494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:23.622494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:27:23.622494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:23.622494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:27:23.622494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:23.622494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:27:23.622494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:27:23.622494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:23.622494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:27:23.623011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:23.623011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:27:23.623011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:27:23.623011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:23.623011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:27:23.623011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:23.623011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:27:23.623756 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:27:23.624027 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:27:23.624180 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:27:23.624360 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:27:23.624768 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:27:23.625022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:23.625588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:27:23.625588 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:23.625588 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:27:23.625588 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:27:23.625588 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:23.625588 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:27:23.625588 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:23.625588 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:27:23.626508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:23.626508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:27:23.626508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:27:23.626508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:27:23.626508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:23.626508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:27:23.626508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:23.626508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:27:23.626508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:23.626508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:27:23.626508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:23.626508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:27:23.626508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:23.626508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:27:23.626508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:27:23.627157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:23.627157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:27:23.627157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:23.627157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:27:23.627157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:27:23.627157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:23.627157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:27:23.627157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:23.627157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:27:23.627772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:23.627772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:27:23.627772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:27:23.627772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:27:23.627772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:27:23.627772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:27:23.627772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:27:23.627772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:27:23.627772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:27:23.627772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:23.627772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:27:23.628306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:23.628306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:27:23.628306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:23.628306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:27:23.628306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:23.628306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:27:23.628306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:23.628306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:27:23.628306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:27:23.628306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:23.628306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:27:23.628306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:23.628306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:27:23.628306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:27:23.628948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:23.628948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:27:23.628948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:23.628948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:27:23.629385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:23.629385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:27:23.629385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:27:23.629385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:27:23.629385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:27:23.629385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:27:23.629385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:27:23.629385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:27:23.629385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:27:23.629385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:27:23.629385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:27:23.629385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:27:23.629385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:27:23.629385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:27:23.629385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:27:23.629385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:27:23.629385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:27:23.629945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:27:23.629945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:27:23.629945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:27:23.629945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:23.629945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:27:23.629945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:23.629945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:27:23.629945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:23.629945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:27:23.629945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:23.629945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:27:23.629945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:23.629945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:27:23.629945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:27:23.629945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:23.629945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:27:23.629945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:23.629945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:27:23.629945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:27:23.630447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:23.630447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:27:23.630447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:23.630447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:27:23.631348 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:27:23.631769 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:27:23.631944 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:27:23.632151 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:27:23.632606 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:27:23.632937 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:27:23.633111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:23.633716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:27:23.633716 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:23.633716 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:27:23.633716 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:27:23.633716 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:23.633716 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:27:23.633716 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:23.633716 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:27:23.634761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:23.634761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:27:23.634761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:27:23.634761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:27:23.634761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:27:23.634761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:27:23.634761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:27:23.634761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:23.634761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:27:23.634761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:23.634761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:27:23.634761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:23.635276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:27:23.635276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:23.635276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:27:23.635276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:23.635276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:27:23.635276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:27:23.635276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:23.635276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:27:23.635276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:23.635276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:27:23.635276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:27:23.635276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:23.635276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:27:23.635780 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:23.635780 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:27:23.636109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:23.636109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:27:23.636109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:27:23.636109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:27:23.636109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:27:23.636109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:27:23.636109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:27:23.636109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:27:23.636109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:27:23.636109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:27:23.636109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:27:23.636109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:27:23.636109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:27:23.636109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:27:23.636109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:27:23.636109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:27:23.636608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:27:23.636608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:27:23.636608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:23.636608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:27:23.636608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:23.636608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:27:23.636608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:23.636608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:27:23.636608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:23.636608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:27:23.636608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:23.636608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:27:23.636608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:27:23.636608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:23.636608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:27:23.636608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:23.636608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:27:23.636608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:27:23.636608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:23.637133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:27:23.637133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:23.637133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:27:28.457057 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:27:28.457985 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:27:28.458150 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:27:28.459740 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:27:28.487897 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:27:28.489941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:28.489941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:27:28.489941 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:28.489941 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:27:28.489941 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:27:28.489941 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:28.489941 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:27:28.489941 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:28.489941 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:27:28.491526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.032 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:28.491526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:27:28.491526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:27:28.491526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:27:28.491526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:27:28.491526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:27:28.491526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:27:28.491526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:28.491526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:27:28.491526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:28.491526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:27:28.491526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:28.492634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:27:28.492634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:28.492634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:27:28.492634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:28.492634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:27:28.492634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:27:28.492634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:28.492634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:27:28.492634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:28.492634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:27:28.492634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:27:28.492634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:28.492634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:27:28.493127 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:28.493127 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:27:28.493342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:28.493342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:27:28.493342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:27:28.493342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:27:28.493342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:27:28.493342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:27:28.493342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:27:28.493342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:27:28.493342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:27:28.493342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:27:28.493342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:27:28.493342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:27:28.493342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:27:28.493342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:27:28.493342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:27:28.493342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:27:28.494220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:27:28.494220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:27:28.494220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:28.494220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:27:28.494220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:28.494220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:27:28.494220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:28.494220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:27:28.494220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:28.494220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:27:28.494220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:28.494220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:27:28.494220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:27:28.494220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:28.494220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:27:28.494220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:28.494220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:27:28.494220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:27:28.494220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:28.494841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:27:28.494841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:28.494841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:27:28.608562 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:27:28.609332 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:27:28.609462 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:27:28.611072 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:27:28.612142 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:27:28.612142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:28.612584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:27:28.612584 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:28.612584 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:27:28.612584 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:27:28.612584 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:28.612584 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:27:28.612584 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:28.612584 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:27:28.614128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:28.614128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:27:28.614128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:27:28.614128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:27:28.614128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:27:28.614128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:27:28.614128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:27:28.614128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:28.614128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:27:28.614128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:28.614128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:27:28.614128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:28.615048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:27:28.615048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:28.615048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:27:28.615048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:28.615048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:27:28.615048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:27:28.615048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:28.615048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:27:28.615048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:28.615048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:27:28.615048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:27:28.615048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:28.615048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:27:28.615864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:28.615864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:27:28.616188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:28.616188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:27:28.616188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:27:28.616188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:27:28.616188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:27:28.616188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:27:28.616188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:27:28.616188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:27:28.616188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:27:28.616188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:27:28.616188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:27:28.616188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:27:28.616188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:27:28.616188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:27:28.616188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:27:28.616188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:27:28.616942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:27:28.616942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:27:28.616942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:28.616942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:27:28.616942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:28.616942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:27:28.616942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:28.616942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:27:28.616942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:28.616942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:27:28.616942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:28.616942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:27:28.616942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:27:28.616942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:28.616942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:27:28.616942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:28.616942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:27:28.616942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:27:28.616942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:28.617606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:27:28.617606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:28.617606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:27:28.617606 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:27:28.617606 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:27:28.617606 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:27:28.617606 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:27:28.618105 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:27:28.618381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:28.619004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:27:28.619004 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:28.619004 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:27:28.619004 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:27:28.619004 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:28.619004 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:27:28.619004 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:28.619004 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:27:28.619769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:28.619769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:27:28.619769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:27:28.619769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:27:28.619769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:28.619769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:27:28.619769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:28.619769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:27:28.619769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:28.619769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:27:28.619769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:28.619769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:27:28.619769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:28.619769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:27:28.619769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:27:28.620516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:28.620516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:27:28.620516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:28.620516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:27:28.620516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:27:28.620516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:28.620516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:27:28.620516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:28.620516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:27:28.621028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:28.621028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:27:28.621028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:27:28.621028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:27:28.621028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:27:28.621028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:27:28.621028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:27:28.621028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:27:28.621028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:27:28.621028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:27:28.621028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:28.621028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:27:28.621556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:28.621556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:27:28.621556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:28.621556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:27:28.621556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:28.621556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:27:28.621556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:28.621556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:27:28.621556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:27:28.621556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:28.621556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:27:28.621556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:28.621556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:27:28.621556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:27:28.622113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:28.622113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:27:28.622113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:28.622113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:27:28.622113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:28.622113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:27:28.622113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:27:28.622113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:27:28.622113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:27:28.622113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:27:28.622113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:27:28.622113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:27:28.622113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:27:28.622113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:27:28.622553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:27:28.622553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:27:28.622553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:27:28.622553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:27:28.622553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:27:28.622553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:27:28.622553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:27:28.622553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:27:28.622553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:27:28.622553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:27:28.622553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:27:28.622553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:28.622553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:27:28.622553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:28.622553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:27:28.622553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:28.622553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:27:28.622553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:28.622553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:27:28.623125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:28.623125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:27:28.623125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:27:28.623125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:28.623125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:27:28.623125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:28.623125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:27:28.623125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:27:28.623125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:28.623125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:27:28.623125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:28.623125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:27:28.624329 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:27:28.624575 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:27:28.624835 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:27:28.625106 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:27:28.625440 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:27:28.625621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:28.626137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:27:28.626137 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:28.626137 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:27:28.626137 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:27:28.626137 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:28.626137 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:27:28.626137 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:28.626137 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:27:28.627013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:28.627013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:27:28.627013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:27:28.627013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:27:28.627013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:28.627013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:27:28.627013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:28.627013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:27:28.627013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:28.627013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:27:28.627013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:28.627013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:27:28.627013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:28.627013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:27:28.627013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:27:28.627440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:28.627440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:27:28.627440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:28.627440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:27:28.627440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:27:28.627440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:28.627440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:27:28.627440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:28.627440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:27:28.627885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:28.627885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:27:28.627885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:27:28.627885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:27:28.627885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:27:28.627885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:27:28.627885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:27:28.627885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:27:28.627885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:27:28.627885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:28.627885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:27:28.628247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:28.628247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:27:28.628247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:28.628247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:27:28.628247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:28.628247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:27:28.628247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:28.628247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:27:28.628247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:27:28.628247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:28.628247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:27:28.628247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:28.628247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:27:28.628247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:27:28.628635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:28.628635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:27:28.628635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:28.628635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:27:28.628984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:28.628984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:27:28.628984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:27:28.628984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:27:28.628984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:27:28.628984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:27:28.628984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:27:28.628984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:27:28.628984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:27:28.628984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:27:28.628984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:27:28.628984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:27:28.628984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:27:28.628984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:27:28.628984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:27:28.628984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:27:28.628984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:27:28.629486 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:27:28.629486 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:27:28.629486 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:27:28.629486 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:28.629486 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:27:28.629486 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:28.629486 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:27:28.629486 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:28.629486 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:27:28.629486 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:28.629486 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:27:28.629486 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:28.629486 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:27:28.629486 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:27:28.629486 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:28.629486 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:27:28.629486 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:28.629486 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:27:28.629486 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:27:28.630162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:28.630162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:27:28.630162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:28.630162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:27:28.631904 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:27:28.632395 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:27:28.632497 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:27:28.632736 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:27:28.633111 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:27:28.633332 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:27:28.633483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:28.634135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:27:28.634135 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:28.634135 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:27:28.634135 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:27:28.634135 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:28.634135 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:27:28.634135 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:28.634135 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:27:28.635049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:28.635049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:27:28.635049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:27:28.635049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:27:28.635049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:27:28.635049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:27:28.635049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:27:28.635049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:28.635049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:27:28.635049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:28.635049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:27:28.635049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:28.635597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:27:28.635597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:28.635597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:27:28.635597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:28.635597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:27:28.635597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:27:28.635597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:28.635597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:27:28.635597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:28.635597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:27:28.635597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:27:28.635597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:28.635597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:27:28.636124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:28.636124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:27:28.636124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:28.636124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:27:28.636124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:27:28.636124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:27:28.636124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:27:28.636124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:27:28.636124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:27:28.636124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:27:28.636124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:27:28.636124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:27:28.636124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:27:28.636124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:27:28.636124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:27:28.636793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:27:28.636793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:27:28.636793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:27:28.636793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:27:28.636793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:27:28.636793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:28.636793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:27:28.636793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:28.636793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:27:28.636793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:28.636793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:27:28.636793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:28.636793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:27:28.636793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:28.636793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:27:28.636793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:27:28.636793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:28.636793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:27:28.636793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:28.637586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:27:28.637586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:27:28.637586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:28.637586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:27:28.637586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:28.637586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:27:33.457931 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:27:33.458917 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:27:33.458917 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:27:33.459412 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:27:33.496086 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:27:33.496352 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:33.497045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:27:33.497045 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:33.497045 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:27:33.497045 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:27:33.497045 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:33.497045 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:27:33.497045 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:33.497045 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:27:33.497873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:33.497873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:27:33.497873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:27:33.497873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:27:33.497873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:27:33.497873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:27:33.497873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:27:33.497873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:33.497873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:27:33.497873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:33.497873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:27:33.497873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:33.498585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:27:33.498585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:33.498585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:27:33.498585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:33.498585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:27:33.498585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:27:33.498585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:33.498585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:27:33.498585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:33.498585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:27:33.498585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:27:33.498585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:33.498585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:27:33.499699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:33.499699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:27:33.499699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:33.499699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:27:33.499699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:27:33.499699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:27:33.499699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:27:33.499699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:27:33.499699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:27:33.499699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:27:33.499699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:27:33.499699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:27:33.499699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:27:33.499699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:27:33.499699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:27:33.500587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:27:33.500587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:27:33.500587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:27:33.500587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:27:33.500587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:27:33.500587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:33.500587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:27:33.500587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:33.500587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:27:33.500587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:33.500587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:27:33.500587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:33.500587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:27:33.500587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:33.500587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:27:33.500587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:27:33.500587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:33.500587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:27:33.500587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:33.501595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:27:33.501595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:27:33.501595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:33.501595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:27:33.501595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:33.501595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:27:33.609158 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:27:33.609722 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:27:33.609995 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:27:33.611279 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:27:33.611734 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:27:33.612193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:33.613189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:27:33.613189 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:33.613189 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:27:33.613189 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:27:33.613189 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:33.613189 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:27:33.613189 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:33.613189 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:27:33.614592 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:33.614592 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:27:33.614592 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:27:33.614592 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:27:33.614592 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:27:33.614592 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:27:33.614592 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:27:33.614592 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:33.614592 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:27:33.614592 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:33.614592 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:27:33.614592 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:33.615358 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:27:33.615358 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:33.615358 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:27:33.615358 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:33.615358 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:27:33.615358 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:27:33.615358 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:33.615358 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:27:33.615358 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:33.615358 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:27:33.615358 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:27:33.615358 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:33.615358 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:27:33.616119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:33.616119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:27:33.616119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:33.616119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:27:33.616119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:27:33.616119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:27:33.616119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:27:33.616119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:27:33.616119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:27:33.616119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:27:33.616119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:27:33.616119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:27:33.616119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:27:33.616119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:27:33.616119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:27:33.617214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:27:33.617214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:27:33.617214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:27:33.617214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:27:33.617214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:27:33.617214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:33.617214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:27:33.617214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:33.617214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:27:33.617214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:33.617214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:27:33.617214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:33.617214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:27:33.617214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:33.617214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:27:33.617214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:27:33.617214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:33.617214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:27:33.617214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:33.618272 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:27:33.618272 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:27:33.618272 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:33.618272 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:27:33.618272 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:33.618272 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:27:33.618272 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:27:33.618272 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:27:33.618272 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:27:33.618272 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:27:33.618638 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:27:33.618638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:33.619084 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:27:33.619084 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:33.619084 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:27:33.619084 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:27:33.619084 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:33.619084 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:27:33.619084 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:33.619084 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:27:33.620117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:33.620117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:27:33.620117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:27:33.620117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:27:33.620117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:33.620117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:27:33.620117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:33.620117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:27:33.620117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:33.620117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:27:33.620117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:33.620117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:27:33.620117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:33.620117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:27:33.620117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:27:33.620977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:33.620977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:27:33.620977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:33.620977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:27:33.620977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:27:33.620977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:33.620977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:27:33.620977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:33.620977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:27:33.621371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:33.621371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:27:33.621371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:27:33.621371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:27:33.621371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:27:33.621371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:27:33.621371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:27:33.621371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:27:33.621371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:27:33.621371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:27:33.621371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:33.621371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:27:33.624927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:33.624927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:27:33.624927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:33.624927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:27:33.624927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:33.624927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:27:33.624927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:33.624927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:27:33.624927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:27:33.624927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:33.624927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:27:33.624927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:33.624927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:27:33.624927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:27:33.625251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:33.625251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:27:33.625251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:33.625251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:27:33.625251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:33.625251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:27:33.625251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:27:33.625251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:27:33.625251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:27:33.625251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:27:33.625251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:27:33.625251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:27:33.625251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:27:33.625251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:27:33.625942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:27:33.625942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:27:33.625942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:27:33.625942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:27:33.625942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:27:33.625942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:27:33.625942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:27:33.625942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:27:33.625942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:27:33.625942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:27:33.625942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:27:33.625942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:33.625942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:27:33.625942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:33.625942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:27:33.625942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:33.625942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:27:33.625942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:33.625942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:27:33.626948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:33.626948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:27:33.626948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:27:33.626948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:33.626948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:27:33.626948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:33.626948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:27:33.626948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:27:33.626948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:33.626948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:27:33.626948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:33.626948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:27:33.626948 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:27:33.626948 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:27:33.627235 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:27:33.627235 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:27:33.627663 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:27:33.627994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:33.628627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:27:33.628627 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:33.628627 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:27:33.628627 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:27:33.628627 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:33.628627 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:27:33.628627 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:33.628627 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:27:33.629507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:33.629507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:27:33.629507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:27:33.629507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:27:33.629507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:33.629507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:27:33.629507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:33.629507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:27:33.629507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:33.629507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:27:33.629507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:33.629507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:27:33.629507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:33.629507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:27:33.629507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:27:33.629947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:33.629947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:27:33.629947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:33.629947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:27:33.629947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:27:33.629947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:33.629947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:27:33.629947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:33.629947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:27:33.630416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:33.630416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:27:33.630416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:27:33.630416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:27:33.630416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:27:33.630416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:27:33.630416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:27:33.630416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:27:33.630416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:27:33.630416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:33.630416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:27:33.630870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:33.630870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:27:33.630870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:33.630870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:27:33.630870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:33.630870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:27:33.630870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:33.630870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:27:33.630870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:27:33.630870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:33.630870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:27:33.630870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:33.630870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:27:33.630870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:27:33.631232 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:33.631232 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:27:33.631232 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:33.631232 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:27:33.631407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:33.631407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:27:33.631407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:27:33.631407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:27:33.631407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:27:33.631407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:27:33.631407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:27:33.631407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:27:33.631407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:27:33.631407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:27:33.631407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:27:33.631407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:27:33.631407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:27:33.631407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:27:33.631407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:27:33.631407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:27:33.631407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:27:33.631883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:27:33.631883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:27:33.631883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:27:33.631883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:33.631883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:27:33.631883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:33.631883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:27:33.631883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:33.631883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:27:33.631883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:33.631883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:27:33.631883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:33.631883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:27:33.631883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:27:33.631883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:33.631883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:27:33.631883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:33.631883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:27:33.631883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:27:33.632341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:33.632341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:27:33.632341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:33.632341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:27:33.632538 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:27:33.632830 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:27:33.632899 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:27:33.633032 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:27:33.633414 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:27:33.633708 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:27:33.633916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:33.634679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:27:33.634679 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:33.634679 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:27:33.634679 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:27:33.634679 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:33.634679 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:27:33.634679 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:33.634679 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:27:33.635534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:33.635534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:27:33.635534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:27:33.635534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:27:33.635534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:27:33.635534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:27:33.635534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:27:33.635534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:33.635534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:27:33.635534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:33.635534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:27:33.635534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:33.635853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:27:33.635853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:33.635853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:27:33.635853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:33.635853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:27:33.635853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:27:33.635853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:33.635853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:27:33.635853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:33.635853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:27:33.635853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:27:33.635853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:33.635853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:27:33.636104 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:33.636104 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:27:33.636393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:33.636393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:27:33.636393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:27:33.636393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:27:33.636393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:27:33.636393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:27:33.636393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:27:33.636393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:27:33.636393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:27:33.636393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:27:33.636393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:27:33.636393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:27:33.636393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:27:33.636393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:27:33.636393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:27:33.636393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:27:33.636814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:27:33.636814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:27:33.636814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:33.636814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:27:33.636814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:33.636814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:27:33.636814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:33.636814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:27:33.636814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:33.636814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:27:33.636814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:33.636814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:27:33.636814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:27:33.636814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:33.636814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:27:33.636814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:33.636814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:27:33.636814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:27:33.636814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:33.637494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:27:33.637494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:33.637494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:27:38.458448 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:27:38.459501 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:27:38.459701 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:27:38.461021 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:27:38.507726 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:27:38.508167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:38.509274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:27:38.509274 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:38.509274 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:27:38.509274 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:27:38.509274 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:38.509274 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:27:38.509274 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:38.509274 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:27:38.510297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.049 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:38.510297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:27:38.510297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:27:38.510297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:27:38.510297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:27:38.510297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:27:38.510297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:27:38.510297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:38.510297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:27:38.510297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:38.510297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:27:38.510297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:38.510686 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:27:38.510686 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:38.510686 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:27:38.510686 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:38.510686 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:27:38.510686 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:27:38.510686 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:38.510686 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:27:38.510686 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:38.510686 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:27:38.510686 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:27:38.510686 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:38.510686 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:27:38.511091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:38.511091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:27:38.511470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:38.511470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:27:38.511470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:27:38.511470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:27:38.511470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:27:38.511470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:27:38.511470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:27:38.511470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:27:38.511470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:27:38.511470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:27:38.511470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:27:38.511470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:27:38.511470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:27:38.511470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:27:38.511470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:27:38.511470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:27:38.512088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:27:38.512088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:27:38.512088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:38.512088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:27:38.512088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:38.512088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:27:38.512088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:38.512088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:27:38.512088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:38.512088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:27:38.512088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:38.512088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:27:38.512088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:27:38.512088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:38.512088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:27:38.512088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:38.512088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:27:38.512088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:27:38.512088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:38.512653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:27:38.512653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:38.512653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:27:38.609912 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:27:38.610400 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:27:38.610621 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:27:38.611838 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:27:38.612105 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:27:38.612277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:38.613385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:27:38.613385 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:38.613385 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:27:38.613385 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:27:38.613385 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:38.613385 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:27:38.613385 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:38.613385 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:27:38.615121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:38.615121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:27:38.615121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:27:38.615121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:27:38.615121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:27:38.615121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:27:38.615121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:27:38.615121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:38.615121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:27:38.615121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:38.615121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:27:38.615121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:38.615722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:27:38.615722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:38.615722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:27:38.615722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:38.615722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:27:38.615722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:27:38.615722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:38.615722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:27:38.615722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:38.615722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:27:38.615722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:27:38.615722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:38.615722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:27:38.616339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:38.616339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:27:38.616339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:38.616339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:27:38.616339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:27:38.616339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:27:38.616339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:27:38.616339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:27:38.616339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:27:38.616339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:27:38.616339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:27:38.616339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:27:38.616339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:27:38.616339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:27:38.616339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:27:38.616947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:27:38.616947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:27:38.616947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:27:38.616947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:27:38.616947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:27:38.616947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:38.616947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:27:38.616947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:38.616947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:27:38.616947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:38.616947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:27:38.616947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:38.616947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:27:38.616947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:38.616947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:27:38.616947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:27:38.616947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:38.616947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:27:38.616947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:38.617804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:27:38.617804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:27:38.617804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:38.617804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:27:38.617804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:38.617804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:27:38.617804 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:27:38.617804 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:27:38.618256 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:27:38.618256 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:27:38.618413 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:27:38.618646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:38.619061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:27:38.619061 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:38.619061 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:27:38.619061 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:27:38.619061 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:38.619061 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:27:38.619061 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:38.619061 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:27:38.619847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:38.619847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:27:38.619847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:27:38.619847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:27:38.619847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:38.619847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:27:38.619847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:38.619847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:27:38.619847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:38.619847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:27:38.619847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:38.619847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:27:38.619847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:38.619847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:27:38.619847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:27:38.620261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:38.620261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:27:38.620261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:38.620261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:27:38.620261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:27:38.620261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:38.620261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:27:38.620261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:38.620261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:27:38.620930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:38.620930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:27:38.620930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:27:38.620930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:27:38.620930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:27:38.620930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:27:38.620930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:27:38.620930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:27:38.620930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:27:38.620930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:27:38.620930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:38.620930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:27:38.621275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:38.621275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:27:38.621275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:38.621275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:27:38.621275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:38.621275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:27:38.621275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:38.621275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:27:38.621275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:27:38.621275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:38.621275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:27:38.621275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:38.621275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:27:38.621275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:27:38.621731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:38.621731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:27:38.621731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:38.621731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:27:38.622069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:38.622069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:27:38.622069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:27:38.622069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:27:38.622069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:27:38.622069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:27:38.622069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:27:38.622069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:27:38.622069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:27:38.622069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:27:38.622069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:27:38.622069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:27:38.622069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:27:38.622069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:27:38.622069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:27:38.622069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:27:38.622493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:27:38.622493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:27:38.622493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:27:38.622493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:27:38.622493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:27:38.622493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:38.622493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:27:38.622493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:38.622493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:27:38.622493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:38.622493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:27:38.622493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:38.622493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:27:38.622493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:38.622493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:27:38.622493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:27:38.622493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:38.622493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:27:38.623098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:38.623098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:27:38.623098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:27:38.623098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:38.623098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:27:38.623098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:38.623098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:27:38.625072 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:27:38.625318 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:27:38.625435 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:27:38.625569 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:27:38.625896 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:27:38.626064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:38.627177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:27:38.627177 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:38.627177 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:27:38.627177 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:27:38.627177 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:38.627177 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:27:38.627177 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:38.627177 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:27:38.628029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:38.628029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:27:38.628029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:27:38.628029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:27:38.628029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:38.628029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:27:38.628029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:38.628029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:27:38.628029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:38.628029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:27:38.628029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:38.628029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:27:38.628029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:38.628029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:27:38.628029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:27:38.628833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:38.628833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:27:38.628833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:38.628833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:27:38.628833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:27:38.628833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:38.628833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:27:38.628833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:38.628833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:27:38.629278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:38.629278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:27:38.629278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:27:38.629278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:27:38.629278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:27:38.629278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:27:38.629278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:27:38.629278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:27:38.629278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:27:38.629278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:38.629278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:27:38.629855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:38.629855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:27:38.629855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:38.629855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:27:38.629855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:38.629855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:27:38.629855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:38.629855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:27:38.629855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:27:38.629855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:38.629855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:27:38.629855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:38.629855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:27:38.629855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:27:38.630500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:38.630500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:27:38.630500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:38.630500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:27:38.630500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:38.630500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:27:38.630500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:27:38.630500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:27:38.630500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:27:38.630500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:27:38.630500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:27:38.630500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:27:38.630500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:27:38.630500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:27:38.631133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:27:38.631133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:27:38.631133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:27:38.631133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:27:38.631133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:27:38.631133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:27:38.631133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:27:38.631133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:27:38.631133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:27:38.631133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:27:38.631133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:38.631133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:27:38.631133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:38.631133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:27:38.631133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:38.631133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:27:38.631133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:38.631133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:27:38.631133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:38.631593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:27:38.631593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:27:38.631593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:38.631593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:27:38.631593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:38.631593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:27:38.631593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:27:38.631593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:38.631593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:27:38.631593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:38.631593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:27:38.632930 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:27:38.633183 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:27:38.633304 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:27:38.633466 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:27:38.633879 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:27:38.634036 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:27:38.634146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:38.634634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:27:38.634634 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:38.634634 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:27:38.634634 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:27:38.634634 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:38.634634 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:27:38.634634 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:38.634634 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:27:38.635699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:38.635699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:27:38.635699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:27:38.635699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:27:38.635699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:27:38.635699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:27:38.635699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:27:38.635699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:38.635699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:27:38.635699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:38.635699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:27:38.635699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:38.636100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:27:38.636100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:38.636100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:27:38.636100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:38.636100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:27:38.636100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:27:38.636100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:38.636100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:27:38.636100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:38.636100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:27:38.636100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:27:38.636100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:38.636100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:27:38.636443 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:38.636443 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:27:38.636863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:38.636863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:27:38.636863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:27:38.636863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:27:38.636863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:27:38.636863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:27:38.636863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:27:38.636863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:27:38.636863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:27:38.636863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:27:38.636863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:27:38.636863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:27:38.636863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:27:38.636863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:27:38.636863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:27:38.636863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:27:38.637881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:27:38.637881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:27:38.637881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:38.637881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:27:38.637881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:38.637881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:27:38.637881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:38.637881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:27:38.637881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:38.637881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:27:38.637881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:38.637881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:27:38.637881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:27:38.637881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:38.637881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:27:38.637881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:38.637881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:27:38.637881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:27:38.637881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:38.638870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:27:38.638870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:38.638870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:27:43.459591 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:27:43.460666 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:27:43.460666 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:27:43.460833 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:27:43.495341 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:27:43.495545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:43.496178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:27:43.496178 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:43.496178 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:27:43.496178 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:27:43.496178 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:43.496178 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:27:43.496178 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:43.496178 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:27:43.496898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:43.496898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:27:43.496898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:27:43.496898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:27:43.496898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:27:43.496898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:27:43.496898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:27:43.496898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:43.496898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:27:43.496898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:43.496898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:27:43.496898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:43.497339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:27:43.497339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:43.497339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:27:43.497339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:43.497339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:27:43.497339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:27:43.497339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:43.497339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:27:43.497339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:43.497339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:27:43.497339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:27:43.497339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:43.497339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:27:43.497779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:43.497779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:27:43.497779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:43.497779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:27:43.497779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:27:43.497779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:27:43.497779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:27:43.497779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:27:43.497779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:27:43.497779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:27:43.497779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:27:43.497779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:27:43.497779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:27:43.497779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:27:43.497779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:27:43.498315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:27:43.498315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:27:43.498315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:27:43.498315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:27:43.498315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:27:43.498315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:43.498315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:27:43.498315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:43.498315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:27:43.498315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:43.498315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:27:43.498315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:43.498315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:27:43.498315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:43.498315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:27:43.498315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:27:43.498315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:43.498315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:27:43.498315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:43.498961 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:27:43.498961 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:27:43.498961 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:43.498961 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:27:43.498961 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:43.498961 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:27:43.610631 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:27:43.611417 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:27:43.611641 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:27:43.612866 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:27:43.613237 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:27:43.613491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:43.614433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:27:43.614433 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:43.614433 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:27:43.614433 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:27:43.614433 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:43.614433 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:27:43.614433 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:43.614433 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:27:43.615991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:43.615991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:27:43.615991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:27:43.615991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:27:43.615991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:27:43.615991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:27:43.615991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:27:43.615991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:43.615991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:27:43.615991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:43.615991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:27:43.615991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:43.616788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:27:43.616788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:43.616788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:27:43.616788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:43.616788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:27:43.616788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:27:43.616788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:43.616788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:27:43.616788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:43.616788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:27:43.616788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:27:43.616788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:43.616788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:27:43.617434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:43.617434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:27:43.617619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:43.617619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:27:43.617619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:27:43.617619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:27:43.617619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:27:43.617619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:27:43.617619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:27:43.617619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:27:43.617619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:27:43.617619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:27:43.617619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:27:43.617619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:27:43.617619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:27:43.617619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:27:43.617619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:27:43.617619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:27:43.618517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:27:43.618517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:27:43.618517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:43.618517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:27:43.618517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:43.618517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:27:43.618517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:43.618517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:27:43.618517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:43.618517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:27:43.618517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:43.618517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:27:43.618517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:27:43.618517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:43.618517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:27:43.618517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:43.618517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:27:43.618517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:27:43.618517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:43.619543 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:27:43.619543 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:43.619543 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:27:43.619543 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:27:43.619543 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:27:43.619543 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:27:43.619543 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:27:43.619543 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:27:43.620112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:43.620990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:27:43.620990 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:43.620990 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:27:43.620990 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:27:43.620990 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:43.620990 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:27:43.620990 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:43.620990 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:27:43.622652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:43.622652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:27:43.622652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:27:43.622652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:27:43.622652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:43.622652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:27:43.622652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:43.622652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:27:43.622652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:43.622652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:27:43.622652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:43.622652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:27:43.622652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:43.622652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:27:43.622652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:27:43.623584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:43.623584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:27:43.623584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:43.623584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:27:43.623584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:27:43.623584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:43.623584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:27:43.623584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:43.623584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:27:43.625001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:43.625001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:27:43.625001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:27:43.625001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:27:43.625001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:27:43.625001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:27:43.625001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:27:43.625001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:27:43.625001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:27:43.625001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:27:43.625001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:43.625001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:27:43.625461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:43.625461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:27:43.625461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:43.625461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:27:43.625461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:43.625461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:27:43.625461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:43.625461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:27:43.625461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:27:43.625461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:43.625461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:27:43.625461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:43.625461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:27:43.625461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:27:43.625882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:43.625882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:27:43.625882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:43.625882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:27:43.626846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:43.626846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:27:43.626846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:27:43.626846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:27:43.626846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:27:43.626846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:27:43.626846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:27:43.626846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:27:43.626846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:27:43.626846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:27:43.626846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:27:43.626846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:27:43.626846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:27:43.626846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:27:43.626846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:27:43.626846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:27:43.627326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:27:43.627326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:27:43.627326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:27:43.627326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:27:43.627326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:27:43.627326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:43.627326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:27:43.627326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:43.627326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:27:43.627326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:43.627326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:27:43.627326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:43.627326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:27:43.627326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:43.627326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:27:43.627326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:27:43.627326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:43.627326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:27:43.627905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:43.627905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:27:43.627905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:27:43.627905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:43.627905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:27:43.627905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:43.627905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:27:43.627905 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:27:43.627905 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:27:43.627905 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:27:43.628257 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:27:43.628257 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:27:43.628685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:43.629490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:27:43.629490 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:43.629490 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:27:43.629490 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:27:43.629490 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:43.629490 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:27:43.629490 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:43.629490 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:27:43.630384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:43.630384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:27:43.630384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:27:43.630384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:27:43.630384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:43.630384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:27:43.630384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:43.630384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:27:43.630384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:43.630384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:27:43.630384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:43.630384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:27:43.630384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:43.630384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:27:43.630384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:27:43.630877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:43.630877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:27:43.630877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:43.630877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:27:43.630877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:27:43.630877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:43.630877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:27:43.630877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:43.630877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:27:43.631433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:43.631433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:27:43.631433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:27:43.631433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:27:43.631433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:27:43.631433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:27:43.631433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:27:43.631433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:27:43.631433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:27:43.631433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:43.631433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:27:43.631820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:43.631820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:27:43.631820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:43.631820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:27:43.631820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:43.631820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:27:43.631820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:43.631820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:27:43.631820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:27:43.631820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:43.631820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:27:43.631820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:43.631820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:27:43.631820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:27:43.632276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:43.632276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:27:43.632276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:43.632276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:27:43.632831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:43.632831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:27:43.632831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:27:43.632831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:27:43.632831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:27:43.632831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:27:43.632831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:27:43.632831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:27:43.632831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:27:43.632831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:27:43.632831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:27:43.632831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:27:43.632831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:27:43.632831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:27:43.632831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:27:43.632831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:27:43.632831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:27:43.633315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:27:43.633315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:27:43.633315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:27:43.633315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:43.633315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:27:43.633315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:43.633315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:27:43.633315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:43.633315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:27:43.633315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:43.633315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:27:43.633315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:43.633315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:27:43.633315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:27:43.633315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:43.633315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:27:43.633315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:43.633315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:27:43.633315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:27:43.633987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:43.633987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:27:43.633987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:43.633987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:27:43.633987 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:27:43.633987 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:27:43.633987 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:27:43.633987 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:27:43.634261 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:27:43.634604 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:27:43.634873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:43.635614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:27:43.635614 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:43.635614 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:27:43.635614 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:27:43.635614 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:43.635614 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:27:43.635614 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:43.635614 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:27:43.636588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:43.636588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:27:43.636588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:27:43.636588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:27:43.636588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:27:43.636588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:27:43.636588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:27:43.636588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:43.636588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:27:43.636588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:43.636588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:27:43.636588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:43.637003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:27:43.637003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:43.637003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:27:43.637003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:43.637003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:27:43.637003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:27:43.637003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:43.637003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:27:43.637003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:43.637003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:27:43.637003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:27:43.637003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:43.637003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:27:43.637365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:43.637365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:27:43.637624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:43.637624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:27:43.637624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:27:43.637624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:27:43.637624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:27:43.637624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:27:43.637624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:27:43.637624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:27:43.637624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:27:43.637624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:27:43.637624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:27:43.637624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:27:43.637624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:27:43.637624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:27:43.637624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:27:43.637624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:27:43.638169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:27:43.638169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:27:43.638169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:43.638169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:27:43.638169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:43.638169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:27:43.638169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:43.638169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:27:43.638169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:43.638169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:27:43.638169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:43.638169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:27:43.638169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:27:43.638169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:43.638169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:27:43.638169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:43.638169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:27:43.638169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:27:43.638169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:43.638683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:27:43.638683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:43.638683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:27:48.460128 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:27:48.460882 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:27:48.460882 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:27:48.461652 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:27:48.506074 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:27:48.506589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:48.507240 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:27:48.507240 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:48.507240 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:27:48.507240 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:27:48.507240 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:48.507240 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:27:48.507240 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:48.507240 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:27:48.507935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.046 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:48.507935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:27:48.507935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:27:48.507935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:27:48.507935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:27:48.507935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:27:48.507935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:27:48.507935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:48.507935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:27:48.507935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:48.507935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:27:48.507935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:48.508397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:27:48.508397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:48.508397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:27:48.508397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:48.508397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:27:48.508397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:27:48.508397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:48.508397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:27:48.508397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:48.508397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:27:48.508397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:27:48.508397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:48.508397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:27:48.508900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:48.508900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:27:48.508900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:48.508900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:27:48.508900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:27:48.508900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:27:48.508900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:27:48.508900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:27:48.508900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:27:48.508900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:27:48.508900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:27:48.508900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:27:48.508900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:27:48.508900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:27:48.508900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:27:48.509398 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:27:48.509398 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:27:48.509398 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:27:48.509398 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:27:48.509398 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:27:48.509398 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:48.509398 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:27:48.509398 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:48.509398 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:27:48.509398 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:48.509398 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:27:48.509398 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:48.509398 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:27:48.509398 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:48.509398 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:27:48.509398 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:27:48.509398 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:48.509398 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:27:48.509398 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:48.510051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:27:48.510051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:27:48.510051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:48.510051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:27:48.510051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:48.510051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:27:48.611827 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:27:48.612279 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:27:48.612470 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:27:48.613423 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:27:48.613722 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:27:48.613982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:48.614814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:27:48.614814 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:48.614814 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:27:48.614814 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:27:48.614814 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:48.614814 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:27:48.614814 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:48.614814 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:27:48.616231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:48.616231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:27:48.616231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:27:48.616231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:27:48.616231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:27:48.616231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:27:48.616231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:27:48.616231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:48.616231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:27:48.616231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:48.616231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:27:48.616231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:48.616831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:27:48.616831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:48.616831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:27:48.616831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:48.616831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:27:48.616831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:27:48.616831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:48.616831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:27:48.616831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:48.616831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:27:48.616831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:27:48.616831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:48.616831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:27:48.617410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:48.617410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:27:48.617899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:48.617899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:27:48.617899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:27:48.617899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:27:48.617899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:27:48.617899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:27:48.617899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:27:48.617899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:27:48.617899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:27:48.617899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:27:48.617899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:27:48.617899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:27:48.617899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:27:48.617899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:27:48.617899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:27:48.617899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:27:48.618640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:27:48.618640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:27:48.618640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:48.618640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:27:48.618640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:48.618640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:27:48.618640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:48.618640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:27:48.618640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:48.618640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:27:48.618640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:48.618640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:27:48.618640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:27:48.618640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:48.618640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:27:48.618640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:48.618640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:27:48.618640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:27:48.618640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:48.619536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:27:48.619536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:48.619536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:27:48.619536 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:27:48.619536 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:27:48.619536 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:27:48.619536 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:27:48.619920 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:27:48.620094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:48.620764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:27:48.620764 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:48.620764 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:27:48.620764 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:27:48.620764 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:48.620764 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:27:48.620764 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:48.620764 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:27:48.621728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:48.621728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:27:48.621728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:27:48.621728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:27:48.621728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:48.621728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:27:48.621728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:48.621728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:27:48.621728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:48.621728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:27:48.621728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:48.621728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:27:48.621728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:48.621728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:27:48.621728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:27:48.622915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:48.622915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:27:48.622915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:48.622915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:27:48.622915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:27:48.622915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:48.622915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:27:48.622915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:48.622915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:27:48.622915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:48.622915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:27:48.622915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:27:48.622915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:27:48.622915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:27:48.623716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:27:48.623716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:27:48.623716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:27:48.623716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:27:48.623716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:27:48.623716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:48.623716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:27:48.623716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:48.623716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:27:48.623716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:48.623716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:27:48.623716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:48.623716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:27:48.624524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:48.624524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:27:48.624524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:27:48.624524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:48.624524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:27:48.624524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:48.624524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:27:48.624524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:27:48.624524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:48.624524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:27:48.624524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:48.624524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:27:48.625303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:48.625303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:27:48.625303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:27:48.625303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:27:48.625303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:27:48.625303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:27:48.625303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:27:48.625303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:27:48.625303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:27:48.625303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:27:48.625303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:27:48.625303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:27:48.625303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:27:48.625303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:27:48.625303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:27:48.625303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:27:48.626169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:27:48.626169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:27:48.626169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:27:48.626169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:27:48.626169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:27:48.626169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:48.626169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:27:48.626169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:48.626169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:27:48.626169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:48.626169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:27:48.626169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:48.626169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:27:48.626169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:48.626169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:27:48.626169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:27:48.626169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:48.626169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:27:48.627352 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:48.627352 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:27:48.627352 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:27:48.627352 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:48.627352 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:27:48.627352 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:48.627352 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:27:48.627529 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:27:48.627806 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:27:48.627917 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:27:48.628032 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:27:48.628324 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:27:48.628545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:48.628987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:27:48.628987 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:48.628987 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:27:48.628987 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:27:48.628987 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:48.628987 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:27:48.628987 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:48.628987 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:27:48.629680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:48.629680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:27:48.629680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:27:48.629680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:27:48.629680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:48.629680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:27:48.629680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:48.629680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:27:48.629680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:48.629680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:27:48.629680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:48.629680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:27:48.629680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:48.629680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:27:48.629680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:27:48.630109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:48.630109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:27:48.630109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:48.630109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:27:48.630109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:27:48.630109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:48.630109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:27:48.630109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:48.630109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:27:48.630570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:48.630570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:27:48.630570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:27:48.630570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:27:48.630570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:27:48.630570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:27:48.630570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:27:48.630570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:27:48.630570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:27:48.630570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:48.630570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:27:48.630943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:48.630943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:27:48.630943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:48.630943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:27:48.630943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:48.630943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:27:48.630943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:48.630943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:27:48.630943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:27:48.630943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:48.630943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:27:48.630943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:48.630943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:27:48.630943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:27:48.631310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:48.631310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:27:48.631310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:48.631310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:27:48.631609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:48.631609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:27:48.631609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:27:48.631609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:27:48.631609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:27:48.631609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:27:48.631609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:27:48.631609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:27:48.631609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:27:48.631609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:27:48.631609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:27:48.631609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:27:48.631609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:27:48.631609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:27:48.631609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:27:48.631609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:27:48.631609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:27:48.632111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:27:48.632111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:27:48.632111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:27:48.632111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:48.632111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:27:48.632111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:48.632111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:27:48.632111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:48.632111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:27:48.632111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:48.632111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:27:48.632111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:48.632111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:27:48.632111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:27:48.632111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:48.632111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:27:48.632111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:48.632111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:27:48.632111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:27:48.632674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:48.632674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:27:48.632674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:48.632674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:27:48.633608 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:27:48.633852 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:27:48.633963 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:27:48.634116 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:27:48.634570 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:27:48.634953 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:27:48.635189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:48.635942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:27:48.635942 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:48.635942 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:27:48.635942 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:27:48.635942 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:48.635942 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:27:48.635942 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:48.635942 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:27:48.636883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:48.636883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:27:48.636883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:27:48.636883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:27:48.636883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:27:48.636883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:27:48.636883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:27:48.636883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:48.636883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:27:48.636883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:48.636883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:27:48.636883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:48.637682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:27:48.637682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:48.637682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:27:48.637682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:48.637682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:27:48.637682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:27:48.637682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:48.637682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:27:48.637682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:48.637682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:27:48.637682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:27:48.637682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:48.637682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:27:48.638549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:48.638549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:27:48.638549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:48.638549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:27:48.638549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:27:48.638549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:27:48.638549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:27:48.638549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:27:48.638549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:27:48.638549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:27:48.638549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:27:48.638549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:27:48.638549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:27:48.638549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:27:48.638549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:27:48.639344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:27:48.639344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:27:48.639344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:27:48.639344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:27:48.639344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:27:48.639344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:48.639344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:27:48.639344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:48.639344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:27:48.639344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:48.639344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:27:48.639344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:48.639344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:27:48.639344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:48.639344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:27:48.639344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:27:48.639344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:48.639344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:27:48.639344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:48.640365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:27:48.640365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:27:48.640365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:48.640365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:27:48.640365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:48.640365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:27:53.460770 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:27:53.461510 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:27:53.461510 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:27:53.462291 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:27:53.498999 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:27:53.499473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:53.500526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:27:53.500526 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:53.500526 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:27:53.500526 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:27:53.500526 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:53.500526 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:27:53.500526 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:53.500526 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:27:53.501316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:53.501316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:27:53.501316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:27:53.501316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:27:53.501316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:27:53.501316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:27:53.501316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:27:53.501316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:53.501316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:27:53.501316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:53.501316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:27:53.501316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:53.502086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:27:53.502086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:53.502086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:27:53.502086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:53.502086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:27:53.502086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:27:53.502086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:53.502086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:27:53.502086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:53.502086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:27:53.502086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:27:53.502086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:53.502086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:27:53.502888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:53.502888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:27:53.502888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:53.502888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:27:53.502888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:27:53.502888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:27:53.502888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:27:53.502888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:27:53.502888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:27:53.502888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:27:53.502888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:27:53.502888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:27:53.502888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:27:53.502888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:27:53.502888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:27:53.503669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:27:53.503669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:27:53.503669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:27:53.503669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:27:53.503669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:27:53.503669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:53.503669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:27:53.503669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:53.503669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:27:53.503669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:53.503669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:27:53.503669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:53.503669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:27:53.503669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:53.503669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:27:53.503669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:27:53.503669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:53.503669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:27:53.503669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:53.504627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:27:53.504627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:27:53.504627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:53.504627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:27:53.504627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:53.504627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:27:53.612300 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:27:53.612610 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:27:53.612800 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:27:53.614045 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:27:53.614361 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:27:53.614585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:53.615405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:27:53.615405 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:53.615405 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:27:53.615405 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:27:53.615405 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:53.615405 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:27:53.615405 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:53.615405 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:27:53.616604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:53.616604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:27:53.616604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:27:53.616604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:27:53.616604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:27:53.616604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:27:53.616604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:27:53.616604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:53.616604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:27:53.616604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:53.616604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:27:53.616604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:53.617425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:27:53.617425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:53.617425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:27:53.617425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:53.617425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:27:53.617425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:27:53.617425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:53.617425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:27:53.617425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:53.617425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:27:53.617425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:27:53.617425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:53.617425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:27:53.618310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:53.618310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:27:53.618619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:53.618619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:27:53.618619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:27:53.618619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:27:53.618619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:27:53.618619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:27:53.618619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:27:53.618619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:27:53.618619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:27:53.618619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:27:53.618619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:27:53.618619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:27:53.618619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:27:53.618619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:27:53.618619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:27:53.618619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:27:53.619546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:27:53.619546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:27:53.619546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:53.619546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:27:53.619546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:53.619546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:27:53.619546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:53.619546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:27:53.619546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:53.619546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:27:53.619546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:53.619546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:27:53.619546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:27:53.619546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:53.619546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:27:53.619546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:53.619546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:27:53.619546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:27:53.619546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:53.620467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:27:53.620467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:53.620467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:27:53.620467 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:27:53.620467 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:27:53.620467 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:27:53.620467 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:27:53.620467 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:27:53.620970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:53.620970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:27:53.620970 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:53.620970 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:27:53.620970 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:27:53.620970 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:53.620970 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:27:53.620970 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:53.620970 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:27:53.622992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:53.622992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:27:53.622992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:27:53.622992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:27:53.622992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:53.622992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:27:53.622992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:53.622992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:27:53.622992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:53.622992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:27:53.622992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:53.622992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:27:53.622992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:53.622992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:27:53.622992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:27:53.623518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:53.623518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:27:53.623518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:53.623518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:27:53.623518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:27:53.623518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:53.623518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:27:53.623518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:53.623518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:27:53.624202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:53.624202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:27:53.624202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:27:53.624202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:27:53.624202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:27:53.624202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:27:53.624202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:27:53.624202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:27:53.624202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:27:53.624202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:27:53.624202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:53.624202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:27:53.624585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:53.624585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:27:53.624585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:53.624585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:27:53.624585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:53.624585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:27:53.624585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:53.624585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:27:53.624585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:27:53.624585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:53.624585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:27:53.624585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:53.624585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:27:53.624585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:27:53.624991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:53.624991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:27:53.624991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:53.624991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:27:53.625434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:53.625434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:27:53.625434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:27:53.625434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:27:53.625434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:27:53.625434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:27:53.625434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:27:53.625434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:27:53.625434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:27:53.625434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:27:53.625434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:27:53.625434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:27:53.625434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:27:53.625434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:27:53.625434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:27:53.625434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:27:53.626107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:27:53.626107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:27:53.626107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:27:53.626107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:27:53.626107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:27:53.626107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:53.626107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:27:53.626107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:53.626107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:27:53.626107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:53.626107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:27:53.626107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:53.626107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:27:53.626107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:53.626107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:27:53.626107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:27:53.626107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:53.626107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:27:53.626762 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:53.626762 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:27:53.626762 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:27:53.626762 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:53.626762 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:27:53.626762 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:53.626762 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:27:53.627899 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:27:53.628119 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:27:53.628240 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:27:53.628362 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:27:53.629037 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:27:53.629255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:53.629960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:27:53.629960 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:53.629960 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:27:53.629960 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:27:53.629960 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:53.629960 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:27:53.629960 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:53.629960 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:27:53.631040 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:53.631040 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:27:53.631040 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:27:53.631040 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:27:53.631040 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:53.631040 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:27:53.631040 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:53.631040 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:27:53.631040 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:53.631040 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:27:53.631040 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:53.631040 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:27:53.631040 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:53.631040 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:27:53.631040 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:27:53.631520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:53.631520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:27:53.631520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:53.631520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:27:53.631520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:27:53.631520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:53.631520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:27:53.631520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:53.631520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:27:53.631520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:53.631520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:27:53.631520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:27:53.631520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:27:53.631520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:27:53.631930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:27:53.631930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:27:53.631930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:27:53.631930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:27:53.631930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:53.631930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:27:53.631930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:53.631930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:27:53.631930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:53.631930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:27:53.631930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:53.631930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:27:53.631930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:53.632319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:27:53.632319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:27:53.632319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:53.632319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:27:53.632319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:53.632319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:27:53.632319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:27:53.632319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:53.632319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:27:53.632319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:53.632319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:27:53.632319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:53.632670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:27:53.632670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:27:53.632670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:27:53.632670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:27:53.632670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:27:53.632670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:27:53.632670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:27:53.632670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:27:53.632670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:27:53.632670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:27:53.632670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:27:53.632670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:27:53.632670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:27:53.632670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:27:53.632670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:27:53.632670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:27:53.632670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:27:53.632670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:27:53.632670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:27:53.633302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:53.633302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:27:53.633302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:53.633302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:27:53.633302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:53.633302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:27:53.633302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:53.633302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:27:53.633302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:53.633302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:27:53.633302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:27:53.633302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:53.633302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:27:53.633302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:53.633302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:27:53.633302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:27:53.633302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:53.633302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:27:53.633302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:53.633302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:27:53.633970 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:27:53.634384 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:27:53.634384 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:27:53.634504 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:27:53.634918 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:27:53.635183 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:27:53.635351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:53.636025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:27:53.636025 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:53.636025 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:27:53.636025 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:27:53.636025 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:53.636025 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:27:53.636025 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:53.636025 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:27:53.636874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:53.636874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:27:53.636874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:27:53.636874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:27:53.636874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:27:53.636874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:27:53.636874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:27:53.636874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:53.636874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:27:53.636874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:53.636874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:27:53.636874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:53.637420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:27:53.637420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:53.637420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:27:53.637420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:53.637420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:27:53.637420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:27:53.637420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:53.637420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:27:53.637420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:53.637420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:27:53.637420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:27:53.637420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:53.637420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:27:53.637872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:53.637872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:27:53.637872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:53.637872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:27:53.637872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:27:53.637872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:27:53.637872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:27:53.637872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:27:53.637872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:27:53.637872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:27:53.637872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:27:53.637872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:27:53.637872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:27:53.637872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:27:53.637872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:27:53.638414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:27:53.638414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:27:53.638414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:27:53.638414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:27:53.638414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:27:53.638414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:53.638414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:27:53.638414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:53.638414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:27:53.638414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:53.638414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:27:53.638414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:53.638414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:27:53.638414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:53.638414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:27:53.638414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:27:53.638414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:53.638414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:27:53.638414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:53.639168 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:27:53.639168 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:27:53.639168 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:53.639168 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:27:53.639168 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:53.639168 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:27:58.461790 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:27:58.462487 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:27:58.462487 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:27:58.462927 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:27:58.464398 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG dbcounter [-] [106314] Writing DB stats neutron:SELECT=240 {{(pid=106314) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 09 09:27:58.495695 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:27:58.496256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:58.496942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:27:58.496942 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:58.496942 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:27:58.496942 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:27:58.496942 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:58.496942 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:27:58.496942 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:58.496942 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:27:58.497819 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:58.497819 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:27:58.497819 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:27:58.497819 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:27:58.497819 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:27:58.497819 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:27:58.497819 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:27:58.497819 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:58.497819 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:27:58.497819 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:58.497819 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:27:58.497819 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:58.498328 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:27:58.498328 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:58.498328 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:27:58.498328 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:58.498328 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:27:58.498328 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:27:58.498328 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:58.498328 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:27:58.498328 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:58.498328 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:27:58.498328 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:27:58.498328 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:58.498328 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:27:58.498876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:58.498876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:27:58.498876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:58.498876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:27:58.498876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:27:58.498876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:27:58.498876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:27:58.498876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:27:58.498876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:27:58.498876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:27:58.498876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:27:58.498876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:27:58.498876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:27:58.498876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:27:58.498876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:27:58.499452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:27:58.499452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:27:58.499452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:27:58.499452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:27:58.499452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:27:58.499452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:58.499452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:27:58.499452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:58.499452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:27:58.499452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:58.499452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:27:58.499452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:58.499452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:27:58.499452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:58.499452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:27:58.499452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:27:58.499452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:58.499452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:27:58.499452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:58.500216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:27:58.500216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:27:58.500216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:58.500216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:27:58.500216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:58.500216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:27:58.612984 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:27:58.613631 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:27:58.613854 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:27:58.614946 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:27:58.615157 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:27:58.615319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:58.616067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:27:58.616067 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:58.616067 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:27:58.616067 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:27:58.616067 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:58.616067 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:27:58.616067 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:58.616067 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:27:58.617684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:58.617684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:27:58.617684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:27:58.617684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:27:58.617684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:27:58.617684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:27:58.617684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:27:58.617684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:58.617684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:27:58.617684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:58.617684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:27:58.617684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:58.618362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:27:58.618362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:58.618362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:27:58.618362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:58.618362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:27:58.618362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:27:58.618362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:58.618362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:27:58.618362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:58.618362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:27:58.618362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:27:58.618362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:58.618362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:27:58.618978 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:58.618978 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:27:58.619878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:58.619878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:27:58.619878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:27:58.619878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:27:58.619878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:27:58.619878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:27:58.619878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:27:58.619878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:27:58.619878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:27:58.619878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:27:58.619878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:27:58.619878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:27:58.619878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:27:58.619878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:27:58.619878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:27:58.619878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:27:58.620591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:27:58.620591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:27:58.620591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:58.620591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:27:58.620591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:58.620591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:27:58.620591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:58.620591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:27:58.620591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:58.620591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:27:58.620591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:58.620591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:27:58.620591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:27:58.620591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:58.620591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:27:58.620591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:58.620591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:27:58.620591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:27:58.620591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:58.621507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:27:58.621507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:58.621507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:27:58.621507 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:27:58.621507 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:27:58.621507 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:27:58.621507 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:27:58.621507 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:27:58.621890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:58.622255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:27:58.622255 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:58.622255 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:27:58.622255 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:27:58.622255 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:58.622255 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:27:58.622255 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:58.622255 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:27:58.623045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:58.623045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:27:58.623045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:27:58.623045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:27:58.623045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:58.623045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:27:58.623045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:58.623045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:27:58.623045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:58.623045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:27:58.623045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:58.623045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:27:58.623045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:58.623045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:27:58.623045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:27:58.623724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:58.623724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:27:58.623724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:58.623724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:27:58.623724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:27:58.623724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:58.623724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:27:58.623724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:58.623724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:27:58.624243 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:58.624243 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:27:58.624243 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:27:58.624243 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:27:58.624243 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:27:58.624243 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:27:58.624243 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:27:58.624243 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:27:58.624243 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:27:58.624243 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:27:58.624243 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:58.624243 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:27:58.624806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:58.624806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:27:58.624806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:58.624806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:27:58.624806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:58.624806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:27:58.624806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:58.624806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:27:58.624806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:27:58.624806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:58.624806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:27:58.624806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:58.624806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:27:58.624806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:27:58.625450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:58.625450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:27:58.625450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:58.625450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:27:58.625450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:58.625450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:27:58.625450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:27:58.625450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:27:58.625450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:27:58.625450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:27:58.625450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:27:58.625450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:27:58.625450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:27:58.625450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:27:58.626260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:27:58.626260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:27:58.626260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:27:58.626260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:27:58.626260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:27:58.626260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:27:58.626260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:27:58.626260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:27:58.626260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:27:58.626260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:27:58.626260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:27:58.626260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:58.626260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:27:58.626260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:58.626260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:27:58.626260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:58.626260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:27:58.626260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:58.626260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:27:58.627136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:58.627136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:27:58.627136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:27:58.627136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:58.627136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:27:58.627136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:58.627136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:27:58.627136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:27:58.627136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:58.627136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:27:58.627136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:58.627136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:27:58.628490 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:27:58.628490 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:27:58.628490 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:27:58.628709 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:27:58.628935 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:27:58.629178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:58.629664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:27:58.629664 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:58.629664 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:27:58.629664 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:27:58.629664 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:58.629664 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:27:58.629664 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:58.629664 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:27:58.630635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:58.630635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:27:58.630635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:27:58.630635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:27:58.630635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:58.630635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:27:58.630635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:58.630635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:27:58.630635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:58.630635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:27:58.630635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:58.630635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:27:58.630635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:58.630635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:27:58.630635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:27:58.630944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:58.630944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:27:58.630944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:58.630944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:27:58.630944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:27:58.630944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:58.630944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:27:58.630944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:58.630944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:27:58.631846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:58.631846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:27:58.631846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:27:58.631846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:27:58.631846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:27:58.631846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:27:58.631846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:27:58.631846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:27:58.631846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:27:58.631846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:58.631846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:27:58.632092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:58.632092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:27:58.632092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:58.632092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:27:58.632092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:58.632092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:27:58.632092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:58.632092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:27:58.632092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:27:58.632092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:58.632092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:27:58.632092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:58.632092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:27:58.632092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:27:58.632457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:58.632457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:27:58.632457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:58.632457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:27:58.633339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:58.633339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:27:58.633339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:27:58.633339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:27:58.633339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:27:58.633339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:27:58.633339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:27:58.633339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:27:58.633339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:27:58.633339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:27:58.633339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:27:58.633339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:27:58.633339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:27:58.633339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:27:58.633339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:27:58.633339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:27:58.633339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:27:58.633674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:27:58.633674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:27:58.633674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:27:58.633674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:58.633674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:27:58.633674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:58.633674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:27:58.633674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:58.633674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:27:58.633674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:58.633674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:27:58.633674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:58.633674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:27:58.633674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:27:58.633674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:58.633674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:27:58.633674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:58.633674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:27:58.633674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:27:58.634033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:58.634033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:27:58.634033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:58.634033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:27:58.634276 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:27:58.634480 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:27:58.634595 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:27:58.634761 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:27:58.635069 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:27:58.635298 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:27:58.635419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:58.635989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:27:58.635989 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:58.635989 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:27:58.635989 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:27:58.635989 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:58.635989 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:27:58.635989 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:58.635989 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:27:58.636881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:58.636881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:27:58.636881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:27:58.636881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:27:58.636881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:27:58.636881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:27:58.636881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:27:58.636881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:58.636881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:27:58.636881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:58.636881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:27:58.636881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:58.637491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:27:58.637491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:58.637491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:27:58.637491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:58.637491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:27:58.637491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:27:58.637491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:58.637491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:27:58.637491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:58.637491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:27:58.637491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:27:58.637491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:58.637491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:27:58.637987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:58.637987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:27:58.637987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:58.637987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:27:58.637987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:27:58.637987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:27:58.637987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:27:58.637987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:27:58.637987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:27:58.637987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:27:58.637987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:27:58.637987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:27:58.637987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:27:58.637987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:27:58.637987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:27:58.638671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:27:58.638671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:27:58.638671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:27:58.638671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:27:58.638671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:27:58.638671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:58.638671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:27:58.638671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:27:58.638671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:27:58.638671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:27:58.638671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:27:58.638671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:27:58.638671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:27:58.638671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:27:58.638671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:27:58.638671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:27:58.638671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:27:58.638671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:27:58.638671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:27:58.639682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:27:58.639682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:27:58.639682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:27:58.639682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:27:58.639682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:27:58.639682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:28:03.462843 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:28:03.462843 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:28:03.464013 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:28:03.464721 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:28:03.501935 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:28:03.502406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:03.503438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:28:03.503438 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:03.503438 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:28:03.503438 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:28:03.503438 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:03.503438 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:28:03.503438 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:03.503438 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:28:03.504076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:03.504076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:28:03.504076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:28:03.504076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:28:03.504076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:28:03.504076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:28:03.504076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:28:03.504076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:03.504076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:28:03.504076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:03.504076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:28:03.504076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:03.504493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:28:03.504493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:03.504493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:28:03.504493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:03.504493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:28:03.504493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:28:03.504493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:03.504493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:28:03.504493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:03.504493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:28:03.504493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:28:03.504493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:03.504493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:28:03.504896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:03.504896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:28:03.505347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:03.505347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:28:03.505347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:28:03.505347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:28:03.505347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:28:03.505347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:28:03.505347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:28:03.505347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:28:03.505347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:28:03.505347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:28:03.505347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:28:03.505347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:28:03.505347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:28:03.505347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:28:03.505347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:28:03.505347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:28:03.505977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:28:03.505977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:28:03.505977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:03.505977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:28:03.505977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:03.505977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:28:03.505977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:03.505977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:28:03.505977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:03.505977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:28:03.505977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:03.505977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:28:03.505977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:28:03.505977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:03.505977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:28:03.505977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:03.505977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:28:03.505977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:28:03.505977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:03.506550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:28:03.506550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:03.506550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:28:03.613832 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:28:03.614339 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:28:03.614544 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:28:03.615620 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:28:03.616092 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:28:03.616434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:03.617549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:28:03.617549 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:03.617549 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:28:03.617549 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:28:03.617549 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:03.617549 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:28:03.617549 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:03.617549 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:28:03.619244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:03.619244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:28:03.619244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:28:03.619244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:28:03.619244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:28:03.619244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:28:03.619244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:28:03.619244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:03.619244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:28:03.619244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:03.619244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:28:03.619244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:03.619655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:28:03.619655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:03.619655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:28:03.619655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:03.619655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:28:03.619655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:28:03.619655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:03.619655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:28:03.619655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:03.619655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:28:03.619655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:28:03.619655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:03.619655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:28:03.620116 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:03.620116 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:28:03.621236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:03.621236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:28:03.621236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:28:03.621236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:28:03.621236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:28:03.621236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:28:03.621236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:28:03.621236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:28:03.621236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:28:03.621236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:28:03.621236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:28:03.621236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:28:03.621236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:28:03.621236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:28:03.621236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:28:03.621236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:28:03.622005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:28:03.622005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:28:03.622005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:03.622005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:28:03.622005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:03.622005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:28:03.622005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:03.622005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:28:03.622005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:03.622005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:28:03.622005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:03.622005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:28:03.622005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:28:03.622005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:03.622005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:28:03.622005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:03.622005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:28:03.622005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:28:03.622005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:03.622943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:28:03.622943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:03.622943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:28:03.622943 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:28:03.622943 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:28:03.622943 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:28:03.622943 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:28:03.622943 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:28:03.623379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:03.623379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:28:03.623379 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:03.623379 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:28:03.623379 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:28:03.623379 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:03.623379 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:28:03.623379 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:03.623379 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:28:03.624161 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:03.624161 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:28:03.624161 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:28:03.624161 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:28:03.624161 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:03.624161 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:28:03.624161 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:03.624161 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:28:03.624161 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:03.624161 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:28:03.624161 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:03.624161 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:28:03.624161 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:03.624161 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:28:03.624161 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:28:03.625009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:03.625009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:28:03.625009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:03.625009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:28:03.625009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:28:03.625009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:03.625009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:28:03.625009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:03.625009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:28:03.625009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:03.625009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:28:03.625009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:28:03.625009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:28:03.625009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:28:03.625885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:28:03.625885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:28:03.625885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:28:03.625885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:28:03.625885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:28:03.625885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:03.625885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:28:03.625885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:03.625885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:28:03.625885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:03.625885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:28:03.625885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:03.625885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:28:03.626544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:03.626544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:28:03.626544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:28:03.626544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:03.626544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:28:03.626544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:03.626544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:28:03.626544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:28:03.626544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:03.626544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:28:03.626544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:03.626544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:28:03.627118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:03.627118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:28:03.627118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:28:03.627118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:28:03.627118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:28:03.627118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:28:03.627118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:28:03.627118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:28:03.627118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:28:03.627118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:28:03.627118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:28:03.627118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:28:03.627118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:28:03.627118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:28:03.627118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:28:03.627118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:28:03.627855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:28:03.627855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:28:03.627855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:28:03.627855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:28:03.627855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:28:03.627855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:03.627855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:28:03.627855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:03.627855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:28:03.627855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:03.627855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:28:03.627855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:03.627855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:28:03.627855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:03.627855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:28:03.627855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:28:03.627855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:03.627855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:28:03.628796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:03.628796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:28:03.628796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:28:03.628796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:03.628796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:28:03.628796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:03.628796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:28:03.628796 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:28:03.628796 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:28:03.628796 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:28:03.629486 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:28:03.629486 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:28:03.629928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:03.630642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:28:03.630642 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:03.630642 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:28:03.630642 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:28:03.630642 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:03.630642 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:28:03.630642 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:03.630642 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:28:03.631357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:03.631357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:28:03.631357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:28:03.631357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:28:03.631357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:03.631357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:28:03.631357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:03.631357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:28:03.631357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:03.631357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:28:03.631357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:03.631357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:28:03.631357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:03.631357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:28:03.631357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:28:03.631653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:03.631653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:28:03.631653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:03.631653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:28:03.631653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:28:03.631653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:03.631653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:28:03.631653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:03.631653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:28:03.632256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:03.632256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:28:03.632256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:28:03.632256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:28:03.632256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:28:03.632256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:28:03.632256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:28:03.632256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:28:03.632256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:28:03.632256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:03.632256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:28:03.632513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:03.632513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:28:03.632513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:03.632513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:28:03.632513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:03.632513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:28:03.632513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:03.632513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:28:03.632513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:28:03.632513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:03.632513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:28:03.632513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:03.632513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:28:03.632513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:28:03.632817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:03.632817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:28:03.632817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:03.632817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:28:03.633382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:03.633382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:28:03.633382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:28:03.633382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:28:03.633382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:28:03.633382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:28:03.633382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:28:03.633382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:28:03.633382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:28:03.633382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:28:03.633382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:28:03.633382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:28:03.633382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:28:03.633382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:28:03.633382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:28:03.633382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:28:03.633382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:28:03.633733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:28:03.633733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:28:03.633733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:28:03.633733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:03.633733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:28:03.633733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:03.633733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:28:03.633733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:03.633733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:28:03.633733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:03.633733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:28:03.633733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:03.633733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:28:03.633733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:28:03.633733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:03.633733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:28:03.633733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:03.633733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:28:03.633733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:28:03.634128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:03.634128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:28:03.634128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:03.634128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:28:03.634536 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:28:03.634736 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:28:03.634871 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:28:03.635021 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:28:03.635451 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:28:03.635884 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:28:03.636216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:03.636933 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:28:03.636933 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:03.636933 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:28:03.636933 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:28:03.636933 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:03.636933 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:28:03.636933 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:03.636933 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:28:03.637814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:03.637814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:28:03.637814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:28:03.637814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:28:03.637814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:28:03.637814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:28:03.637814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:28:03.637814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:03.637814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:28:03.637814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:03.637814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:28:03.637814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:03.638154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:28:03.638154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:03.638154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:28:03.638154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:03.638154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:28:03.638154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:28:03.638154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:03.638154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:28:03.638154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:03.638154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:28:03.638154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:28:03.638154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:03.638154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:28:03.638554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:03.638554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:28:03.638789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:03.638789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:28:03.638789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:28:03.638789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:28:03.638789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:28:03.638789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:28:03.638789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:28:03.638789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:28:03.638789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:28:03.638789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:28:03.638789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:28:03.638789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:28:03.638789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:28:03.638789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:28:03.638789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:28:03.638789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:28:03.639345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:28:03.639345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:28:03.639345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:03.639345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:28:03.639345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:03.639345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:28:03.639345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:03.639345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:28:03.639345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:03.639345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:28:03.639345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:03.639345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:28:03.639345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:28:03.639345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:03.639345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:28:03.639345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:03.639345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:28:03.639345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:28:03.639345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:03.640162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:28:03.640162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:03.640162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:28:08.462972 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:28:08.463641 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:28:08.463641 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:28:08.465126 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:28:08.501667 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:28:08.501877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:08.502539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:28:08.502539 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:08.502539 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:28:08.502539 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:28:08.502539 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:08.502539 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:28:08.502539 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:08.502539 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:28:08.504211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:08.504211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:28:08.504211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:28:08.504211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:28:08.504211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:28:08.504211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:28:08.504211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:28:08.504211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:08.504211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:28:08.504211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:08.504211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:28:08.504211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:08.505098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:28:08.505098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:08.505098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:28:08.505098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:08.505098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:28:08.505098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:28:08.505098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:08.505098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:28:08.505098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:08.505098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:28:08.505098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:28:08.505098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:08.505098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:28:08.505926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:08.505926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:28:08.505926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:08.505926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:28:08.505926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:28:08.505926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:28:08.505926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:28:08.505926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:28:08.505926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:28:08.505926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:28:08.505926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:28:08.505926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:28:08.505926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:28:08.505926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:28:08.505926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:28:08.506909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:28:08.506909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:28:08.506909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:28:08.506909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:28:08.506909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:28:08.506909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:08.506909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:28:08.506909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:08.506909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:28:08.506909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:08.506909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:28:08.506909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:08.506909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:28:08.506909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:08.506909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:28:08.506909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:28:08.506909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:08.506909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:28:08.506909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:08.508004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:28:08.508004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:28:08.508004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:08.508004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:28:08.508004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:08.508004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:28:08.614559 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:28:08.615021 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:28:08.615214 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:28:08.616256 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:28:08.616508 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:28:08.616658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:08.617401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:28:08.617401 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:08.617401 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:28:08.617401 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:28:08.617401 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:08.617401 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:28:08.617401 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:08.617401 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:28:08.619055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:08.619055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:28:08.619055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:28:08.619055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:28:08.619055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:28:08.619055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:28:08.619055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:28:08.619055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:08.619055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:28:08.619055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:08.619055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:28:08.619055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:08.619501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:28:08.619501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:08.619501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:28:08.619501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:08.619501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:28:08.619501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:28:08.619501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:08.619501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:28:08.619501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:08.619501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:28:08.619501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:28:08.619501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:08.619501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:28:08.619855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:08.619855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:28:08.621056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:08.621056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:28:08.621056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:28:08.621056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:28:08.621056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:28:08.621056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:28:08.621056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:28:08.621056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:28:08.621056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:28:08.621056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:28:08.621056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:28:08.621056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:28:08.621056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:28:08.621056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:28:08.621056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:28:08.621056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:28:08.621544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:28:08.621544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:28:08.621544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:08.621544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:28:08.621544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:08.621544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:28:08.621544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:08.621544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:28:08.621544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:08.621544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:28:08.621544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:08.621544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:28:08.621544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:28:08.621544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:08.621544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:28:08.621544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:08.621544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:28:08.621544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:28:08.621544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:08.622059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:28:08.622059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:08.622059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:28:08.622059 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:28:08.622186 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:28:08.622288 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:28:08.622405 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:28:08.622924 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:28:08.623277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:08.624201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:28:08.624201 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:08.624201 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:28:08.624201 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:28:08.624201 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:08.624201 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:28:08.624201 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:08.624201 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:28:08.624764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:08.624764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:28:08.624764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:28:08.624764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:28:08.624764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:08.624764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:28:08.624764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:08.624764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:28:08.624764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:08.624764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:28:08.624764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:08.624764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:28:08.624764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:08.624764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:28:08.624764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:28:08.625304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:08.625304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:28:08.625304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:08.625304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:28:08.625304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:28:08.625304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:08.625304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:28:08.625304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:08.625304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:28:08.625800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:08.625800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:28:08.625800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:28:08.625800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:28:08.625800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:28:08.625800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:28:08.625800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:28:08.625800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:28:08.625800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:28:08.625800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:28:08.625800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:08.625800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:28:08.626316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:08.626316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:28:08.626316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:08.626316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:28:08.626316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:08.626316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:28:08.626316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:08.626316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:28:08.626316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:28:08.626316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:08.626316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:28:08.626316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:08.626316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:28:08.626316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:28:08.626703 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:08.626703 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:28:08.626703 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:08.626703 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:28:08.626843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:08.626843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:28:08.626843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:28:08.626843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:28:08.626843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:28:08.626843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:28:08.626843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:28:08.626843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:28:08.626843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:28:08.626843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:28:08.626843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:28:08.626843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:28:08.626843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:28:08.626843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:28:08.626843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:28:08.626843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:28:08.627286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:28:08.627286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:28:08.627286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:28:08.627286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:28:08.627286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:28:08.627286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:08.627286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:28:08.627286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:08.627286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:28:08.627286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:08.627286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:28:08.627286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:08.627286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:28:08.627286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:08.627286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:28:08.627286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:28:08.627286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:08.627286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:28:08.627810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:08.627810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:28:08.627810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:28:08.627810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:08.627810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:28:08.627810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:08.627810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:28:08.628717 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:28:08.628918 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:28:08.629026 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:28:08.629153 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:28:08.629981 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:28:08.630464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:08.631356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:28:08.631356 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:08.631356 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:28:08.631356 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:28:08.631356 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:08.631356 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:28:08.631356 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:08.631356 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:28:08.632107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:08.632107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:28:08.632107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:28:08.632107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:28:08.632107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:08.632107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:28:08.632107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:08.632107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:28:08.632107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:08.632107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:28:08.632107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:08.632107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:28:08.632107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:08.632107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:28:08.632107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:28:08.632523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:08.632523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:28:08.632523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:08.632523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:28:08.632523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:28:08.632523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:08.632523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:28:08.632523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:08.632523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:28:08.632986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:08.632986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:28:08.632986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:28:08.632986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:28:08.632986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:28:08.632986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:28:08.632986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:28:08.632986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:28:08.632986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:28:08.632986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:08.632986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:28:08.633390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:08.633390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:28:08.633390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:08.633390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:28:08.633390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:08.633390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:28:08.633390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:08.633390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:28:08.633390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:28:08.633390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:08.633390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:28:08.633390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:08.633390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:28:08.633390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:28:08.633786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:08.633786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:28:08.633786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:08.633786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:28:08.634120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:08.634120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:28:08.634120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:28:08.634120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:28:08.634120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:28:08.634120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:28:08.634120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:28:08.634120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:28:08.634120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:28:08.634120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:28:08.634120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:28:08.634120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:28:08.634120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:28:08.634120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:28:08.634120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:28:08.634120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:28:08.634120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:28:08.634661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:28:08.634661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:28:08.634661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:28:08.634661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:08.634661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:28:08.634661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:08.634661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:28:08.634661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:08.634661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:28:08.634661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:08.634661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:28:08.634661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:08.634661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:28:08.634661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:28:08.634661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:08.634661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:28:08.634661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:08.634661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:28:08.634661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:28:08.635183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:08.635183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:28:08.635183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:08.635183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:28:08.635183 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:28:08.635183 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:28:08.635183 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:28:08.635399 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:28:08.635689 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:28:08.636117 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:28:08.636374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:08.637157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:28:08.637157 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:08.637157 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:28:08.637157 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:28:08.637157 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:08.637157 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:28:08.637157 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:08.637157 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:28:08.638008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:08.638008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:28:08.638008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:28:08.638008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:28:08.638008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:28:08.638008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:28:08.638008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:28:08.638008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:08.638008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:28:08.638008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:08.638008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:28:08.638008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:08.638480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:28:08.638480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:08.638480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:28:08.638480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:08.638480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:28:08.638480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:28:08.638480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:08.638480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:28:08.638480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:08.638480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:28:08.638480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:28:08.638480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:08.638480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:28:08.638859 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:08.638859 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:28:08.639078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:08.639078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:28:08.639078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:28:08.639078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:28:08.639078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:28:08.639078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:28:08.639078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:28:08.639078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:28:08.639078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:28:08.639078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:28:08.639078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:28:08.639078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:28:08.639078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:28:08.639078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:28:08.639078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:28:08.639078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:28:08.639600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:28:08.639600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:28:08.639600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:08.639600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:28:08.639600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:08.639600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:28:08.639600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:08.639600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:28:08.639600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:08.639600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:28:08.639600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:08.639600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:28:08.639600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:28:08.639600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:08.639600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:28:08.639600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:08.639600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:28:08.639600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:28:08.639600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:08.640192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:28:08.640192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:08.640192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:28:13.463956 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:28:13.465095 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:28:13.465095 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:28:13.466294 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:28:13.511211 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:28:13.511460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:13.512067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:28:13.512067 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:13.512067 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:28:13.512067 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:28:13.512067 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:13.512067 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:28:13.512067 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:13.512067 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:28:13.513112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.047 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:13.513112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:28:13.513112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:28:13.513112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:28:13.513112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:28:13.513112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:28:13.513112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:28:13.513112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:13.513112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:28:13.513112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:13.513112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:28:13.513112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:13.514009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:28:13.514009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:13.514009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:28:13.514009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:13.514009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:28:13.514009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:28:13.514009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:13.514009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:28:13.514009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:13.514009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:28:13.514009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:28:13.514009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:13.514009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:28:13.514518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:13.514518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:28:13.514518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:13.514518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:28:13.514518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:28:13.514518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:28:13.514518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:28:13.514518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:28:13.514518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:28:13.514518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:28:13.514518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:28:13.514518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:28:13.514518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:28:13.514518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:28:13.514518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:28:13.514997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:28:13.514997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:28:13.514997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:28:13.514997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:28:13.514997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:28:13.514997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:13.514997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:28:13.514997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:13.514997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:28:13.514997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:13.514997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:28:13.514997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:13.514997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:28:13.514997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:13.514997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:28:13.514997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:28:13.514997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:13.514997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:28:13.514997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:13.515584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:28:13.515584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:28:13.515584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:13.515584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:28:13.515584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:13.515584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:28:13.616292 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:28:13.616843 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:28:13.616965 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:28:13.617679 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:28:13.617923 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:28:13.618150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:13.618634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:28:13.618634 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:13.618634 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:28:13.618634 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:28:13.618634 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:13.618634 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:28:13.618634 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:13.618634 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:28:13.619555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:13.619555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:28:13.619555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:28:13.619555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:28:13.619555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:28:13.619555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:28:13.619555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:28:13.619555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:13.619555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:28:13.619555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:13.619555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:28:13.619555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:13.619967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:28:13.619967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:13.619967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:28:13.619967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:13.619967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:28:13.619967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:28:13.619967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:13.619967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:28:13.619967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:13.619967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:28:13.619967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:28:13.619967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:13.619967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:28:13.620374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:13.620374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:28:13.621007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:13.621007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:28:13.621007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:28:13.621007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:28:13.621007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:28:13.621007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:28:13.621007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:28:13.621007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:28:13.621007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:28:13.621007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:28:13.621007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:28:13.621007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:28:13.621007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:28:13.621007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:28:13.621007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:28:13.621007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:28:13.621531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:28:13.621531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:28:13.621531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:13.621531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:28:13.621531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:13.621531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:28:13.621531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:13.621531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:28:13.621531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:13.621531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:28:13.621531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:13.621531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:28:13.621531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:28:13.621531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:13.621531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:28:13.621531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:13.621531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:28:13.621531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:28:13.621531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:13.622106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:28:13.622106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:13.622106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:28:13.622399 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:28:13.622717 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:28:13.622960 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:28:13.623164 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:28:13.623757 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:28:13.624123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:13.625039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:28:13.625039 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:13.625039 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:28:13.625039 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:28:13.625039 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:13.625039 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:28:13.625039 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:13.625039 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:28:13.626428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:13.626428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:28:13.626428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:28:13.626428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:28:13.626428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:13.626428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:28:13.626428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:13.626428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:28:13.626428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:13.626428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:28:13.626428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:13.626428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:28:13.626428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:13.626428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:28:13.626428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:28:13.627001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:13.627001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:28:13.627001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:13.627001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:28:13.627001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:28:13.627001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:13.627001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:28:13.627001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:13.627001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:28:13.627453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:13.627453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:28:13.627453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:28:13.627453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:28:13.627453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:28:13.627453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:28:13.627453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:28:13.627453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:28:13.627453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:28:13.627453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:28:13.627453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:13.627453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:28:13.627966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:13.627966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:28:13.627966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:13.627966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:28:13.627966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:13.627966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:28:13.627966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:13.627966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:28:13.627966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:28:13.627966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:13.627966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:28:13.627966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:13.627966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:28:13.627966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:28:13.628362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:13.628362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:28:13.628362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:13.628362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:28:13.628362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:13.628362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:28:13.628362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:28:13.628362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:28:13.628362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:28:13.628362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:28:13.628362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:28:13.628362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:28:13.628362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:28:13.628362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:28:13.628785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:28:13.628785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:28:13.628785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:28:13.628785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:28:13.628785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:28:13.628785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:28:13.628785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:28:13.628785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:28:13.628785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:28:13.628785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:28:13.628785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:28:13.628785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:13.628785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:28:13.628785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:13.628785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:28:13.628785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:13.628785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:28:13.628785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:13.628785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:28:13.629424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:13.629424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:28:13.629424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:28:13.629424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:13.629424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:28:13.629424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:13.629424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:28:13.629424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:28:13.629424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:13.629424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:28:13.629424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:13.629424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:28:13.629424 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:28:13.629424 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:28:13.629989 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:28:13.629989 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:28:13.630114 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:28:13.630482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:13.631015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:28:13.631015 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:13.631015 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:28:13.631015 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:28:13.631015 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:13.631015 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:28:13.631015 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:13.631015 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:28:13.631696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:13.631696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:28:13.631696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:28:13.631696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:28:13.631696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:13.631696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:28:13.631696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:13.631696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:28:13.631696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:13.631696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:28:13.631696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:13.631696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:28:13.631696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:13.631696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:28:13.631696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:28:13.632486 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:13.632486 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:28:13.632486 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:13.632486 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:28:13.632486 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:28:13.632486 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:13.632486 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:28:13.632486 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:13.632486 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:28:13.632953 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:13.632953 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:28:13.632953 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:28:13.632953 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:28:13.632953 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:28:13.632953 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:28:13.632953 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:28:13.632953 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:28:13.632953 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:28:13.632953 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:13.632953 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:28:13.633523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:13.633523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:28:13.633523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:13.633523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:28:13.633523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:13.633523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:28:13.633523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:13.633523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:28:13.633523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:28:13.633523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:13.633523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:28:13.633523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:13.633523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:28:13.633523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:28:13.634321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:13.634321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:28:13.634321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:13.634321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:28:13.634321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:13.634321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:28:13.634321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:28:13.634321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:28:13.634321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:28:13.634321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:28:13.634321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:28:13.634321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:28:13.634321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:28:13.634321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:28:13.635059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:28:13.635059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:28:13.635059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:28:13.635059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:28:13.635059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:28:13.635059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:28:13.635059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:28:13.635059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:28:13.635059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:28:13.635059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:28:13.635059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:13.635059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:28:13.635059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:13.635059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:28:13.635059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:13.635059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:28:13.635059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:13.635059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:28:13.635059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:13.635993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:28:13.635993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:28:13.635993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:13.635993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:28:13.635993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:13.635993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:28:13.635993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:28:13.635993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:13.635993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:28:13.635993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:13.635993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:28:13.635993 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:28:13.635993 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:28:13.636661 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:28:13.636661 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:28:13.636661 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:28:13.636661 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:28:13.636661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:13.637028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:28:13.637028 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:13.637028 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:28:13.637028 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:28:13.637028 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:13.637028 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:28:13.637028 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:13.637028 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:28:13.638054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:13.638054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:28:13.638054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:28:13.638054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:28:13.638054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:28:13.638054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:28:13.638054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:28:13.638054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:13.638054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:28:13.638054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:13.638054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:28:13.638054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:13.638509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:28:13.638509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:13.638509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:28:13.638509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:13.638509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:28:13.638509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:28:13.638509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:13.638509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:28:13.638509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:13.638509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:28:13.638509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:28:13.638509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:13.638509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:28:13.638897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:13.638897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:28:13.639100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:13.639100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:28:13.639100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:28:13.639100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:28:13.639100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:28:13.639100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:28:13.639100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:28:13.639100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:28:13.639100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:28:13.639100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:28:13.639100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:28:13.639100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:28:13.639100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:28:13.639100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:28:13.639100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:28:13.639100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:28:13.639557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:28:13.639557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:28:13.639557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:13.639557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:28:13.639557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:13.639557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:28:13.639557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:13.639557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:28:13.639557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:13.639557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:28:13.639557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:13.639557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:28:13.639557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:28:13.639557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:13.639557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:28:13.639557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:13.639557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:28:13.639557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:28:13.639557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:13.640098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:28:13.640098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:13.640098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:28:18.464683 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:28:18.465617 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:28:18.465617 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:28:18.467097 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:28:18.508274 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:28:18.508512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:18.509341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:28:18.509341 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:18.509341 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:28:18.509341 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:28:18.509341 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:18.509341 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:28:18.509341 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:18.509341 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:28:18.510591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.044 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:18.510591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:28:18.510591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:28:18.510591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:28:18.510591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:28:18.510591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:28:18.510591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:28:18.510591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:18.510591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:28:18.510591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:18.510591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:28:18.510591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:18.511539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:28:18.511539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:18.511539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:28:18.511539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:18.511539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:28:18.511539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:28:18.511539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:18.511539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:28:18.511539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:18.511539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:28:18.511539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:28:18.511539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:18.511539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:28:18.512262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:18.512262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:28:18.512262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:18.512262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:28:18.512262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:28:18.512262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:28:18.512262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:28:18.512262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:28:18.512262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:28:18.512262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:28:18.512262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:28:18.512262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:28:18.512262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:28:18.512262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:28:18.512262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:28:18.513189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:28:18.513189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:28:18.513189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:28:18.513189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:28:18.513189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:28:18.513189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:18.513189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:28:18.513189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:18.513189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:28:18.513189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:18.513189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:28:18.513189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:18.513189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:28:18.513189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:18.513189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:28:18.513189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:28:18.513189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:18.513189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:28:18.513189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:18.514356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:28:18.514356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:28:18.514356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:18.514356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:28:18.514356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:18.514356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:28:18.617627 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:28:18.618055 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:28:18.618055 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:28:18.619273 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:28:18.619734 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:28:18.619889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:18.621272 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:28:18.621272 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:18.621272 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:28:18.621272 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:28:18.621272 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:18.621272 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:28:18.621272 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:18.621272 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:28:18.622766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:18.622766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:28:18.622766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:28:18.622766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:28:18.622766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:28:18.622766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:28:18.622766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:28:18.622766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:18.622766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:28:18.622766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:18.622766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:28:18.622766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:18.623167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:28:18.623167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:18.623167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:28:18.623167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:18.623167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:28:18.623167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:28:18.623167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:18.623167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:28:18.623167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:18.623167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:28:18.623167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:28:18.623167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:18.623167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:28:18.623524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:18.623524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:28:18.625015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:18.625015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:28:18.625015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:28:18.625015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:28:18.625015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:28:18.625015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:28:18.625015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:28:18.625015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:28:18.625015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:28:18.625015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:28:18.625015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:28:18.625015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:28:18.625015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:28:18.625015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:28:18.625015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:28:18.625015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:28:18.625502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:28:18.625502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:28:18.625502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:18.625502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:28:18.625502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:18.625502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:28:18.625502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:18.625502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:28:18.625502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:18.625502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:28:18.625502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:18.625502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:28:18.625502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:28:18.625502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:18.625502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:28:18.625502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:18.625502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:28:18.625502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:28:18.625502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:18.626031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:28:18.626031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:18.626031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:28:18.626031 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:28:18.626031 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:28:18.626031 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:28:18.626341 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:28:18.626611 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:28:18.626915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:18.627566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:28:18.627566 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:18.627566 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:28:18.627566 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:28:18.627566 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:18.627566 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:28:18.627566 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:18.627566 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:28:18.629040 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:18.629040 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:28:18.629040 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:28:18.629040 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:28:18.629040 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:18.629040 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:28:18.629040 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:18.629040 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:28:18.629040 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:18.629040 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:28:18.629040 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:18.629040 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:28:18.629040 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:18.629040 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:28:18.629040 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:28:18.629909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:18.629909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:28:18.629909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:18.629909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:28:18.629909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:28:18.629909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:18.629909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:28:18.629909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:18.629909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:28:18.631347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:18.631347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:28:18.631347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:28:18.631347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:28:18.631347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:28:18.631347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:28:18.631347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:28:18.631347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:28:18.631347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:28:18.631347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:28:18.631347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:18.631347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:28:18.632047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:18.632047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:28:18.632047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:18.632047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:28:18.632047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:18.632047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:28:18.632047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:18.632047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:28:18.632047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:28:18.632047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:18.632047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:28:18.632047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:18.632047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:28:18.632047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:28:18.632784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:18.632784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:28:18.632784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:18.632784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:28:18.632784 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:28:18.632784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:18.632784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:28:18.632784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:28:18.632784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:28:18.632784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:28:18.632784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:28:18.632784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:28:18.633405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:28:18.633405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:28:18.633405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:28:18.633405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:28:18.633405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:28:18.633405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:28:18.633405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:28:18.633405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:28:18.633405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:28:18.633405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:28:18.633405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:28:18.633405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:28:18.633405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:28:18.633405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:28:18.633405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:18.633405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:28:18.633405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:18.633405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:28:18.633405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:18.634445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:28:18.634445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:18.634445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:28:18.634445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:18.634445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:28:18.634445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:28:18.634445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:18.634445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:28:18.634445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:18.634445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:28:18.634445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:28:18.634445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:18.634445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:28:18.634445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:18.634445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:28:18.634445 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:28:18.635270 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:28:18.635270 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:28:18.635270 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:28:18.635270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:18.635270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:28:18.635270 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:18.635270 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:28:18.635270 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:28:18.635270 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:18.635270 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:28:18.635270 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:18.635270 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:28:18.635838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:18.635838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:28:18.635838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:28:18.635838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:28:18.635838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:18.635838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:28:18.635838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:18.635838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:28:18.635838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:18.635838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:28:18.635838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:18.635838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:28:18.635838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:18.635838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:28:18.635838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:28:18.636523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:18.636523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:28:18.636523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:18.636523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:28:18.636523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:28:18.636523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:18.636523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:28:18.636523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:18.636523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:28:18.636523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:18.636523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:28:18.636523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:28:18.636523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:28:18.636523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:28:18.637191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:28:18.637191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:28:18.637191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:28:18.637191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:28:18.637191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:18.637191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:28:18.637191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:18.637191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:28:18.637191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:18.637191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:28:18.637191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:18.637191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:28:18.637191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:18.637864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:28:18.637864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:28:18.637864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:18.637864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:28:18.637864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:18.637864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:28:18.637864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:28:18.637864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:18.637864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:28:18.637864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:18.637864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:28:18.637864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:18.638469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:28:18.638469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:28:18.638469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:28:18.638469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:28:18.638469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:28:18.638469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:28:18.638469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:28:18.638469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:28:18.638469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:28:18.638469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:28:18.638469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:28:18.638469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:28:18.638469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:28:18.638469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:28:18.638469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:28:18.638469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:28:18.638469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:28:18.638469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:28:18.638469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:28:18.639327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:18.639327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:28:18.639327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:18.639327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:28:18.639327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:18.639327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:28:18.639327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:18.639327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:28:18.639327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:18.639327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:28:18.639327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:28:18.639327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:18.639327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:28:18.639327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:18.639327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:28:18.639327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:28:18.639327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:18.639327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:28:18.639327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:18.639327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:28:18.639797 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:28:18.639797 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:28:18.639797 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:28:18.639797 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:28:18.639797 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:28:18.639797 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:28:18.639962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:18.639962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:28:18.639962 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:18.639962 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:28:18.639962 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:28:18.639962 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:18.639962 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:28:18.639962 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:18.639962 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:28:18.640711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:18.640711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:28:18.640711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:28:18.640711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:28:18.640711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:28:18.640711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:28:18.640711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:28:18.640711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:18.640711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:28:18.640711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:18.640711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:28:18.640711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:18.640999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:28:18.640999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:18.640999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:28:18.640999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:18.640999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:28:18.640999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:28:18.640999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:18.640999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:28:18.640999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:18.640999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:28:18.640999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:28:18.640999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:18.640999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:28:18.641252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:18.641252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:28:18.641799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:18.641799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:28:18.641799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:28:18.641799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:28:18.641799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:28:18.641799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:28:18.641799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:28:18.641799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:28:18.641799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:28:18.641799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:28:18.641799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:28:18.641799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:28:18.641799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:28:18.641799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:28:18.641799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:28:18.641799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:28:18.642123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:28:18.642123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:28:18.642123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:18.642123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:28:18.642123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:18.642123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:28:18.642123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:18.642123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:28:18.642123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:18.642123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:28:18.642123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:18.642123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:28:18.642123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:28:18.642123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:18.642123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:28:18.642123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:18.642123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:28:18.642123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:28:18.642123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:18.642516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:28:18.642516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:18.642516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:28:23.465185 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:28:23.465726 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:28:23.465726 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:28:23.466520 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:28:23.505858 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:28:23.506272 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:23.507269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:28:23.507269 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:23.507269 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:28:23.507269 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:28:23.507269 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:23.507269 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:28:23.507269 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:23.507269 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:28:23.508261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:23.508261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:28:23.508261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:28:23.508261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:28:23.508261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:28:23.508261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:28:23.508261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:28:23.508261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:23.508261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:28:23.508261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:23.508261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:28:23.508261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:23.508663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:28:23.508663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:23.508663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:28:23.508663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:23.508663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:28:23.508663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:28:23.508663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:23.508663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:28:23.508663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:23.508663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:28:23.508663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:28:23.508663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:23.508663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:28:23.509062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:23.509062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:28:23.509314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:23.509314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:28:23.509314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:28:23.509314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:28:23.509314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:28:23.509314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:28:23.509314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:28:23.509314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:28:23.509314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:28:23.509314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:28:23.509314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:28:23.509314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:28:23.509314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:28:23.509314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:28:23.509314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:28:23.509314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:28:23.510364 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:28:23.510364 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:28:23.510364 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:23.510364 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:28:23.510364 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:23.510364 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:28:23.510364 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:23.510364 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:28:23.510364 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:23.510364 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:28:23.510364 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:23.510364 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:28:23.510364 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:28:23.510364 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:23.510364 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:28:23.510364 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:23.510364 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:28:23.510364 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:28:23.510364 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:23.511295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:28:23.511295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:23.511295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:28:23.617789 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:28:23.618426 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:28:23.618615 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:28:23.619878 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:28:23.620243 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:28:23.620553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:23.621506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:28:23.621506 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:23.621506 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:28:23.621506 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:28:23.621506 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:23.621506 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:28:23.621506 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:23.621506 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:28:23.623103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:23.623103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:28:23.623103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:28:23.623103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:28:23.623103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:28:23.623103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:28:23.623103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:28:23.623103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:23.623103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:28:23.623103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:23.623103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:28:23.623103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:23.624082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:28:23.624082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:23.624082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:28:23.624082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:23.624082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:28:23.624082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:28:23.624082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:23.624082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:28:23.624082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:23.624082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:28:23.624082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:28:23.624082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:23.624082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:28:23.624873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:23.624873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:28:23.625044 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:23.625044 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:28:23.625044 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:28:23.625044 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:28:23.625044 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:28:23.625044 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:28:23.625044 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:28:23.625044 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:28:23.625044 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:28:23.625044 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:28:23.625044 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:28:23.625044 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:28:23.625044 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:28:23.625044 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:28:23.625044 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:28:23.625044 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:28:23.626342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:28:23.626342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:28:23.626342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:23.626342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:28:23.626342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:23.626342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:28:23.626342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:23.626342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:28:23.626342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:23.626342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:28:23.626342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:23.626342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:28:23.626342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:28:23.626342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:23.626342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:28:23.626342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:23.626342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:28:23.626342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:28:23.626342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:23.627583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:28:23.627583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:23.627583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:28:23.627583 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:28:23.627583 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:28:23.627583 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:28:23.627583 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:28:23.627583 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:28:23.628170 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:23.628170 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:28:23.628170 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:23.628170 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:28:23.628170 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:28:23.628170 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:23.628170 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:28:23.628170 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:23.628170 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:28:23.628889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:23.628889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:28:23.628889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:28:23.628889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:28:23.628889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:23.628889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:28:23.628889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:23.628889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:28:23.628889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:23.628889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:28:23.628889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:23.628889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:28:23.628889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:23.628889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:28:23.628889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:28:23.629723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:23.629723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:28:23.629723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:23.629723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:28:23.629723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:28:23.629723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:23.629723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:28:23.629723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:23.629723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:28:23.629723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:23.629723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:28:23.629723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:28:23.629723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:28:23.629723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:28:23.630259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:28:23.630259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:28:23.630259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:28:23.630259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:28:23.630259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:28:23.630259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:23.630259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:28:23.630259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:23.630259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:28:23.630259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:23.630259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:28:23.630259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:23.630259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:28:23.630651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:23.630651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:28:23.630651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:28:23.630651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:23.630651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:28:23.630651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:23.630651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:28:23.630651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:28:23.630651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:23.630651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:28:23.630651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:23.630651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:28:23.631017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:23.631017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:28:23.631017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:28:23.631017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:28:23.631017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:28:23.631017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:28:23.631017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:28:23.631017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:28:23.631017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:28:23.631017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:28:23.631017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:28:23.631017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:28:23.631017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:28:23.631017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:28:23.631017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:28:23.631017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:28:23.631507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:28:23.631507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:28:23.631507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:28:23.631507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:28:23.631507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:28:23.631507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:23.631507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:28:23.631507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:23.631507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:28:23.631507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:23.631507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:28:23.631507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:23.631507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:28:23.631507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:23.631507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:28:23.631507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:28:23.631507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:23.631507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:28:23.632043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:23.632043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:28:23.632043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:28:23.632043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:23.632043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:28:23.632043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:23.632043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:28:23.632989 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:28:23.633310 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:28:23.633426 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:28:23.633559 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:28:23.633976 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:28:23.634460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:23.635086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:28:23.635086 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:23.635086 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:28:23.635086 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:28:23.635086 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:23.635086 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:28:23.635086 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:23.635086 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:28:23.635835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:23.635835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:28:23.635835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:28:23.635835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:28:23.635835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:23.635835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:28:23.635835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:23.635835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:28:23.635835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:23.635835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:28:23.635835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:23.635835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:28:23.635835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:23.635835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:28:23.635835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:28:23.636262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:23.636262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:28:23.636262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:23.636262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:28:23.636262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:28:23.636262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:23.636262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:28:23.636262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:23.636262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:28:23.636702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:23.636702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:28:23.636702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:28:23.636702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:28:23.636702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:28:23.636702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:28:23.636702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:28:23.636702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:28:23.636702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:28:23.636702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:23.636702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:28:23.637133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:23.637133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:28:23.637133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:23.637133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:28:23.637133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:23.637133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:28:23.637133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:23.637133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:28:23.637133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:28:23.637133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:23.637133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:28:23.637133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:23.637133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:28:23.637133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:28:23.637511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:23.637511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:28:23.637511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:23.637511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:28:23.637765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:23.637765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:28:23.637765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:28:23.637765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:28:23.637765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:28:23.637765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:28:23.637765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:28:23.637765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:28:23.637765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:28:23.637765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:28:23.637765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:28:23.637765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:28:23.637765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:28:23.637765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:28:23.637765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:28:23.637765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:28:23.637765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:28:23.638273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:28:23.638273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:28:23.638273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:28:23.638273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:23.638273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:28:23.638273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:23.638273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:28:23.638273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:23.638273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:28:23.638273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:23.638273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:28:23.638273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:23.638273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:28:23.638273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:28:23.638273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:23.638273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:28:23.638273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:23.638273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:28:23.638273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:28:23.638805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:23.638805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:28:23.638805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:23.638805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:28:23.638805 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:28:23.638805 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:28:23.638805 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:28:23.638805 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:28:23.639088 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:28:23.639146 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:28:23.639311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:23.639831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:28:23.639831 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:23.639831 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:28:23.639831 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:28:23.639831 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:23.639831 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:28:23.639831 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:23.639831 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:28:23.640621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:23.640621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:28:23.640621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:28:23.640621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:28:23.640621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:28:23.640621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:28:23.640621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:28:23.640621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:23.640621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:28:23.640621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:23.640621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:28:23.640621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:23.641009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:28:23.641009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:23.641009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:28:23.641009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:23.641009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:28:23.641009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:28:23.641009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:23.641009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:28:23.641009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:23.641009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:28:23.641009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:28:23.641009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:23.641009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:28:23.641357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:23.641357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:28:23.641570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:23.641570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:28:23.641570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:28:23.641570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:28:23.641570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:28:23.641570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:28:23.641570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:28:23.641570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:28:23.641570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:28:23.641570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:28:23.641570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:28:23.641570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:28:23.641570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:28:23.641570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:28:23.641570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:28:23.641570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:28:23.642051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:28:23.642051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:28:23.642051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:23.642051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:28:23.642051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:23.642051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:28:23.642051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:23.642051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:28:23.642051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:23.642051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:28:23.642051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:23.642051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:28:23.642051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:28:23.642051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:23.642051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:28:23.642051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:23.642051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:28:23.642051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:28:23.642051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:23.642566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:28:23.642566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:23.642566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:28:28.466021 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:28:28.466987 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:28:28.466987 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:28:28.467676 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:28:28.499162 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:28:28.499419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:28.499773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:28:28.499773 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:28.499773 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:28:28.499773 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:28:28.499773 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:28.499773 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:28:28.499773 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:28.499773 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:28:28.500739 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.033 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:28.500739 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:28:28.500739 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:28:28.500739 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:28:28.500739 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:28:28.500739 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:28:28.500739 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:28:28.500739 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:28.500739 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:28:28.500739 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:28.500739 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:28:28.500739 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:28.501438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:28:28.501438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:28.501438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:28:28.501438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:28.501438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:28:28.501438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:28:28.501438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:28.501438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:28:28.501438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:28.501438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:28:28.501438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:28:28.501438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:28.501438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:28:28.502136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:28.502136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:28:28.502136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:28.502136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:28:28.502136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:28:28.502136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:28:28.502136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:28:28.502136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:28:28.502136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:28:28.502136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:28:28.502136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:28:28.502136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:28:28.502136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:28:28.502136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:28:28.502136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:28:28.502973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:28:28.502973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:28:28.502973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:28:28.502973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:28:28.502973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:28:28.502973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:28.502973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:28:28.502973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:28.502973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:28:28.502973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:28.502973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:28:28.502973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:28.502973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:28:28.502973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:28.502973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:28:28.502973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:28:28.502973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:28.502973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:28:28.502973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:28.504339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:28:28.504339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:28:28.504339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:28.504339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:28:28.504339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:28.504339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:28:28.618468 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:28:28.619092 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:28:28.619338 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:28:28.620474 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:28:28.620898 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:28:28.621228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:28.622173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:28:28.622173 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:28.622173 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:28:28.622173 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:28:28.622173 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:28.622173 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:28:28.622173 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:28.622173 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:28:28.623474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:28.623474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:28:28.623474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:28:28.623474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:28:28.623474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:28:28.623474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:28:28.623474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:28:28.623474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:28.623474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:28:28.623474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:28.623474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:28:28.623474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:28.623841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:28:28.623841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:28.623841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:28:28.623841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:28.623841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:28:28.623841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:28:28.623841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:28.623841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:28:28.623841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:28.623841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:28:28.623841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:28:28.623841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:28.623841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:28:28.624206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:28.624206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:28:28.624633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:28.624633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:28:28.624633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:28:28.624633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:28:28.624633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:28:28.624633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:28:28.624633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:28:28.624633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:28:28.624633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:28:28.624633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:28:28.624633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:28:28.624633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:28:28.624633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:28:28.624633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:28:28.624633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:28:28.624633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:28:28.625227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:28:28.625227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:28:28.625227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:28.625227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:28:28.625227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:28.625227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:28:28.625227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:28.625227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:28:28.625227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:28.625227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:28:28.625227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:28.625227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:28:28.625227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:28:28.625227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:28.625227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:28:28.625227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:28.625227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:28:28.625227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:28:28.625227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:28.626033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:28:28.626033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:28.626033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:28:28.626353 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:28:28.626782 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:28:28.626970 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:28:28.627083 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:28:28.627520 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:28:28.627807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:28.628347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:28:28.628347 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:28.628347 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:28:28.628347 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:28:28.628347 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:28.628347 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:28:28.628347 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:28.628347 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:28:28.629072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:28.629072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:28:28.629072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:28:28.629072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:28:28.629072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:28.629072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:28:28.629072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:28.629072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:28:28.629072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:28.629072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:28:28.629072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:28.629072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:28:28.629072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:28.629072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:28:28.629072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:28:28.629847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:28.629847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:28:28.629847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:28.629847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:28:28.629847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:28:28.629847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:28.629847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:28:28.629847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:28.629847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:28:28.629847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:28.629847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:28:28.629847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:28:28.629847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:28:28.629847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:28:28.630249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:28:28.630249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:28:28.630249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:28:28.630249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:28:28.630249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:28:28.630249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:28.630249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:28:28.630249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:28.630249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:28:28.630249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:28.630249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:28:28.630249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:28.630249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:28:28.630619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:28.630619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:28:28.630619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:28:28.630619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:28.630619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:28:28.630619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:28.630619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:28:28.630619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:28:28.630619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:28.630619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:28:28.630619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:28.630619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:28:28.631008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:28.631008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:28:28.631008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:28:28.631008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:28:28.631008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:28:28.631008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:28:28.631008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:28:28.631008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:28:28.631008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:28:28.631008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:28:28.631008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:28:28.631008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:28:28.631008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:28:28.631008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:28:28.631008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:28:28.631008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:28:28.631458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:28:28.631458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:28:28.631458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:28:28.631458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:28:28.631458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:28:28.631458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:28.631458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:28:28.631458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:28.631458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:28:28.631458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:28.631458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:28:28.631458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:28.631458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:28:28.631458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:28.631458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:28:28.631458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:28:28.631458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:28.631458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:28:28.631924 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:28.631924 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:28:28.631924 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:28:28.631924 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:28.631924 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:28:28.631924 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:28.631924 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:28:28.633410 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:28:28.633734 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:28:28.633819 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:28:28.633935 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:28:28.634249 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:28:28.634539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:28.635092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:28:28.635092 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:28.635092 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:28:28.635092 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:28:28.635092 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:28.635092 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:28:28.635092 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:28.635092 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:28:28.635844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:28.635844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:28:28.635844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:28:28.635844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:28:28.635844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:28.635844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:28:28.635844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:28.635844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:28:28.635844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:28.635844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:28:28.635844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:28.635844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:28:28.635844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:28.635844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:28:28.635844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:28:28.636329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:28.636329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:28:28.636329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:28.636329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:28:28.636329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:28:28.636329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:28.636329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:28:28.636329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:28.636329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:28:28.636767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:28.636767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:28:28.636767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:28:28.636767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:28:28.636767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:28:28.636767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:28:28.636767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:28:28.636767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:28:28.636767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:28:28.636767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:28.636767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:28:28.637123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:28.637123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:28:28.637123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:28.637123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:28:28.637123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:28.637123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:28:28.637123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:28.637123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:28:28.637123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:28:28.637123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:28.637123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:28:28.637123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:28.637123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:28:28.637123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:28:28.637504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:28.637504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:28:28.637504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:28.637504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:28:28.637817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:28.637817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:28:28.637817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:28:28.637817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:28:28.637817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:28:28.637817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:28:28.637817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:28:28.637817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:28:28.637817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:28:28.637817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:28:28.637817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:28:28.637817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:28:28.637817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:28:28.637817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:28:28.637817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:28:28.637817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:28:28.637817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:28:28.638294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:28:28.638294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:28:28.638294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:28:28.638294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:28.638294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:28:28.638294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:28.638294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:28:28.638294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:28.638294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:28:28.638294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:28.638294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:28:28.638294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:28.638294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:28:28.638294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:28:28.638294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:28.638294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:28:28.638294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:28.638294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:28:28.638294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:28:28.638854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:28.638854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:28:28.638854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:28.638854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:28:28.638854 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:28:28.638854 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:28:28.638854 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:28:28.638854 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:28:28.639143 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:28:28.639426 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:28:28.639609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:28.640226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:28:28.640226 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:28.640226 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:28:28.640226 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:28:28.640226 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:28.640226 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:28:28.640226 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:28.640226 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:28:28.641064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:28.641064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:28:28.641064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:28:28.641064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:28:28.641064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:28:28.641064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:28:28.641064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:28:28.641064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:28.641064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:28:28.641064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:28.641064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:28:28.641064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:28.641514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:28:28.641514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:28.641514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:28:28.641514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:28.641514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:28:28.641514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:28:28.641514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:28.641514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:28:28.641514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:28.641514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:28:28.641514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:28:28.641514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:28.641514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:28:28.642014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:28.642014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:28:28.642014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:28.642014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:28:28.642014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:28:28.642014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:28:28.642014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:28:28.642014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:28:28.642014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:28:28.642014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:28:28.642014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:28:28.642014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:28:28.642014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:28:28.642014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:28:28.642014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:28:28.642537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:28:28.642537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:28:28.642537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:28:28.642537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:28:28.642537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:28:28.642537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:28.642537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:28:28.642537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:28.642537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:28:28.642537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:28.642537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:28:28.642537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:28.642537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:28:28.642537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:28.642537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:28:28.642537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:28:28.642537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:28.642537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:28:28.642537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:28.643045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:28:28.643045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:28:28.643045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:28.643045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:28:28.643045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:28.643045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:28:33.467162 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:28:33.467708 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:28:33.467872 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:28:33.468718 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:28:33.505288 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:28:33.505707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:33.506666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:28:33.506666 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:33.506666 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:28:33.506666 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:28:33.506666 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:33.506666 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:28:33.506666 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:33.506666 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:28:33.507465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:33.507465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:28:33.507465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:28:33.507465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:28:33.507465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:28:33.507465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:28:33.507465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:28:33.507465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:33.507465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:28:33.507465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:33.507465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:28:33.507465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:33.508155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:28:33.508155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:33.508155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:28:33.508155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:33.508155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:28:33.508155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:28:33.508155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:33.508155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:28:33.508155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:33.508155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:28:33.508155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:28:33.508155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:33.508155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:28:33.508768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:33.508768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:28:33.508768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:33.508768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:28:33.508768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:28:33.508768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:28:33.508768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:28:33.508768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:28:33.508768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:28:33.508768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:28:33.508768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:28:33.508768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:28:33.508768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:28:33.508768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:28:33.508768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:28:33.509614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:28:33.509614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:28:33.509614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:28:33.509614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:28:33.509614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:28:33.509614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:33.509614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:28:33.509614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:33.509614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:28:33.509614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:33.509614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:28:33.509614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:33.509614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:28:33.509614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:33.509614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:28:33.509614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:28:33.509614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:33.509614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:28:33.509614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:33.510840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:28:33.510840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:28:33.510840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:33.510840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:28:33.510840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:33.510840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:28:33.619308 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:28:33.619857 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:28:33.620043 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:28:33.621241 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:28:33.621609 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:28:33.621953 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:33.622688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:28:33.622688 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:33.622688 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:28:33.622688 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:28:33.622688 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:33.622688 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:28:33.622688 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:33.622688 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:28:33.624275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:33.624275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:28:33.624275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:28:33.624275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:28:33.624275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:28:33.624275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:28:33.624275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:28:33.624275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:33.624275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:28:33.624275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:33.624275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:28:33.624275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:33.624832 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:28:33.624832 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:33.624832 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:28:33.624832 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:33.624832 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:28:33.624832 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:28:33.624832 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:33.624832 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:28:33.624832 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:33.624832 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:28:33.624832 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:28:33.624832 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:33.624832 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:28:33.625365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:33.625365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:28:33.626198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:33.626198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:28:33.626198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:28:33.626198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:28:33.626198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:28:33.626198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:28:33.626198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:28:33.626198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:28:33.626198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:28:33.626198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:28:33.626198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:28:33.626198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:28:33.626198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:28:33.626198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:28:33.626198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:28:33.626198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:28:33.627356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:28:33.627356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:28:33.627356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:33.627356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:28:33.627356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:33.627356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:28:33.627356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:33.627356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:28:33.627356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:33.627356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:28:33.627356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:33.627356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:28:33.627356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:28:33.627356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:33.627356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:28:33.627356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:33.627356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:28:33.627356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:28:33.627356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:33.628454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:28:33.628454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:33.628454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:28:33.628454 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:28:33.628454 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:28:33.628454 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:28:33.628454 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:28:33.628454 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:28:33.628962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:33.628962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:28:33.628962 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:33.628962 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:28:33.628962 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:28:33.628962 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:33.628962 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:28:33.628962 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:33.628962 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:28:33.629452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:33.629452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:28:33.629452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:28:33.629452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:28:33.629452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:33.629452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:28:33.629452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:33.629452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:28:33.629452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:33.629452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:28:33.629452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:33.629452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:28:33.629452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:33.629452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:28:33.629452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:28:33.630390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:33.630390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:28:33.630390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:33.630390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:28:33.630390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:28:33.630390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:33.630390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:28:33.630390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:33.630390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:28:33.630390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:33.630390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:28:33.630390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:28:33.630390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:28:33.630390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:28:33.631132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:28:33.631132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:28:33.631132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:28:33.631132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:28:33.631132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:28:33.631132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:33.631132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:28:33.631132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:33.631132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:28:33.631132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:33.631132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:28:33.631132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:33.631132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:28:33.631765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:33.631765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:28:33.631765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:28:33.631765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:33.631765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:28:33.631765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:33.631765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:28:33.631765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:28:33.631765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:33.631765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:28:33.631765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:33.631765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:28:33.632413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:33.632413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:28:33.632413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:28:33.632413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:28:33.632413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:28:33.632413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:28:33.632413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:28:33.632413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:28:33.632413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:28:33.632413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:28:33.632413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:28:33.632413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:28:33.632413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:28:33.632413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:28:33.632413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:28:33.632413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:28:33.633271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:28:33.633271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:28:33.633271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:28:33.633271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:28:33.633271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:28:33.633271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:33.633271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:28:33.633271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:33.633271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:28:33.633271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:33.633271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:28:33.633271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:33.633271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:28:33.633271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:33.633271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:28:33.633271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:28:33.633271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:33.633271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:28:33.634397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:33.634397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:28:33.634397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:28:33.634397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:33.634397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:28:33.634397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:33.634397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:28:33.635004 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:28:33.635004 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:28:33.635004 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:28:33.635194 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:28:33.635623 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:28:33.636055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:33.636834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:28:33.636834 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:33.636834 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:28:33.636834 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:28:33.636834 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:33.636834 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:28:33.636834 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:33.636834 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:28:33.637650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:33.637650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:28:33.637650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:28:33.637650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:28:33.637650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:33.637650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:28:33.637650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:33.637650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:28:33.637650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:33.637650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:28:33.637650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:33.637650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:28:33.637650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:33.637650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:28:33.637650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:28:33.638250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:33.638250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:28:33.638250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:33.638250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:28:33.638250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:28:33.638250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:33.638250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:28:33.638250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:33.638250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:28:33.638702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:33.638702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:28:33.638702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:28:33.638702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:28:33.638702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:28:33.638702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:28:33.638702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:28:33.638702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:28:33.638702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:28:33.638702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:33.638702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:28:33.639054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:33.639054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:28:33.639054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:33.639054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:28:33.639054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:33.639054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:28:33.639054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:33.639054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:28:33.639054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:28:33.639054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:33.639054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:28:33.639054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:33.639054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:28:33.639054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:28:33.639446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:33.639446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:28:33.639446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:33.639446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:28:33.639941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:33.639941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:28:33.639941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:28:33.639941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:28:33.639941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:28:33.639941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:28:33.639941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:28:33.639941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:28:33.639941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:28:33.639941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:28:33.639941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:28:33.639941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:28:33.639941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:28:33.639941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:28:33.639941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:28:33.639941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:28:33.639941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:28:33.640397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:28:33.640397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:28:33.640397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:28:33.640397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:33.640397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:28:33.640397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:33.640397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:28:33.640397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:33.640397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:28:33.640397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:33.640397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:28:33.640397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:33.640397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:28:33.640397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:28:33.640397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:33.640397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:28:33.640397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:33.640397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:28:33.640397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:28:33.640897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:33.640897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:28:33.640897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:33.640897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:28:33.640897 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:28:33.640897 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:28:33.640897 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:28:33.640897 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:28:33.641167 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:28:33.641397 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:28:33.641537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:33.642113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:28:33.642113 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:33.642113 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:28:33.642113 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:28:33.642113 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:33.642113 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:28:33.642113 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:33.642113 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:28:33.643033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:33.643033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:28:33.643033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:28:33.643033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:28:33.643033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:28:33.643033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:28:33.643033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:28:33.643033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:33.643033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:28:33.643033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:33.643033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:28:33.643033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:33.643317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:28:33.643317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:33.643317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:28:33.643317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:33.643317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:28:33.643317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:28:33.643317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:33.643317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:28:33.643317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:33.643317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:28:33.643317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:28:33.643317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:33.643317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:28:33.643564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:33.643564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:28:33.644079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:33.644079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:28:33.644079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:28:33.644079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:28:33.644079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:28:33.644079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:28:33.644079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:28:33.644079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:28:33.644079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:28:33.644079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:28:33.644079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:28:33.644079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:28:33.644079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:28:33.644079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:28:33.644079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:28:33.644079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:28:33.644395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:28:33.644395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:28:33.644395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:33.644395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:28:33.644395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:33.644395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:28:33.644395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:33.644395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:28:33.644395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:33.644395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:28:33.644395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:33.644395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:28:33.644395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:28:33.644395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:33.644395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:28:33.644395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:33.644395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:28:33.644395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:28:33.644395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:33.644805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:28:33.644805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:33.644805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:28:38.468208 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:28:38.468766 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:28:38.468824 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:28:38.469761 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:28:38.503033 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:28:38.503465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:38.504492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:28:38.504492 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:38.504492 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:28:38.504492 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:28:38.504492 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:38.504492 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:28:38.504492 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:38.504492 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:28:38.505532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:38.505532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:28:38.505532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:28:38.505532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:28:38.505532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:28:38.505532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:28:38.505532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:28:38.505532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:38.505532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:28:38.505532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:38.505532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:28:38.505532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:38.505905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:28:38.505905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:38.505905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:28:38.505905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:38.505905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:28:38.505905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:28:38.505905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:38.505905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:28:38.505905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:38.505905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:28:38.505905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:28:38.505905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:38.505905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:28:38.506284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:38.506284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:28:38.506583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:38.506583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:28:38.506583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:28:38.506583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:28:38.506583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:28:38.506583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:28:38.506583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:28:38.506583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:28:38.506583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:28:38.506583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:28:38.506583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:28:38.506583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:28:38.506583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:28:38.506583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:28:38.506583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:28:38.506583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:28:38.507067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:28:38.507067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:28:38.507067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:38.507067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:28:38.507067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:38.507067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:28:38.507067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:38.507067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:28:38.507067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:38.507067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:28:38.507067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:38.507067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:28:38.507067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:28:38.507067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:38.507067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:28:38.507067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:38.507067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:28:38.507067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:28:38.507067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:38.507564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:28:38.507564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:38.507564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:28:38.619903 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:28:38.620468 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:28:38.620680 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:28:38.621730 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:28:38.622206 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:28:38.622644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:38.623646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:28:38.623646 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:38.623646 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:28:38.623646 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:28:38.623646 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:38.623646 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:28:38.623646 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:38.623646 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:28:38.625304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:38.625304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:28:38.625304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:28:38.625304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:28:38.625304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:28:38.625304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:28:38.625304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:28:38.625304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:38.625304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:28:38.625304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:38.625304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:28:38.625304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:38.625935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:28:38.625935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:38.625935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:28:38.625935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:38.625935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:28:38.625935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:28:38.625935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:38.625935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:28:38.625935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:38.625935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:28:38.625935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:28:38.625935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:38.625935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:28:38.626495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:38.626495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:28:38.629029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:38.629029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:28:38.629029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:28:38.629029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:28:38.629029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:28:38.629029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:28:38.629029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:28:38.629029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:28:38.629029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:28:38.629029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:28:38.629029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:28:38.629029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:28:38.629029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:28:38.629029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:28:38.629029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:28:38.629029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:28:38.629811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:28:38.629811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:28:38.629811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:38.629811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:28:38.629811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:38.629811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:28:38.629811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:38.629811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:28:38.629811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:38.629811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:28:38.629811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:38.629811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:28:38.629811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:28:38.629811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:38.629811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:28:38.629811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:38.629811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:28:38.629811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:28:38.629811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:38.630613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:28:38.630613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:38.630613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:28:38.630613 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:28:38.630613 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:28:38.630613 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:28:38.630960 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:28:38.631264 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:28:38.631595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:38.632498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:28:38.632498 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:38.632498 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:28:38.632498 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:28:38.632498 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:38.632498 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:28:38.632498 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:38.632498 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:28:38.633258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:38.633258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:28:38.633258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:28:38.633258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:28:38.633258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:38.633258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:28:38.633258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:38.633258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:28:38.633258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:38.633258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:28:38.633258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:38.633258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:28:38.633258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:38.633258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:28:38.633258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:28:38.633689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:38.633689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:28:38.633689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:38.633689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:28:38.633689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:28:38.633689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:38.633689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:28:38.633689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:38.633689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:28:38.634151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:38.634151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:28:38.634151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:28:38.634151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:28:38.634151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:28:38.634151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:28:38.634151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:28:38.634151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:28:38.634151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:28:38.634151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:28:38.634151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:38.634151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:28:38.634513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:38.634513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:28:38.634513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:38.634513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:28:38.634513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:38.634513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:28:38.634513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:38.634513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:28:38.634513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:28:38.634513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:38.634513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:28:38.634513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:38.634513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:28:38.634513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:28:38.634935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:38.634935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:28:38.634935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:38.634935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:28:38.635178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:38.635178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:28:38.635178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:28:38.635178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:28:38.635178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:28:38.635178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:28:38.635178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:28:38.635178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:28:38.635178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:28:38.635178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:28:38.635178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:28:38.635178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:28:38.635178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:28:38.635178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:28:38.635178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:28:38.635178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:28:38.635638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:28:38.635638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:28:38.635638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:28:38.635638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:28:38.635638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:28:38.635638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:38.635638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:28:38.635638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:38.635638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:28:38.635638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:38.635638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:28:38.635638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:38.635638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:28:38.635638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:38.635638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:28:38.635638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:28:38.635638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:38.635638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:28:38.636244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:38.636244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:28:38.636244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:28:38.636244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:38.636244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:28:38.636244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:38.636244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:28:38.636244 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:28:38.636244 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:28:38.636244 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:28:38.636556 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:28:38.636556 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:28:38.636638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:38.637348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:28:38.637348 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:38.637348 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:28:38.637348 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:28:38.637348 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:38.637348 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:28:38.637348 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:38.637348 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:28:38.638187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:38.638187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:28:38.638187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:28:38.638187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:28:38.638187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:38.638187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:28:38.638187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:38.638187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:28:38.638187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:38.638187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:28:38.638187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:38.638187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:28:38.638187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:38.638187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:28:38.638187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:28:38.638602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:38.638602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:28:38.638602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:38.638602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:28:38.638602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:28:38.638602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:38.638602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:28:38.638602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:38.638602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:28:38.639082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:38.639082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:28:38.639082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:28:38.639082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:28:38.639082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:28:38.639082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:28:38.639082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:28:38.639082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:28:38.639082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:28:38.639082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:38.639082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:28:38.639389 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:38.639389 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:28:38.639389 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:38.639389 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:28:38.639389 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:38.639389 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:28:38.639389 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:38.639389 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:28:38.639389 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:28:38.639389 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:38.639389 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:28:38.639389 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:38.639389 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:28:38.639389 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:28:38.639819 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:38.639819 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:28:38.639819 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:38.639819 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:28:38.640221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:38.640221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:28:38.640221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:28:38.640221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:28:38.640221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:28:38.640221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:28:38.640221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:28:38.640221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:28:38.640221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:28:38.640221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:28:38.640221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:28:38.640221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:28:38.640221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:28:38.640221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:28:38.640221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:28:38.640221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:28:38.640221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:28:38.640836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:28:38.640836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:28:38.640836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:28:38.640836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:38.640836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:28:38.640836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:38.640836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:28:38.640836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:38.640836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:28:38.640836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:38.640836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:28:38.640836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:38.640836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:28:38.640836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:28:38.640836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:38.640836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:28:38.640836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:38.640836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:28:38.640836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:28:38.641350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:38.641350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:28:38.641350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:38.641350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:28:38.641350 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:28:38.641350 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:28:38.641350 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:28:38.641350 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:28:38.641597 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:28:38.641871 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:28:38.642063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:38.642640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:28:38.642640 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:38.642640 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:28:38.642640 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:28:38.642640 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:38.642640 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:28:38.642640 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:38.642640 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:28:38.643595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:38.643595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:28:38.643595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:28:38.643595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:28:38.643595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:28:38.643595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:28:38.643595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:28:38.643595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:38.643595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:28:38.643595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:38.643595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:28:38.643595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:38.643979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:28:38.643979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:38.643979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:28:38.643979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:38.643979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:28:38.643979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:28:38.643979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:38.643979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:28:38.643979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:38.643979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:28:38.643979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:28:38.643979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:38.643979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:28:38.644334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:38.644334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:28:38.644559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:38.644559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:28:38.644559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:28:38.644559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:28:38.644559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:28:38.644559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:28:38.644559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:28:38.644559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:28:38.644559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:28:38.644559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:28:38.644559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:28:38.644559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:28:38.644559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:28:38.644559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:28:38.644559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:28:38.644559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:28:38.645020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:28:38.645020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:28:38.645020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:38.645020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:28:38.645020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:38.645020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:28:38.645020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:38.645020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:28:38.645020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:38.645020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:28:38.645020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:38.645020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:28:38.645020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:28:38.645020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:38.645020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:28:38.645020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:38.645020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:28:38.645020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:28:38.645020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:38.645522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:28:38.645522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:38.645522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:28:43.468887 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:28:43.469911 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:28:43.469911 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:28:43.470268 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:28:43.505264 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:28:43.505697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:43.506657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:28:43.506657 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:43.506657 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:28:43.506657 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:28:43.506657 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:43.506657 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:28:43.506657 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:43.506657 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:28:43.508049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:43.508049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:28:43.508049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:28:43.508049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:28:43.508049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:28:43.508049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:28:43.508049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:28:43.508049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:43.508049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:28:43.508049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:43.508049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:28:43.508049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:43.508828 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:28:43.508828 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:43.508828 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:28:43.508828 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:43.508828 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:28:43.508828 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:28:43.508828 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:43.508828 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:28:43.508828 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:43.508828 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:28:43.508828 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:28:43.508828 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:43.508828 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:28:43.509420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:43.509420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:28:43.509420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:43.509420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:28:43.509420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:28:43.509420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:28:43.509420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:28:43.509420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:28:43.509420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:28:43.509420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:28:43.509420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:28:43.509420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:28:43.509420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:28:43.509420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:28:43.509420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:28:43.510214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:28:43.510214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:28:43.510214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:28:43.510214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:28:43.510214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:28:43.510214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:43.510214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:28:43.510214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:43.510214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:28:43.510214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:43.510214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:28:43.510214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:43.510214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:28:43.510214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:43.510214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:28:43.510214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:28:43.510214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:43.510214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:28:43.510214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:43.511555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:28:43.511555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:28:43.511555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:43.511555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:28:43.511555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:43.511555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:28:43.620372 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:28:43.620933 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:28:43.621163 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:28:43.622019 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:28:43.622316 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:28:43.622509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:43.623334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:28:43.623334 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:43.623334 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:28:43.623334 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:28:43.623334 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:43.623334 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:28:43.623334 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:43.623334 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:28:43.624381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:43.624381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:28:43.624381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:28:43.624381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:28:43.624381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:28:43.624381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:28:43.624381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:28:43.624381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:43.624381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:28:43.624381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:43.624381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:28:43.624381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:43.625354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:28:43.625354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:43.625354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:28:43.625354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:43.625354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:28:43.625354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:28:43.625354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:43.625354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:28:43.625354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:43.625354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:28:43.625354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:28:43.625354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:43.625354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:28:43.626474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:43.626474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:28:43.626474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:43.626474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:28:43.626474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:28:43.626474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:28:43.626474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:28:43.626474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:28:43.626474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:28:43.626474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:28:43.626474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:28:43.626474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:28:43.626474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:28:43.626474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:28:43.626474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:28:43.627915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:28:43.627915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:28:43.627915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:28:43.627915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:28:43.627915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:28:43.627915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:43.627915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:28:43.627915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:43.627915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:28:43.627915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:43.627915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:28:43.627915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:43.627915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:28:43.627915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:43.627915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:28:43.627915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:28:43.627915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:43.627915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:28:43.627915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:43.628869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:28:43.628869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:28:43.628869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:43.628869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:28:43.628869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:43.628869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:28:43.630631 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:28:43.631059 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:28:43.631220 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:28:43.631379 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:28:43.631999 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:28:43.632483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:43.633208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:28:43.633208 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:43.633208 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:28:43.633208 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:28:43.633208 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:43.633208 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:28:43.633208 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:43.633208 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:28:43.634036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:43.634036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:28:43.634036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:28:43.634036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:28:43.634036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:43.634036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:28:43.634036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:43.634036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:28:43.634036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:43.634036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:28:43.634036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:43.634036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:28:43.634036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:43.634036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:28:43.634036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:28:43.635710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:43.635710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:28:43.635710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:43.635710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:28:43.635710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:28:43.635710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:43.635710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:28:43.635710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:43.635710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:28:43.635710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:43.635710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:28:43.635710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:28:43.635710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:28:43.635710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:28:43.636969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:28:43.636969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:28:43.636969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:28:43.636969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:28:43.636969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:28:43.636969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:43.636969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:28:43.636969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:43.636969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:28:43.636969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:43.636969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:28:43.636969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:43.636969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:28:43.637808 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:43.637808 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:28:43.637808 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:28:43.637808 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:43.637808 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:28:43.637808 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:43.637808 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:28:43.637808 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:28:43.637808 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:43.637808 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:28:43.637808 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:43.637808 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:28:43.638103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:43.638103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:28:43.638103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:28:43.638103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:28:43.638103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:28:43.638103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:28:43.638103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:28:43.638103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:28:43.638103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:28:43.638103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:28:43.638103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:28:43.638103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:28:43.638103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:28:43.638103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:28:43.638103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:28:43.638103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:28:43.638504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:28:43.638504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:28:43.638504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:28:43.638504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:28:43.638504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:28:43.638504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:43.638504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:28:43.638504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:43.638504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:28:43.638504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:43.638504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:28:43.638504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:43.638504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:28:43.638504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:43.638504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:28:43.638504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:28:43.638504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:43.638504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:28:43.640433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:43.640433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:28:43.640433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:28:43.640433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:43.640433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:28:43.640433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:43.640433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:28:43.640433 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:28:43.640433 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:28:43.640433 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:28:43.641334 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:28:43.641334 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:28:43.641334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:43.641334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:28:43.641334 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:43.641334 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:28:43.641334 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:28:43.641334 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:43.641334 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:28:43.641334 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:43.641334 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:28:43.641334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:43.642301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:28:43.642301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:28:43.642301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:28:43.642301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:43.642301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:28:43.642301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:43.642301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:28:43.642301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:43.642301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:28:43.642301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:43.642301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:28:43.642301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:43.642301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:28:43.642301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:28:43.642301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:43.642301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:28:43.642301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:43.642702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:28:43.642702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:28:43.642702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:43.642702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:28:43.642702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:43.642702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:28:43.642702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:43.642702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:28:43.642702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:28:43.642702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:28:43.642702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:28:43.642702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:28:43.642702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:28:43.643470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:28:43.643470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:28:43.643470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:43.643470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:28:43.643470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:43.643470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:28:43.643470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:43.643470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:28:43.643470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:43.643470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:28:43.643470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:43.643470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:28:43.643470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:28:43.643697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:43.643697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:28:43.643697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:43.643697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:28:43.643697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:28:43.643697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:43.643697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:28:43.643697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:43.643697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:28:43.643697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:43.643697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:28:43.643697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:28:43.643944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:28:43.643944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:28:43.643944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:28:43.643944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:28:43.643944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:28:43.643944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:28:43.643944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:28:43.643944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:28:43.643944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:28:43.643944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:28:43.643944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:28:43.643944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:28:43.643944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:28:43.643944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:28:43.643944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:28:43.643944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:28:43.643944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:28:43.643944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:43.643944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:28:43.644278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:43.644278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:28:43.644278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:43.644278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:28:43.644278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:43.644278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:28:43.644278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:43.644278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:28:43.644278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:28:43.644278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:43.644278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:28:43.644278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:43.644278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:28:43.644278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:28:43.644278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:43.644278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:28:43.644278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:43.644278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:28:43.644637 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:28:43.644637 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:28:43.644637 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:28:43.644637 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:28:43.644637 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:28:43.644784 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:28:43.644938 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:43.646187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:28:43.646187 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:43.646187 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:28:43.646187 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:28:43.646187 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:43.646187 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:28:43.646187 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:43.646187 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:28:43.646799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:43.646799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:28:43.646799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:28:43.646799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:28:43.646799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:28:43.646799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:28:43.646799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:28:43.646799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:43.646799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:28:43.646799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:43.646799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:28:43.646799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:43.647094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:28:43.647094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:43.647094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:28:43.647094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:43.647094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:28:43.647094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:28:43.647094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:43.647094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:28:43.647094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:43.647094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:28:43.647094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:28:43.647094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:43.647094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:28:43.647327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:43.647327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:28:43.648514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:43.648514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:28:43.648514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:28:43.648514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:28:43.648514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:28:43.648514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:28:43.648514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:28:43.648514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:28:43.648514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:28:43.648514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:28:43.648514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:28:43.648514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:28:43.648514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:28:43.648514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:28:43.648514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:28:43.648514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:28:43.648856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:28:43.648856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:28:43.648856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:43.648856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:28:43.648856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:43.648856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:28:43.648856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:43.648856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:28:43.648856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:43.648856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:28:43.648856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:43.648856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:28:43.648856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:28:43.648856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:43.648856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:28:43.648856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:43.648856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:28:43.648856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:28:43.648856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:43.649209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:28:43.649209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:43.649209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:28:48.469404 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:28:48.470101 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:28:48.470510 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:28:48.471541 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:28:48.504178 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:28:48.504530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:48.504964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:28:48.504964 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:48.504964 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:28:48.504964 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:28:48.504964 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:48.504964 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:28:48.504964 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:48.504964 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:28:48.505642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:48.505642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:28:48.505642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:28:48.505642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:28:48.505642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:28:48.505642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:28:48.505642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:28:48.505642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:48.505642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:28:48.505642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:48.505642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:28:48.505642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:48.506328 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:28:48.506328 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:48.506328 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:28:48.506328 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:48.506328 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:28:48.506328 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:28:48.506328 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:48.506328 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:28:48.506328 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:48.506328 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:28:48.506328 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:28:48.506328 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:48.506328 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:28:48.506963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:48.506963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:28:48.506963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:48.506963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:28:48.506963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:28:48.506963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:28:48.506963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:28:48.506963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:28:48.506963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:28:48.506963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:28:48.506963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:28:48.506963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:28:48.506963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:28:48.506963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:28:48.506963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:28:48.507719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:28:48.507719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:28:48.507719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:28:48.507719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:28:48.507719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:28:48.507719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:48.507719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:28:48.507719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:48.507719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:28:48.507719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:48.507719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:28:48.507719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:48.507719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:28:48.507719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:48.507719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:28:48.507719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:28:48.507719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:48.507719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:28:48.507719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:48.509579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:28:48.509579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:28:48.509579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:48.509579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:28:48.509579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:48.509579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:28:48.621048 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:28:48.621736 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:28:48.622035 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:28:48.623249 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:28:48.623583 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:28:48.623874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:48.624832 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:28:48.624832 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:48.624832 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:28:48.624832 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:28:48.624832 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:48.624832 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:28:48.624832 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:48.624832 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:28:48.626586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:48.626586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:28:48.626586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:28:48.626586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:28:48.626586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:28:48.626586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:28:48.626586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:28:48.626586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:48.626586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:28:48.626586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:48.626586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:28:48.626586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:48.627525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:28:48.627525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:48.627525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:28:48.627525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:48.627525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:28:48.627525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:28:48.627525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:48.627525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:28:48.627525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:48.627525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:28:48.627525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:28:48.627525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:48.627525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:28:48.628521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:48.628521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:28:48.628521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:48.628521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:28:48.628521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:28:48.628521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:28:48.628521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:28:48.628521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:28:48.628521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:28:48.628521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:28:48.628521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:28:48.628521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:28:48.628521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:28:48.628521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:28:48.628521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:28:48.629101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:28:48.629101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:28:48.629101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:28:48.629101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:28:48.629101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:28:48.629101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:48.629101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:28:48.629101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:48.629101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:28:48.629101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:48.629101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:28:48.629101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:48.629101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:28:48.629101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:48.629101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:28:48.629101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:28:48.629101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:48.629101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:28:48.629101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:48.629895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:28:48.629895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:28:48.629895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:48.629895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:28:48.629895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:48.629895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:28:48.630983 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:28:48.631254 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:28:48.631395 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:28:48.631516 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:28:48.632056 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:28:48.632378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:48.633104 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:28:48.633104 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:48.633104 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:28:48.633104 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:28:48.633104 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:48.633104 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:28:48.633104 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:48.633104 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:28:48.634011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:48.634011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:28:48.634011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:28:48.634011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:28:48.634011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:48.634011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:28:48.634011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:48.634011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:28:48.634011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:48.634011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:28:48.634011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:48.634011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:28:48.634011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:48.634011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:28:48.634011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:28:48.634651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:48.634651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:28:48.634651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:48.634651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:28:48.634651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:28:48.634651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:48.634651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:28:48.634651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:48.634651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:28:48.635076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:48.635076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:28:48.635076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:28:48.635076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:28:48.635076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:28:48.635076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:28:48.635076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:28:48.635076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:28:48.635076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:28:48.635076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:28:48.635076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:48.635076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:28:48.635543 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:48.635543 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:28:48.635543 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:48.635543 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:28:48.635543 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:48.635543 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:28:48.635543 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:48.635543 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:28:48.635543 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:28:48.635543 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:48.635543 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:28:48.635543 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:48.635543 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:28:48.635543 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:28:48.636094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:48.636094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:28:48.636094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:48.636094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:28:48.636094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:48.636094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:28:48.636094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:28:48.636094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:28:48.636094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:28:48.636094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:28:48.636094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:28:48.636094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:28:48.636094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:28:48.636094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:28:48.636587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:28:48.636587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:28:48.636587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:28:48.636587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:28:48.636587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:28:48.636587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:28:48.636587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:28:48.636587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:28:48.636587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:28:48.636587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:28:48.636587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:28:48.636587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:48.636587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:28:48.636587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:48.636587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:28:48.636587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:48.636587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:28:48.636587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:48.636587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:28:48.637293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:48.637293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:28:48.637293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:28:48.637293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:48.637293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:28:48.637293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:48.637293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:28:48.637293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:28:48.637293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:48.637293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:28:48.637293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:48.637293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:28:48.637293 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:28:48.637735 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:28:48.637735 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:28:48.637845 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:28:48.638299 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:28:48.638708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:48.639439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:28:48.639439 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:48.639439 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:28:48.639439 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:28:48.639439 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:48.639439 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:28:48.639439 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:48.639439 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:28:48.640219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:48.640219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:28:48.640219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:28:48.640219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:28:48.640219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:48.640219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:28:48.640219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:48.640219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:28:48.640219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:48.640219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:28:48.640219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:48.640219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:28:48.640219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:48.640219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:28:48.640219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:28:48.640726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:48.640726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:28:48.640726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:48.640726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:28:48.640726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:28:48.640726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:48.640726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:28:48.640726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:48.640726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:28:48.641079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:48.641079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:28:48.641079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:28:48.641079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:28:48.641079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:28:48.641079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:28:48.641079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:28:48.641079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:28:48.641079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:28:48.641079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:48.641079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:28:48.641479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:48.641479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:28:48.641479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:48.641479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:28:48.641479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:48.641479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:28:48.641479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:48.641479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:28:48.641479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:28:48.641479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:48.641479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:28:48.641479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:48.641479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:28:48.641479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:28:48.641997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:48.641997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:28:48.641997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:48.641997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:28:48.642177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:48.642177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:28:48.642177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:28:48.642177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:28:48.642177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:28:48.642177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:28:48.642177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:28:48.642177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:28:48.642177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:28:48.642177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:28:48.642177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:28:48.642177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:28:48.642177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:28:48.642177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:28:48.642177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:28:48.642177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:28:48.642177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:28:48.642789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:28:48.642789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:28:48.642789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:28:48.642789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:48.642789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:28:48.642789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:48.642789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:28:48.642789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:48.642789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:28:48.642789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:48.642789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:28:48.642789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:48.642789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:28:48.642789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:28:48.642789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:48.642789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:28:48.642789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:48.642789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:28:48.642789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:28:48.643455 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:48.643455 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:28:48.643455 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:48.643455 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:28:48.644056 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:28:48.644345 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:28:48.644457 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:28:48.644609 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:28:48.645069 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:28:48.645416 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:28:48.645638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:48.646335 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:28:48.646335 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:48.646335 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:28:48.646335 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:28:48.646335 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:48.646335 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:28:48.646335 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:48.646335 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:28:48.647140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:48.647140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:28:48.647140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:28:48.647140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:28:48.647140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:28:48.647140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:28:48.647140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:28:48.647140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:48.647140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:28:48.647140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:48.647140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:28:48.647140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:48.647631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:28:48.647631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:48.647631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:28:48.647631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:48.647631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:28:48.647631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:28:48.647631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:48.647631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:28:48.647631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:48.647631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:28:48.647631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:28:48.647631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:48.647631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:28:48.648158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:48.648158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:28:48.648244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:48.648244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:28:48.648244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:28:48.648244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:28:48.648244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:28:48.648244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:28:48.648244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:28:48.648244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:28:48.648244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:28:48.648244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:28:48.648244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:28:48.648244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:28:48.648244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:28:48.648244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:28:48.648244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:28:48.648244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:28:48.648848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:28:48.648848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:28:48.648848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:48.648848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:28:48.648848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:48.648848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:28:48.648848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:48.648848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:28:48.648848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:48.648848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:28:48.648848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:48.648848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:28:48.648848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:28:48.648848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:48.648848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:28:48.648848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:48.648848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:28:48.648848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:28:48.648848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:48.649531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:28:48.649531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:48.649531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:28:53.470143 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:28:53.470846 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:28:53.471008 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:28:53.472846 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:28:53.505541 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:28:53.506013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:53.506615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:28:53.506615 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:53.506615 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:28:53.506615 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:28:53.506615 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:53.506615 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:28:53.506615 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:53.506615 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:28:53.507655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:53.507655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:28:53.507655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:28:53.507655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:28:53.507655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:28:53.507655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:28:53.507655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:28:53.507655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:53.507655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:28:53.507655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:53.507655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:28:53.507655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:53.508337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:28:53.508337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:53.508337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:28:53.508337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:53.508337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:28:53.508337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:28:53.508337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:53.508337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:28:53.508337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:53.508337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:28:53.508337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:28:53.508337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:53.508337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:28:53.509069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:53.509069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:28:53.509069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:53.509069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:28:53.509069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:28:53.509069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:28:53.509069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:28:53.509069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:28:53.509069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:28:53.509069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:28:53.509069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:28:53.509069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:28:53.509069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:28:53.509069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:28:53.509069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:28:53.509950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:28:53.509950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:28:53.509950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:28:53.509950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:28:53.509950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:28:53.509950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:53.509950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:28:53.509950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:53.509950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:28:53.509950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:53.509950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:28:53.509950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:53.509950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:28:53.509950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:53.509950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:28:53.509950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:28:53.509950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:53.509950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:28:53.509950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:53.511269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:28:53.511269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:28:53.511269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:53.511269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:28:53.511269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:53.511269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:28:53.621624 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:28:53.622288 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:28:53.622493 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:28:53.623501 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:28:53.623822 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:28:53.624066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:53.624977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:28:53.624977 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:53.624977 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:28:53.624977 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:28:53.624977 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:53.624977 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:28:53.624977 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:53.624977 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:28:53.626268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:53.626268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:28:53.626268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:28:53.626268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:28:53.626268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:28:53.626268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:28:53.626268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:28:53.626268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:53.626268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:28:53.626268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:53.626268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:28:53.626268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:53.626679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:28:53.626679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:53.626679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:28:53.626679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:53.626679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:28:53.626679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:28:53.626679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:53.626679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:28:53.626679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:53.626679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:28:53.626679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:28:53.626679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:53.626679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:28:53.627328 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:53.627328 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:28:53.627328 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:53.627328 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:28:53.627328 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:28:53.627328 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:28:53.627328 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:28:53.627328 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:28:53.627328 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:28:53.627328 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:28:53.627328 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:28:53.627328 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:28:53.627328 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:28:53.627328 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:28:53.627328 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:28:53.628264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:28:53.628264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:28:53.628264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:28:53.628264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:28:53.628264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:28:53.628264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:53.628264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:28:53.628264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:53.628264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:28:53.628264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:53.628264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:28:53.628264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:53.628264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:28:53.628264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:53.628264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:28:53.628264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:28:53.628264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:53.628264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:28:53.628264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:53.629245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:28:53.629245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:28:53.629245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:53.629245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:28:53.629245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:53.629245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:28:53.631774 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:28:53.631954 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:28:53.632131 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:28:53.632296 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:28:53.632637 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:28:53.632852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:53.633390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:28:53.633390 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:53.633390 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:28:53.633390 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:28:53.633390 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:53.633390 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:28:53.633390 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:53.633390 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:28:53.634192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:53.634192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:28:53.634192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:28:53.634192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:28:53.634192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:53.634192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:28:53.634192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:53.634192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:28:53.634192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:53.634192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:28:53.634192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:53.634192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:28:53.634192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:53.634192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:28:53.634192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:28:53.635004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:53.635004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:28:53.635004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:53.635004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:28:53.635004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:28:53.635004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:53.635004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:28:53.635004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:53.635004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:28:53.635432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:53.635432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:28:53.635432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:28:53.635432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:28:53.635432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:28:53.635432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:28:53.635432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:28:53.635432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:28:53.635432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:28:53.635432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:28:53.635432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:53.635432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:28:53.635960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:53.635960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:28:53.635960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:53.635960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:28:53.635960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:53.635960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:28:53.635960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:53.635960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:28:53.635960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:28:53.635960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:53.635960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:28:53.635960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:53.635960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:28:53.635960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:28:53.636428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:53.636428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:28:53.636428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:53.636428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:28:53.636428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:53.636428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:28:53.636428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:28:53.636428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:28:53.636428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:28:53.636428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:28:53.636428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:28:53.636428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:28:53.636428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:28:53.636428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:28:53.636864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:28:53.636864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:28:53.636864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:28:53.636864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:28:53.636864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:28:53.636864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:28:53.636864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:28:53.636864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:28:53.636864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:28:53.636864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:28:53.636864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:28:53.636864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:53.636864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:28:53.636864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:53.636864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:28:53.636864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:53.636864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:28:53.636864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:53.636864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:28:53.637436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:53.637436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:28:53.637436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:28:53.637436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:53.637436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:28:53.637436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:53.637436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:28:53.637436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:28:53.637436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:53.637436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:28:53.637436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:53.637436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:28:53.637938 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:28:53.638137 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:28:53.638247 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:28:53.638377 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:28:53.638679 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:28:53.638935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:53.639394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:28:53.639394 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:53.639394 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:28:53.639394 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:28:53.639394 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:53.639394 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:28:53.639394 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:53.639394 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:28:53.640132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:53.640132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:28:53.640132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:28:53.640132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:28:53.640132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:53.640132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:28:53.640132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:53.640132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:28:53.640132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:53.640132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:28:53.640132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:53.640132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:28:53.640132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:53.640132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:28:53.640132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:28:53.640580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:53.640580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:28:53.640580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:53.640580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:28:53.640580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:28:53.640580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:53.640580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:28:53.640580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:53.640580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:28:53.641322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:53.641322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:28:53.641322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:28:53.641322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:28:53.641322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:28:53.641322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:28:53.641322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:28:53.641322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:28:53.641322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:28:53.641322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:53.641322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:28:53.641737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:53.641737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:28:53.641737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:53.641737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:28:53.641737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:53.641737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:28:53.641737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:53.641737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:28:53.641737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:28:53.641737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:53.641737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:28:53.641737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:53.641737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:28:53.641737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:28:53.642322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:53.642322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:28:53.642322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:53.642322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:28:53.642322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:53.642322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:28:53.642322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:28:53.642322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:28:53.642322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:28:53.642322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:28:53.642322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:28:53.642322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:28:53.642322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:28:53.642322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:28:53.642946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:28:53.642946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:28:53.642946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:28:53.642946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:28:53.642946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:28:53.642946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:28:53.642946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:28:53.642946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:28:53.642946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:28:53.642946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:28:53.642946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:53.642946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:28:53.642946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:53.642946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:28:53.642946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:53.642946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:28:53.642946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:53.642946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:28:53.642946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:53.643772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:28:53.643772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:28:53.643772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:53.643772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:28:53.643772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:53.643772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:28:53.643772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:28:53.643772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:53.643772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:28:53.643772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:53.643772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:28:53.644564 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:28:53.644907 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:28:53.645051 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:28:53.645225 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:28:53.645525 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:28:53.645835 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:28:53.646013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:53.646467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:28:53.646467 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:53.646467 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:28:53.646467 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:28:53.646467 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:53.646467 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:28:53.646467 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:53.646467 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:28:53.647274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:53.647274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:28:53.647274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:28:53.647274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:28:53.647274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:28:53.647274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:28:53.647274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:28:53.647274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:53.647274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:28:53.647274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:53.647274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:28:53.647274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:53.647622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:28:53.647622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:53.647622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:28:53.647622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:53.647622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:28:53.647622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:28:53.647622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:53.647622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:28:53.647622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:53.647622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:28:53.647622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:28:53.647622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:53.647622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:28:53.647991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:53.647991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:28:53.648421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:53.648421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:28:53.648421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:28:53.648421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:28:53.648421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:28:53.648421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:28:53.648421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:28:53.648421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:28:53.648421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:28:53.648421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:28:53.648421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:28:53.648421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:28:53.648421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:28:53.648421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:28:53.648421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:28:53.648421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:28:53.648926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:28:53.648926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:28:53.648926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:53.648926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:28:53.648926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:53.648926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:28:53.648926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:53.648926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:28:53.648926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:53.648926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:28:53.648926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:53.648926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:28:53.648926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:28:53.648926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:53.648926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:28:53.648926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:53.648926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:28:53.648926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:28:53.648926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:53.649643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:28:53.649643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:53.649643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:28:58.470734 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:28:58.471305 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:28:58.471585 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:28:58.473309 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:28:58.474900 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG dbcounter [-] [106314] Writing DB stats neutron:SELECT=240 {{(pid=106314) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 09 09:28:58.511521 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:28:58.511932 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:58.512862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:28:58.512862 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:58.512862 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:28:58.512862 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:28:58.512862 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:58.512862 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:28:58.512862 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:58.512862 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:28:58.513954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:58.513954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:28:58.513954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:28:58.513954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:28:58.513954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:28:58.513954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:28:58.513954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:28:58.513954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:58.513954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:28:58.513954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:58.513954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:28:58.513954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:58.514620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:28:58.514620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:58.514620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:28:58.514620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:58.514620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:28:58.514620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:28:58.514620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:58.514620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:28:58.514620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:58.514620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:28:58.514620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:28:58.514620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:58.514620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:28:58.515275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:58.515275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:28:58.515385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:58.515385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:28:58.515385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:28:58.515385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:28:58.515385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:28:58.515385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:28:58.515385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:28:58.515385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:28:58.515385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:28:58.515385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:28:58.515385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:28:58.515385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:28:58.515385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:28:58.515385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:28:58.515385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:28:58.515385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:28:58.517076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:28:58.517076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:28:58.517076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:58.517076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:28:58.517076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:58.517076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:28:58.517076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:58.517076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:28:58.517076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:58.517076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:28:58.517076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:58.517076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:28:58.517076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:28:58.517076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:58.517076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:28:58.517076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:58.517076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:28:58.517076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:28:58.517076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:58.518702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:28:58.518702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:58.518702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:28:58.622385 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:28:58.623863 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:28:58.623863 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:28:58.624030 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:28:58.624471 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:28:58.624783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:58.625795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:28:58.625795 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:58.625795 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:28:58.625795 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:28:58.625795 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:58.625795 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:28:58.625795 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:58.625795 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:28:58.626928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:58.626928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:28:58.626928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:28:58.626928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:28:58.626928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:28:58.626928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:28:58.626928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:28:58.626928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:58.626928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:28:58.626928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:58.626928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:28:58.626928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:58.627484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:28:58.627484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:58.627484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:28:58.627484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:58.627484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:28:58.627484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:28:58.627484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:58.627484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:28:58.627484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:58.627484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:28:58.627484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:28:58.627484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:58.627484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:28:58.628052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:58.628052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:28:58.628052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:58.628052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:28:58.628052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:28:58.628052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:28:58.628052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:28:58.628052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:28:58.628052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:28:58.628052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:28:58.628052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:28:58.628052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:28:58.628052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:28:58.628052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:28:58.628052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:28:58.628684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:28:58.628684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:28:58.628684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:28:58.628684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:28:58.628684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:28:58.628684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:58.628684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:28:58.628684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:58.628684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:28:58.628684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:58.628684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:28:58.628684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:58.628684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:28:58.628684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:58.628684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:28:58.628684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:28:58.628684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:58.628684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:28:58.628684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:58.629500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:28:58.629500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:28:58.629500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:58.629500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:28:58.629500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:58.629500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:28:58.632037 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:28:58.632483 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:28:58.632768 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:28:58.632997 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:28:58.633606 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:28:58.633841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:58.634522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:28:58.634522 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:58.634522 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:28:58.634522 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:28:58.634522 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:58.634522 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:28:58.634522 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:58.634522 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:28:58.635461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:58.635461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:28:58.635461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:28:58.635461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:28:58.635461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:58.635461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:28:58.635461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:58.635461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:28:58.635461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:58.635461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:28:58.635461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:58.635461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:28:58.635461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:58.635461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:28:58.635461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:28:58.636265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:58.636265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:28:58.636265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:58.636265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:28:58.636265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:28:58.636265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:58.636265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:28:58.636265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:58.636265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:28:58.636265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:58.636265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:28:58.636265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:28:58.636265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:28:58.636265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:28:58.636939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:28:58.636939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:28:58.636939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:28:58.636939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:28:58.636939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:28:58.636939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:58.636939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:28:58.636939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:58.636939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:28:58.636939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:58.636939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:28:58.636939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:58.636939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:28:58.637179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:58.637179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:28:58.637179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:28:58.637179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:58.637179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:28:58.637179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:58.637179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:28:58.637179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:28:58.637179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:58.637179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:28:58.637179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:58.637179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:28:58.638465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:58.638465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:28:58.638465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:28:58.638465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:28:58.638465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:28:58.638465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:28:58.638465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:28:58.638465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:28:58.638465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:28:58.638465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:28:58.638465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:28:58.638465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:28:58.638465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:28:58.638465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:28:58.638465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:28:58.638465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:28:58.638800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:28:58.638800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:28:58.638800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:28:58.638800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:28:58.638800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:28:58.638800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:58.638800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:28:58.638800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:58.638800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:28:58.638800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:58.638800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:28:58.638800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:58.638800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:28:58.638800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:58.638800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:28:58.638800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:28:58.638800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:58.638800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:28:58.639130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:58.639130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:28:58.639130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:28:58.639130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:58.639130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:28:58.639130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:58.639130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:28:58.639130 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:28:58.639298 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:28:58.639298 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:28:58.639632 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:28:58.639735 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:28:58.639986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:58.640501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:28:58.640501 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:58.640501 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:28:58.640501 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:28:58.640501 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:58.640501 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:28:58.640501 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:58.640501 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:28:58.641311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:58.641311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:28:58.641311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:28:58.641311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:28:58.641311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:58.641311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:28:58.641311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:58.641311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:28:58.641311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:58.641311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:28:58.641311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:58.641311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:28:58.641311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:58.641311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:28:58.641311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:28:58.642311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:58.642311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:28:58.642311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:58.642311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:28:58.642311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:28:58.642311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:58.642311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:28:58.642311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:58.642311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:28:58.642311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:58.642311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:28:58.642311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:28:58.642311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:28:58.642311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:28:58.642685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:28:58.642685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:28:58.642685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:28:58.642685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:28:58.642685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:58.642685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:28:58.642685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:58.642685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:28:58.642685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:58.642685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:28:58.642685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:58.642685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:28:58.642685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:58.643038 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:28:58.643038 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:28:58.643038 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:58.643038 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:28:58.643038 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:58.643038 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:28:58.643038 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:28:58.643038 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:58.643038 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:28:58.643038 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:58.643038 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:28:58.643704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:58.643704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:28:58.643704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:28:58.643704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:28:58.643704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:28:58.643704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:28:58.643704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:28:58.643704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:28:58.643704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:28:58.643704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:28:58.643704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:28:58.643704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:28:58.643704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:28:58.643704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:28:58.643704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:28:58.643704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:28:58.643704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:28:58.644891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:28:58.644891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:28:58.644891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:28:58.644891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:58.644891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:28:58.644891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:58.644891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:28:58.644891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:58.644891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:28:58.644891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:58.644891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:28:58.644891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:58.644891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:28:58.644891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:28:58.644891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:58.644891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:28:58.644891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:58.644891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:28:58.644891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:28:58.645647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:58.645647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:28:58.645647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:58.645647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:28:58.645647 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:28:58.645647 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:28:58.645647 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:28:58.645647 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:28:58.646116 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:28:58.646334 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:28:58.646483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:58.647016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:28:58.647016 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:58.647016 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:28:58.647016 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:28:58.647016 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:58.647016 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:28:58.647016 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:58.647016 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:28:58.647974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:58.647974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:28:58.647974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:28:58.647974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:28:58.647974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:28:58.647974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:28:58.647974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:28:58.647974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:58.647974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:28:58.647974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:58.647974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:28:58.647974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:58.648732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:28:58.648732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:58.648732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:28:58.648732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:58.648732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:28:58.648732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:28:58.648732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:58.648732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:28:58.648732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:58.648732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:28:58.648732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:28:58.648732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:58.648732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:28:58.649152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:58.649152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:28:58.649152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:58.649152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:28:58.649152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:28:58.649152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:28:58.649152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:28:58.649152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:28:58.649152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:28:58.649152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:28:58.649152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:28:58.649152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:28:58.649152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:28:58.649152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:28:58.649152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:28:58.649513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:28:58.649513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:28:58.649513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:28:58.649513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:28:58.649513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:28:58.649513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:58.649513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:28:58.649513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:28:58.649513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:28:58.649513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:28:58.649513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:28:58.649513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:28:58.649513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:28:58.649513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:28:58.649513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:28:58.649513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:28:58.649513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:28:58.649513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:28:58.649513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:28:58.650107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:28:58.650107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:28:58.650107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:28:58.650107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:28:58.650107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:28:58.650107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:29:03.471660 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:29:03.472304 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:29:03.472586 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:29:03.474826 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:29:03.510005 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:29:03.510415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:03.511504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:29:03.511504 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:03.511504 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:29:03.511504 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:29:03.511504 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:03.511504 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:29:03.511504 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:03.511504 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:29:03.512500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:03.512500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:29:03.512500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:29:03.512500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:29:03.512500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:29:03.512500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:29:03.512500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:29:03.512500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:03.512500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:29:03.512500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:03.512500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:29:03.512500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:03.513184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:29:03.513184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:03.513184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:29:03.513184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:03.513184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:29:03.513184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:29:03.513184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:03.513184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:29:03.513184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:03.513184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:29:03.513184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:29:03.513184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:03.513184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:29:03.513874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:03.513874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:29:03.513874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:03.513874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:29:03.513874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:29:03.513874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:29:03.513874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:29:03.513874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:29:03.513874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:29:03.513874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:29:03.513874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:29:03.513874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:29:03.513874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:29:03.513874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:29:03.513874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:29:03.514663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:29:03.514663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:29:03.514663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:29:03.514663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:29:03.514663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:29:03.514663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:03.514663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:29:03.514663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:03.514663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:29:03.514663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:03.514663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:29:03.514663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:03.514663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:29:03.514663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:03.514663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:29:03.514663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:29:03.514663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:03.514663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:29:03.514663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:03.515648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:29:03.515648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:29:03.515648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:03.515648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:29:03.515648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:03.515648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:29:03.622987 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:29:03.623624 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:29:03.623932 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:29:03.625326 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:29:03.625832 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:29:03.626249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:03.627435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:29:03.627435 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:03.627435 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:29:03.627435 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:29:03.627435 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:03.627435 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:29:03.627435 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:03.627435 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:29:03.629249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:03.629249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:29:03.629249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:29:03.629249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:29:03.629249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:29:03.629249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:29:03.629249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:29:03.629249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:03.629249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:29:03.629249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:03.629249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:29:03.629249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:03.629626 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:29:03.629626 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:03.629626 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:29:03.629626 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:03.629626 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:29:03.629626 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:29:03.629626 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:03.629626 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:29:03.629626 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:03.629626 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:29:03.629626 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:29:03.629626 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:03.629626 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:29:03.630001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:03.630001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:29:03.632230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:03.632230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:29:03.632230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:29:03.632230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:29:03.632230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:29:03.632230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:29:03.632230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:29:03.632230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:29:03.632230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:29:03.632230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:29:03.632230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:29:03.632230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:29:03.632230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:29:03.632230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:29:03.632230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:29:03.632230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:29:03.632623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:29:03.632623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:29:03.632623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:03.632623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:29:03.632623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:03.632623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:29:03.632623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:03.632623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:29:03.632623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:03.632623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:29:03.632623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:03.632623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:29:03.632623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:29:03.632623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:03.632623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:29:03.632623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:03.632623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:29:03.632623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:29:03.632623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:03.633080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:29:03.633080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:03.633080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:29:03.633080 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:29:03.633250 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:29:03.633668 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:29:03.633668 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:29:03.633919 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:29:03.634120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:03.634671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:29:03.634671 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:03.634671 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:29:03.634671 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:29:03.634671 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:03.634671 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:29:03.634671 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:03.634671 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:29:03.635680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:03.635680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:29:03.635680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:29:03.635680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:29:03.635680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:03.635680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:29:03.635680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:03.635680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:29:03.635680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:03.635680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:29:03.635680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:03.635680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:29:03.635680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:03.635680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:29:03.635680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:29:03.636232 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:03.636232 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:29:03.636232 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:03.636232 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:29:03.636232 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:29:03.636232 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:03.636232 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:29:03.636232 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:03.636232 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:29:03.636575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:03.636575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:29:03.636575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:29:03.636575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:29:03.636575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:29:03.636575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:29:03.636575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:29:03.636575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:29:03.636575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:29:03.636575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:29:03.636575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:03.636575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:29:03.637013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:03.637013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:29:03.637013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:03.637013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:29:03.637013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:03.637013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:29:03.637013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:03.637013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:29:03.637013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:29:03.637013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:03.637013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:29:03.637013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:03.637013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:29:03.637013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:29:03.637400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:03.637400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:29:03.637400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:03.637400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:29:03.637639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:03.637639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:29:03.637639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:29:03.637639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:29:03.637639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:29:03.637639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:29:03.637639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:29:03.637639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:29:03.637639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:29:03.637639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:29:03.637639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:29:03.637639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:29:03.637639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:29:03.637639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:29:03.637639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:29:03.637639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:29:03.638193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:29:03.638193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:29:03.638193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:29:03.638193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:29:03.638193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:29:03.638193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:03.638193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:29:03.638193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:03.638193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:29:03.638193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:03.638193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:29:03.638193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:03.638193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:29:03.638193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:03.638193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:29:03.638193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:29:03.638193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:03.638193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:29:03.638789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:03.638789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:29:03.638789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:29:03.638789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:03.638789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:29:03.638789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:03.638789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:29:03.639229 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:29:03.639546 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:29:03.639708 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:29:03.639937 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:29:03.640307 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:29:03.640528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:03.641327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:29:03.641327 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:03.641327 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:29:03.641327 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:29:03.641327 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:03.641327 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:29:03.641327 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:03.641327 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:29:03.641996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:03.641996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:29:03.641996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:29:03.641996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:29:03.641996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:03.641996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:29:03.641996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:03.641996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:29:03.641996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:03.641996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:29:03.641996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:03.641996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:29:03.641996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:03.641996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:29:03.641996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:29:03.642584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:03.642584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:29:03.642584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:03.642584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:29:03.642584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:29:03.642584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:03.642584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:29:03.642584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:03.642584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:29:03.643291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:03.643291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:29:03.643291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:29:03.643291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:29:03.643291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:29:03.643291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:29:03.643291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:29:03.643291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:29:03.643291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:29:03.643291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:03.643291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:29:03.643818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:03.643818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:29:03.643818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:03.643818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:29:03.643818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:03.643818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:29:03.643818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:03.643818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:29:03.643818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:29:03.643818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:03.643818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:29:03.643818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:03.643818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:29:03.643818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:29:03.644378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:03.644378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:29:03.644378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:03.644378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:29:03.644766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:03.644766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:29:03.644766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:29:03.644766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:29:03.644766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:29:03.644766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:29:03.644766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:29:03.644766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:29:03.644766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:29:03.644766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:29:03.644766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:29:03.644766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:29:03.644766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:29:03.644766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:29:03.644766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:29:03.644766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:29:03.644766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:29:03.645461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:29:03.645461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:29:03.645461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:29:03.645461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:03.645461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:29:03.645461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:03.645461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:29:03.645461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:03.645461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:29:03.645461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:03.645461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:29:03.645461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:03.645461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:29:03.645461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:29:03.645461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:03.645461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:29:03.645461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:03.645461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:29:03.645461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:29:03.646243 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:03.646243 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:29:03.646243 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:03.646243 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:29:03.646243 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:29:03.646243 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:29:03.646243 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:29:03.646243 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:29:03.646615 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:29:03.646770 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:29:03.646985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:03.647569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:29:03.647569 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:03.647569 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:29:03.647569 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:29:03.647569 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:03.647569 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:29:03.647569 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:03.647569 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:29:03.648500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:03.648500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:29:03.648500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:29:03.648500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:29:03.648500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:29:03.648500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:29:03.648500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:29:03.648500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:03.648500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:29:03.648500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:03.648500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:29:03.648500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:03.649069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:29:03.649069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:03.649069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:29:03.649069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:03.649069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:29:03.649069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:29:03.649069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:03.649069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:29:03.649069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:03.649069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:29:03.649069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:29:03.649069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:03.649069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:29:03.649579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:03.649579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:29:03.649579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:03.649579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:29:03.649579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:29:03.649579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:29:03.649579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:29:03.649579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:29:03.649579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:29:03.649579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:29:03.649579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:29:03.649579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:29:03.649579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:29:03.649579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:29:03.649579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:29:03.650198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:29:03.650198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:29:03.650198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:29:03.650198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:29:03.650198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:29:03.650198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:03.650198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:29:03.650198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:03.650198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:29:03.650198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:03.650198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:29:03.650198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:03.650198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:29:03.650198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:03.650198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:29:03.650198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:29:03.650198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:03.650198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:29:03.650198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:03.650856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:29:03.650856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:29:03.650856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:03.650856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:29:03.650856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:03.650856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:29:08.472582 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:29:08.473998 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:29:08.473998 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:29:08.474946 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:29:08.511223 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:29:08.511660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:08.512612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:29:08.512612 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:08.512612 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:29:08.512612 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:29:08.512612 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:08.512612 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:29:08.512612 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:08.512612 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:29:08.513799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:08.513799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:29:08.513799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:29:08.513799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:29:08.513799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:29:08.513799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:29:08.513799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:29:08.513799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:08.513799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:29:08.513799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:08.513799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:29:08.513799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:08.514498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:29:08.514498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:08.514498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:29:08.514498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:08.514498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:29:08.514498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:29:08.514498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:08.514498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:29:08.514498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:08.514498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:29:08.514498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:29:08.514498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:08.514498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:29:08.515857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:08.515857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:29:08.515857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:08.515857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:29:08.515857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:29:08.515857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:29:08.515857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:29:08.515857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:29:08.515857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:29:08.515857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:29:08.515857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:29:08.515857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:29:08.515857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:29:08.515857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:29:08.515857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:29:08.517619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:29:08.517619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:29:08.517619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:29:08.517619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:29:08.517619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:29:08.517619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:08.517619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:29:08.517619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:08.517619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:29:08.517619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:08.517619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:29:08.517619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:08.517619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:29:08.517619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:08.517619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:29:08.517619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:29:08.517619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:08.517619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:29:08.517619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:08.519991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:29:08.519991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:29:08.519991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:08.519991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:29:08.519991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:08.519991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:29:08.623928 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:29:08.624316 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:29:08.624422 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:29:08.625316 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:29:08.625503 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:29:08.625641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:08.626399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:29:08.626399 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:08.626399 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:29:08.626399 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:29:08.626399 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:08.626399 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:29:08.626399 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:08.626399 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:29:08.627447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:08.627447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:29:08.627447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:29:08.627447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:29:08.627447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:29:08.627447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:29:08.627447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:29:08.627447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:08.627447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:29:08.627447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:08.627447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:29:08.627447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:08.627963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:29:08.627963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:08.627963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:29:08.627963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:08.627963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:29:08.627963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:29:08.627963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:08.627963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:29:08.627963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:08.627963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:29:08.627963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:29:08.627963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:08.627963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:29:08.628484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:08.628484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:29:08.629045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:08.629045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:29:08.629045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:29:08.629045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:29:08.629045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:29:08.629045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:29:08.629045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:29:08.629045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:29:08.629045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:29:08.629045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:29:08.629045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:29:08.629045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:29:08.629045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:29:08.629045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:29:08.629045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:29:08.629045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:29:08.630174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:29:08.630174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:29:08.630174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:08.630174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:29:08.630174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:08.630174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:29:08.630174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:08.630174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:29:08.630174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:08.630174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:29:08.630174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:08.630174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:29:08.630174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:29:08.630174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:08.630174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:29:08.630174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:08.630174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:29:08.630174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:29:08.630174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:08.630875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:29:08.630875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:08.630875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:29:08.633316 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:29:08.633578 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:29:08.633763 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:29:08.633947 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:29:08.634411 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:29:08.634644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:08.635165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:29:08.635165 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:08.635165 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:29:08.635165 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:29:08.635165 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:08.635165 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:29:08.635165 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:08.635165 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:29:08.635932 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:08.635932 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:29:08.635932 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:29:08.635932 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:29:08.635932 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:08.635932 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:29:08.635932 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:08.635932 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:29:08.635932 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:08.635932 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:29:08.635932 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:08.635932 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:29:08.635932 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:08.635932 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:29:08.635932 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:29:08.636362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:08.636362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:29:08.636362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:08.636362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:29:08.636362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:29:08.636362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:08.636362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:29:08.636362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:08.636362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:29:08.636936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:08.636936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:29:08.636936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:29:08.636936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:29:08.636936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:29:08.636936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:29:08.636936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:29:08.636936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:29:08.636936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:29:08.636936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:29:08.636936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:08.636936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:29:08.637593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:08.637593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:29:08.637593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:08.637593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:29:08.637593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:08.637593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:29:08.637593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:08.637593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:29:08.637593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:29:08.637593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:08.637593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:29:08.637593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:08.637593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:29:08.637593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:29:08.638200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:08.638200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:29:08.638200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:08.638200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:29:08.638200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:08.638200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:29:08.638200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:29:08.638200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:29:08.638200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:29:08.638200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:29:08.638200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:29:08.638200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:29:08.638200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:29:08.638200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:29:08.638517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:29:08.638517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:29:08.638517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:29:08.638517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:29:08.638517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:29:08.638517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:29:08.638517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:29:08.638517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:29:08.638517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:29:08.638517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:29:08.638517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:29:08.638517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:08.638517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:29:08.638517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:08.638517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:29:08.638517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:08.638517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:29:08.638517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:08.638517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:29:08.638893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:08.638893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:29:08.638893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:29:08.638893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:08.638893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:29:08.638893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:08.638893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:29:08.638893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:29:08.638893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:08.638893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:29:08.638893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:08.638893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:29:08.639548 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:29:08.639802 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:29:08.639913 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:29:08.640106 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:29:08.640408 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:29:08.640667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:08.641203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:29:08.641203 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:08.641203 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:29:08.641203 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:29:08.641203 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:08.641203 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:29:08.641203 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:08.641203 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:29:08.642062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:08.642062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:29:08.642062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:29:08.642062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:29:08.642062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:08.642062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:29:08.642062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:08.642062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:29:08.642062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:08.642062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:29:08.642062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:08.642062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:29:08.642062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:08.642062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:29:08.642062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:29:08.642471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:08.642471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:29:08.642471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:08.642471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:29:08.642471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:29:08.642471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:08.642471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:29:08.642471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:08.642471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:29:08.643204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:08.643204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:29:08.643204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:29:08.643204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:29:08.643204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:29:08.643204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:29:08.643204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:29:08.643204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:29:08.643204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:29:08.643204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:08.643204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:29:08.643551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:08.643551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:29:08.643551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:08.643551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:29:08.643551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:08.643551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:29:08.643551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:08.643551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:29:08.643551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:29:08.643551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:08.643551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:29:08.643551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:08.643551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:29:08.643551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:29:08.644055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:08.644055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:29:08.644055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:08.644055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:29:08.644526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:08.644526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:29:08.644526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:29:08.644526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:29:08.644526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:29:08.644526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:29:08.644526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:29:08.644526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:29:08.644526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:29:08.644526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:29:08.644526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:29:08.644526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:29:08.644526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:29:08.644526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:29:08.644526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:29:08.644526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:29:08.644526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:29:08.645549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:29:08.645549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:29:08.645549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:29:08.645549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:08.645549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:29:08.645549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:08.645549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:29:08.645549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:08.645549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:29:08.645549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:08.645549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:29:08.645549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:08.645549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:29:08.645549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:29:08.645549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:08.645549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:29:08.645549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:08.645549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:29:08.645549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:29:08.646180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:08.646180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:29:08.646180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:08.646180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:29:08.646180 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:29:08.646180 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:29:08.646180 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:29:08.646387 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:29:08.647886 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:29:08.648513 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:29:08.648846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:08.649843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:29:08.649843 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:08.649843 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:29:08.649843 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:29:08.649843 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:08.649843 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:29:08.649843 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:08.649843 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:29:08.651034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:08.651034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:29:08.651034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:29:08.651034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:29:08.651034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:29:08.651034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:29:08.651034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:29:08.651034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:08.651034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:29:08.651034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:08.651034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:29:08.651034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:08.651382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:29:08.651382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:08.651382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:29:08.651382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:08.651382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:29:08.651382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:29:08.651382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:08.651382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:29:08.651382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:08.651382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:29:08.651382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:29:08.651382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:08.651382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:29:08.651696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:08.651696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:29:08.652501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:08.652501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:29:08.652501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:29:08.652501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:29:08.652501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:29:08.652501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:29:08.652501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:29:08.652501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:29:08.652501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:29:08.652501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:29:08.652501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:29:08.652501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:29:08.652501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:29:08.652501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:29:08.652501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:29:08.652501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:29:08.653416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:29:08.653416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:29:08.653416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:08.653416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:29:08.653416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:08.653416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:29:08.653416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:08.653416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:29:08.653416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:08.653416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:29:08.653416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:08.653416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:29:08.653416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:29:08.653416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:08.653416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:29:08.653416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:08.653416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:29:08.653416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:29:08.653416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:08.654547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:29:08.654547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:08.654547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:29:13.473424 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:29:13.474413 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:29:13.474413 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:29:13.475878 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:29:13.513924 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:29:13.513924 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:13.514550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:29:13.514550 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:13.514550 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:29:13.514550 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:29:13.514550 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:13.514550 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:29:13.514550 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:13.514550 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:29:13.515468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:13.515468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:29:13.515468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:29:13.515468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:29:13.515468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:29:13.515468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:29:13.515468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:29:13.515468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:13.515468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:29:13.515468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:13.515468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:29:13.515468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:13.515967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:29:13.515967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:13.515967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:29:13.515967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:13.515967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:29:13.515967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:29:13.515967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:13.515967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:29:13.515967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:13.515967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:29:13.515967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:29:13.515967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:13.515967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:29:13.516375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:13.516375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:29:13.516510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:13.516510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:29:13.516510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:29:13.516510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:29:13.516510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:29:13.516510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:29:13.516510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:29:13.516510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:29:13.516510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:29:13.516510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:29:13.516510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:29:13.516510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:29:13.516510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:29:13.516510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:29:13.516510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:29:13.516510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:29:13.517123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:29:13.517123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:29:13.517123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:13.517123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:29:13.517123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:13.517123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:29:13.517123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:13.517123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:29:13.517123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:13.517123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:29:13.517123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:13.517123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:29:13.517123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:29:13.517123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:13.517123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:29:13.517123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:13.517123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:29:13.517123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:29:13.517123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:13.517702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:29:13.517702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:13.517702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:29:13.624444 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:29:13.625165 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:29:13.625371 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:29:13.626559 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:29:13.626867 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:29:13.627073 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:13.627778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:29:13.627778 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:13.627778 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:29:13.627778 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:29:13.627778 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:13.627778 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:29:13.627778 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:13.627778 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:29:13.629415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:13.629415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:29:13.629415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:29:13.629415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:29:13.629415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:29:13.629415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:29:13.629415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:29:13.629415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:13.629415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:29:13.629415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:13.629415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:29:13.629415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:13.629802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:29:13.629802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:13.629802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:29:13.629802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:13.629802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:29:13.629802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:29:13.629802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:13.629802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:29:13.629802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:13.629802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:29:13.629802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:29:13.629802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:13.629802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:29:13.630219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:13.630219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:29:13.631118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:13.631118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:29:13.631118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:29:13.631118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:29:13.631118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:29:13.631118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:29:13.631118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:29:13.631118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:29:13.631118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:29:13.631118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:29:13.631118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:29:13.631118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:29:13.631118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:29:13.631118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:29:13.631118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:29:13.631118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:29:13.631628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:29:13.631628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:29:13.631628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:13.631628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:29:13.631628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:13.631628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:29:13.631628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:13.631628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:29:13.631628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:13.631628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:29:13.631628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:13.631628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:29:13.631628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:29:13.631628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:13.631628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:29:13.631628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:13.631628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:29:13.631628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:29:13.631628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:13.632134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:29:13.632134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:13.632134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:29:13.633642 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:29:13.634068 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:29:13.634239 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:29:13.634426 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:29:13.635110 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:29:13.635672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:13.636848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:29:13.636848 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:13.636848 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:29:13.636848 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:29:13.636848 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:13.636848 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:29:13.636848 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:13.636848 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:29:13.638191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:13.638191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:29:13.638191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:29:13.638191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:29:13.638191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:13.638191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:29:13.638191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:13.638191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:29:13.638191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:13.638191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:29:13.638191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:13.638191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:29:13.638191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:13.638191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:29:13.638191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:29:13.638777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:13.638777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:29:13.638777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:13.638777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:29:13.638777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:29:13.638777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:13.638777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:29:13.638777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:13.638777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:29:13.639083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:13.639083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:29:13.639083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:29:13.639083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:29:13.639083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:29:13.639083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:29:13.639083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:29:13.639083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:29:13.639083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:29:13.639083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:29:13.639083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:13.639083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:29:13.639483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:13.639483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:29:13.639483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:13.639483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:29:13.639483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:13.639483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:29:13.639483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:13.639483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:29:13.639483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:29:13.639483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:13.639483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:29:13.639483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:13.639483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:29:13.639483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:29:13.639956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:13.639956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:29:13.639956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:13.639956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:29:13.640235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:13.640235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:29:13.640235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:29:13.640235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:29:13.640235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:29:13.640235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:29:13.640235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:29:13.640235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:29:13.640235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:29:13.640235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:29:13.640235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:29:13.640235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:29:13.640235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:29:13.640235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:29:13.640235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:29:13.640235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:29:13.640782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:29:13.640782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:29:13.640782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:29:13.640782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:29:13.640782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:29:13.640782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:13.640782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:29:13.640782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:13.640782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:29:13.640782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:13.640782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:29:13.640782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:13.640782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:29:13.640782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:13.640782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:29:13.640782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:29:13.640782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:13.640782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:29:13.641381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:13.641381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:29:13.641381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:29:13.641381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:13.641381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:29:13.641381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:13.641381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:29:13.641381 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:29:13.641381 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:29:13.641381 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:29:13.641682 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:29:13.641682 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:29:13.641846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:13.642248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:29:13.642248 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:13.642248 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:29:13.642248 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:29:13.642248 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:13.642248 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:29:13.642248 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:13.642248 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:29:13.643092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:13.643092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:29:13.643092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:29:13.643092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:29:13.643092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:13.643092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:29:13.643092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:13.643092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:29:13.643092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:13.643092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:29:13.643092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:13.643092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:29:13.643092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:13.643092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:29:13.643092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:29:13.643562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:13.643562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:29:13.643562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:13.643562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:29:13.643562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:29:13.643562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:13.643562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:29:13.643562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:13.643562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:29:13.643874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:13.643874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:29:13.643874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:29:13.643874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:29:13.643874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:29:13.643874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:29:13.643874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:29:13.643874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:29:13.643874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:29:13.643874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:13.643874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:29:13.644229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:13.644229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:29:13.644229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:13.644229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:29:13.644229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:13.644229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:29:13.644229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:13.644229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:29:13.644229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:29:13.644229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:13.644229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:29:13.644229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:13.644229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:29:13.644229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:29:13.644637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:13.644637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:29:13.644637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:13.644637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:29:13.644925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:13.644925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:29:13.644925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:29:13.644925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:29:13.644925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:29:13.644925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:29:13.644925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:29:13.644925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:29:13.644925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:29:13.644925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:29:13.644925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:29:13.644925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:29:13.644925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:29:13.644925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:29:13.644925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:29:13.644925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:29:13.644925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:29:13.645500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:29:13.645500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:29:13.645500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:29:13.645500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:13.645500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:29:13.645500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:13.645500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:29:13.645500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:13.645500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:29:13.645500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:13.645500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:29:13.645500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:13.645500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:29:13.645500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:29:13.645500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:13.645500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:29:13.645500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:13.645500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:29:13.645500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:29:13.646055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:13.646055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:29:13.646055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:13.646055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:29:13.646242 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:29:13.646542 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:29:13.646594 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:29:13.646714 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:29:13.647131 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:29:13.647432 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:29:13.647621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:13.648202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:29:13.648202 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:13.648202 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:29:13.648202 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:29:13.648202 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:13.648202 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:29:13.648202 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:13.648202 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:29:13.648824 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:13.648824 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:29:13.648824 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:29:13.648824 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:29:13.648824 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:29:13.648824 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:29:13.648824 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:29:13.648824 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:13.648824 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:29:13.648824 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:13.648824 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:29:13.648824 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:13.649209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:29:13.649209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:13.649209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:29:13.649209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:13.649209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:29:13.649209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:29:13.649209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:13.649209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:29:13.649209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:13.649209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:29:13.649209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:29:13.649209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:13.649209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:29:13.649565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:13.649565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:29:13.649565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:13.649565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:29:13.649565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:29:13.649565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:29:13.649565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:29:13.649565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:29:13.649565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:29:13.649565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:29:13.649565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:29:13.649565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:29:13.649565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:29:13.649565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:29:13.649565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:29:13.649980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:29:13.649980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:29:13.649980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:29:13.649980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:29:13.649980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:29:13.649980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:13.649980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:29:13.649980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:13.649980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:29:13.649980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:13.649980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:29:13.649980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:13.649980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:29:13.649980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:13.649980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:29:13.649980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:29:13.649980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:13.649980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:29:13.649980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:13.650482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:29:13.650482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:29:13.650482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:13.650482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:29:13.650482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:13.650482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:29:18.473925 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:29:18.475291 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:29:18.475592 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:29:18.477594 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:29:18.514395 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:29:18.514739 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:18.515646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:29:18.515646 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:18.515646 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:29:18.515646 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:29:18.515646 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:18.515646 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:29:18.515646 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:18.515646 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:29:18.516575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:18.516575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:29:18.516575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:29:18.516575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:29:18.516575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:29:18.516575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:29:18.516575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:29:18.516575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:18.516575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:29:18.516575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:18.516575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:29:18.516575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:18.517017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:29:18.517017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:18.517017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:29:18.517017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:18.517017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:29:18.517017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:29:18.517017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:18.517017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:29:18.517017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:18.517017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:29:18.517017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:29:18.517017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:18.517017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:29:18.517452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:18.517452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:29:18.517452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:18.517452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:29:18.517452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:29:18.517452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:29:18.517452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:29:18.517452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:29:18.517452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:29:18.517452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:29:18.517452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:29:18.517452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:29:18.517452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:29:18.517452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:29:18.517452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:29:18.518379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:29:18.518379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:29:18.518379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:29:18.518379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:29:18.518379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:29:18.518379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:18.518379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:29:18.518379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:18.518379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:29:18.518379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:18.518379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:29:18.518379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:18.518379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:29:18.518379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:18.518379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:29:18.518379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:29:18.518379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:18.518379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:29:18.518379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:18.518979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:29:18.518979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:29:18.518979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:18.518979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:29:18.518979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:18.518979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:29:18.625409 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:29:18.626167 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:29:18.626380 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:29:18.627963 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:29:18.628169 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:29:18.628869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:18.629274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:29:18.629274 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:18.629274 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:29:18.629274 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:29:18.629274 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:18.629274 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:29:18.629274 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:18.629274 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:29:18.631310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:18.631310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:29:18.631310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:29:18.631310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:29:18.631310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:29:18.631310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:29:18.631310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:29:18.631310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:18.631310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:29:18.631310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:18.631310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:29:18.631310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:18.631938 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:29:18.631938 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:18.631938 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:29:18.631938 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:18.631938 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:29:18.631938 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:29:18.631938 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:18.631938 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:29:18.631938 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:18.631938 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:29:18.631938 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:29:18.631938 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:18.631938 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:29:18.632579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:18.632579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:29:18.633149 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:18.633149 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:29:18.633149 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:29:18.633149 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:29:18.633149 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:29:18.633149 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:29:18.633149 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:29:18.633149 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:29:18.633149 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:29:18.633149 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:29:18.633149 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:29:18.633149 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:29:18.633149 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:29:18.633149 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:29:18.633149 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:29:18.633149 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:29:18.634704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:29:18.634704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:29:18.634704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:18.634704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:29:18.634704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:18.634704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:29:18.634704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:18.634704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:29:18.634704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:18.634704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:29:18.634704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:18.634704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:29:18.634704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:29:18.634704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:18.634704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:29:18.634704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:18.634704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:29:18.634704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:29:18.634704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:18.635807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:29:18.635807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:18.635807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:29:18.635807 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:29:18.635807 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:29:18.635807 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:29:18.635807 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:29:18.635807 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:29:18.636123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:18.636430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:29:18.636430 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:18.636430 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:29:18.636430 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:29:18.636430 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:18.636430 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:29:18.636430 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:18.636430 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:29:18.637274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:18.637274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:29:18.637274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:29:18.637274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:29:18.637274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:18.637274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:29:18.637274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:18.637274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:29:18.637274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:18.637274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:29:18.637274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:18.637274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:29:18.637274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:18.637274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:29:18.637274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:29:18.637816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:18.637816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:29:18.637816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:18.637816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:29:18.637816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:29:18.637816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:18.637816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:29:18.637816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:18.637816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:29:18.638221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:18.638221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:29:18.638221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:29:18.638221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:29:18.638221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:29:18.638221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:29:18.638221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:29:18.638221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:29:18.638221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:29:18.638221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:29:18.638221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:18.638221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:29:18.638677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:18.638677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:29:18.638677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:18.638677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:29:18.638677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:18.638677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:29:18.638677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:18.638677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:29:18.638677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:29:18.638677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:18.638677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:29:18.638677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:18.638677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:29:18.638677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:29:18.639194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:18.639194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:29:18.639194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:18.639194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:29:18.639194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:18.639194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:29:18.639194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:29:18.639194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:29:18.639194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:29:18.639194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:29:18.639194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:29:18.639194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:29:18.639194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:29:18.639194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:29:18.639595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:29:18.639595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:29:18.639595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:29:18.639595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:29:18.639595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:29:18.639595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:29:18.639595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:29:18.639595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:29:18.639595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:29:18.639595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:29:18.639595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:29:18.639595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:18.639595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:29:18.639595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:18.639595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:29:18.639595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:18.639595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:29:18.639595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:18.639595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:29:18.640201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:18.640201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:29:18.640201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:29:18.640201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:18.640201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:29:18.640201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:18.640201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:29:18.640201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:29:18.640201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:18.640201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:29:18.640201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:18.640201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:29:18.641396 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:29:18.642210 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:29:18.642210 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:29:18.642210 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:29:18.642406 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:29:18.642693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:18.643334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:29:18.643334 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:18.643334 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:29:18.643334 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:29:18.643334 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:18.643334 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:29:18.643334 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:18.643334 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:29:18.644244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:18.644244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:29:18.644244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:29:18.644244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:29:18.644244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:18.644244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:29:18.644244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:18.644244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:29:18.644244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:18.644244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:29:18.644244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:18.644244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:29:18.644244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:18.644244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:29:18.644244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:29:18.644580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:18.644580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:29:18.644580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:18.644580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:29:18.644580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:29:18.644580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:18.644580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:29:18.644580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:18.644580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:29:18.645171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:18.645171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:29:18.645171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:29:18.645171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:29:18.645171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:29:18.645171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:29:18.645171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:29:18.645171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:29:18.645171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:29:18.645171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:18.645171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:29:18.645419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:18.645419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:29:18.645419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:18.645419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:29:18.645419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:18.645419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:29:18.645419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:18.645419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:29:18.645419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:29:18.645419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:18.645419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:29:18.645419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:18.645419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:29:18.645419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:29:18.645681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:18.645681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:29:18.645681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:18.645681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:29:18.646299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:18.646299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:29:18.646299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:29:18.646299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:29:18.646299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:29:18.646299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:29:18.646299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:29:18.646299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:29:18.646299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:29:18.646299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:29:18.646299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:29:18.646299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:29:18.646299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:29:18.646299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:29:18.646299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:29:18.646299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:29:18.646299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:29:18.646628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:29:18.646628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:29:18.646628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:29:18.646628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:18.646628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:29:18.646628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:18.646628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:29:18.646628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:18.646628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:29:18.646628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:18.646628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:29:18.646628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:18.646628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:29:18.646628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:29:18.646628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:18.646628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:29:18.646628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:18.646628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:29:18.646628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:29:18.647006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:18.647006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:29:18.647006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:18.647006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:29:18.647006 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:29:18.647167 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:29:18.647167 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:29:18.647329 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:29:18.648112 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:29:18.648421 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:29:18.648629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:18.649352 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:29:18.649352 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:18.649352 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:29:18.649352 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:29:18.649352 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:18.649352 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:29:18.649352 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:18.649352 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:29:18.650268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:18.650268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:29:18.650268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:29:18.650268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:29:18.650268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:29:18.650268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:29:18.650268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:29:18.650268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:18.650268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:29:18.650268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:18.650268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:29:18.650268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:18.650645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:29:18.650645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:18.650645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:29:18.650645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:18.650645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:29:18.650645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:29:18.650645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:18.650645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:29:18.650645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:18.650645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:29:18.650645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:29:18.650645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:18.650645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:29:18.650989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:18.650989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:29:18.651263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:18.651263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:29:18.651263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:29:18.651263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:29:18.651263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:29:18.651263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:29:18.651263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:29:18.651263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:29:18.651263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:29:18.651263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:29:18.651263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:29:18.651263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:29:18.651263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:29:18.651263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:29:18.651263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:29:18.651263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:29:18.651710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:29:18.651710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:29:18.651710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:18.651710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:29:18.651710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:18.651710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:29:18.651710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:18.651710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:29:18.651710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:18.651710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:29:18.651710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:18.651710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:29:18.651710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:29:18.651710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:18.651710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:29:18.651710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:18.651710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:29:18.651710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:29:18.651710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:18.652250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:29:18.652250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:18.652250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:29:23.474980 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:29:23.475670 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:29:23.475670 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:29:23.477948 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:29:23.514789 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:29:23.515333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:23.516444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:29:23.516444 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:23.516444 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:29:23.516444 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:29:23.516444 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:23.516444 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:29:23.516444 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:23.516444 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:29:23.517678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:23.517678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:29:23.517678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:29:23.517678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:29:23.517678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:29:23.517678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:29:23.517678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:29:23.517678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:23.517678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:29:23.517678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:23.517678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:29:23.517678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:23.518794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:29:23.518794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:23.518794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:29:23.518794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:23.518794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:29:23.518794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:29:23.518794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:23.518794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:29:23.518794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:23.518794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:29:23.518794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:29:23.518794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:23.518794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:29:23.519475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:23.519475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:29:23.519475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:23.519475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:29:23.519475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:29:23.519475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:29:23.519475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:29:23.519475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:29:23.519475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:29:23.519475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:29:23.519475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:29:23.519475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:29:23.519475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:29:23.519475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:29:23.519475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:29:23.520049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:29:23.520049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:29:23.520049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:29:23.520049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:29:23.520049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:29:23.520049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:23.520049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:29:23.520049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:23.520049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:29:23.520049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:23.520049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:29:23.520049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:23.520049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:29:23.520049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:23.520049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:29:23.520049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:29:23.520049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:23.520049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:29:23.520049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:23.520706 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:29:23.520706 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:29:23.520706 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:23.520706 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:29:23.520706 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:23.520706 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:29:23.627499 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:29:23.627916 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:29:23.628455 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:29:23.629845 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:29:23.630308 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:29:23.630534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:23.631479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:29:23.631479 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:23.631479 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:29:23.631479 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:29:23.631479 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:23.631479 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:29:23.631479 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:23.631479 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:29:23.632659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:23.632659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:29:23.632659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:29:23.632659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:29:23.632659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:29:23.632659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:29:23.632659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:29:23.632659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:23.632659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:29:23.632659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:23.632659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:29:23.632659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:23.633557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:29:23.633557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:23.633557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:29:23.633557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:23.633557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:29:23.633557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:29:23.633557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:23.633557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:29:23.633557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:23.633557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:29:23.633557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:29:23.633557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:23.633557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:29:23.634373 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:23.634373 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:29:23.634373 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:23.634373 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:29:23.634373 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:29:23.634373 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:29:23.634373 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:29:23.634373 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:29:23.634373 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:29:23.634373 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:29:23.634373 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:29:23.634373 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:29:23.634373 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:29:23.634373 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:29:23.634373 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:29:23.634857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:29:23.634857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:29:23.634857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:29:23.634857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:29:23.634857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:29:23.634857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:23.634857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:29:23.634857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:23.634857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:29:23.634857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:23.634857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:29:23.634857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:23.634857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:29:23.634857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:23.634857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:29:23.634857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:29:23.634857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:23.634857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:29:23.634857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:23.636297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:29:23.636297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:29:23.636297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:23.636297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:29:23.636297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:23.636297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:29:23.636297 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:29:23.636297 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:29:23.636297 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:29:23.636297 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:29:23.636589 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:29:23.636589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:23.636589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:29:23.636589 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:23.636589 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:29:23.636589 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:29:23.636589 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:23.636589 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:29:23.636589 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:23.636589 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:29:23.637494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:23.637494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:29:23.637494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:29:23.637494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:29:23.637494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:23.637494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:29:23.637494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:23.637494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:29:23.637494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:23.637494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:29:23.637494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:23.637494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:29:23.637494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:23.637494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:29:23.637494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:29:23.638166 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:23.638166 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:29:23.638166 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:23.638166 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:29:23.638166 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:29:23.638166 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:23.638166 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:29:23.638166 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:23.638166 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:29:23.638166 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:23.638166 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:29:23.638166 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:29:23.638166 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:29:23.638166 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:29:23.638614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:29:23.638614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:29:23.638614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:29:23.638614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:29:23.638614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:29:23.638614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:23.638614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:29:23.638614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:23.638614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:29:23.638614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:23.638614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:29:23.638614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:23.638614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:29:23.639729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:23.639729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:29:23.639729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:29:23.639729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:23.639729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:29:23.639729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:23.639729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:29:23.639729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:29:23.639729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:23.639729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:29:23.639729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:23.639729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:29:23.640320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:23.640320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:29:23.640320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:29:23.640320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:29:23.640320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:29:23.640320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:29:23.640320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:29:23.640320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:29:23.640320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:29:23.640320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:29:23.640320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:29:23.640320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:29:23.640320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:29:23.640320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:29:23.640320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:29:23.640320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:29:23.640864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:29:23.640864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:29:23.640864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:29:23.640864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:29:23.640864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:29:23.640864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:23.640864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:29:23.640864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:23.640864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:29:23.640864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:23.640864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:29:23.640864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:23.640864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:29:23.640864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:23.640864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:29:23.640864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:29:23.640864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:23.640864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:29:23.642083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:23.642083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:29:23.642083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:29:23.642083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:23.642083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:29:23.642083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:23.642083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:29:23.642083 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:29:23.642083 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:29:23.642083 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:29:23.642425 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:29:23.642833 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:29:23.642833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:23.643461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:29:23.643461 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:23.643461 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:29:23.643461 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:29:23.643461 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:23.643461 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:29:23.643461 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:23.643461 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:29:23.644379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:23.644379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:29:23.644379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:29:23.644379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:29:23.644379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:23.644379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:29:23.644379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:23.644379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:29:23.644379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:23.644379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:29:23.644379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:23.644379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:29:23.644379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:23.644379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:29:23.644379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:29:23.645775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:23.645775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:29:23.645775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:23.645775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:29:23.645775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:29:23.645775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:23.645775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:29:23.645775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:23.645775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:29:23.645775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:23.645775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:29:23.645775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:29:23.645775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:29:23.645775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:29:23.646422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:29:23.646422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:29:23.646422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:29:23.646422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:29:23.646422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:23.646422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:29:23.646422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:23.646422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:29:23.646422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:23.646422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:29:23.646422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:23.646422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:29:23.646422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:23.647166 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:29:23.647166 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:29:23.647166 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:23.647166 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:29:23.647166 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:23.647166 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:29:23.647166 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:29:23.647166 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:23.647166 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:29:23.647166 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:23.647166 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:29:23.647166 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:23.647818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:29:23.647818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:29:23.647818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:29:23.647818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:29:23.647818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:29:23.647818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:29:23.647818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:29:23.647818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:29:23.647818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:29:23.647818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:29:23.647818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:29:23.647818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:29:23.647818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:29:23.647818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:29:23.647818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:29:23.647818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:29:23.647818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:29:23.647818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:29:23.647818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:29:23.648472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:23.648472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:29:23.648472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:23.648472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:29:23.648472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:23.648472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:29:23.648472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:23.648472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:29:23.648472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:23.648472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:29:23.648472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:29:23.648472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:23.648472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:29:23.648472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:23.648472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:29:23.648472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:29:23.648472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:23.648472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:29:23.648472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:23.648472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:29:23.649037 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:29:23.649037 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:29:23.649037 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:29:23.649037 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:29:23.649037 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:29:23.649037 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:29:23.649265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:23.649265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:29:23.649265 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:23.649265 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:29:23.649265 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:29:23.649265 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:23.649265 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:29:23.649265 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:23.649265 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:29:23.650228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:23.650228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:29:23.650228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:29:23.650228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:29:23.650228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:29:23.650228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:29:23.650228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:29:23.650228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:23.650228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:29:23.650228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:23.650228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:29:23.650228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:23.650635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:29:23.650635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:23.650635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:29:23.650635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:23.650635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:29:23.650635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:29:23.650635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:23.650635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:29:23.650635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:23.650635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:29:23.650635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:29:23.650635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:23.650635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:29:23.651158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:23.651158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:29:23.651293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:23.651293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:29:23.651293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:29:23.651293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:29:23.651293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:29:23.651293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:29:23.651293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:29:23.651293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:29:23.651293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:29:23.651293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:29:23.651293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:29:23.651293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:29:23.651293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:29:23.651293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:29:23.651293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:29:23.651293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:29:23.652032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:29:23.652032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:29:23.652032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:23.652032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:29:23.652032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:23.652032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:29:23.652032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:23.652032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:29:23.652032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:23.652032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:29:23.652032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:23.652032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:29:23.652032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:29:23.652032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:23.652032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:29:23.652032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:23.652032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:29:23.652032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:29:23.652032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:23.653154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:29:23.653154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:23.653154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:29:28.475828 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:29:28.476614 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:29:28.476614 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:29:28.477068 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:29:28.517048 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:29:28.517048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:28.517822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:29:28.517822 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:28.517822 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:29:28.517822 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:29:28.517822 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:28.517822 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:29:28.517822 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:28.517822 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:29:28.518879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:28.518879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:29:28.518879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:29:28.518879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:29:28.518879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:29:28.518879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:29:28.518879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:29:28.518879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:28.518879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:29:28.518879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:28.518879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:29:28.518879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:28.519718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:29:28.519718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:28.519718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:29:28.519718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:28.519718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:29:28.519718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:29:28.519718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:28.519718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:29:28.519718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:28.519718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:29:28.519718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:29:28.519718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:28.519718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:29:28.520482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:28.520482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:29:28.520482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:28.520482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:29:28.520482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:29:28.520482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:29:28.520482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:29:28.520482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:29:28.520482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:29:28.520482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:29:28.520482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:29:28.520482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:29:28.520482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:29:28.520482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:29:28.520482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:29:28.521217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:29:28.521217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:29:28.521217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:29:28.521217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:29:28.521217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:29:28.521217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:28.521217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:29:28.521217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:28.521217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:29:28.521217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:28.521217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:29:28.521217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:28.521217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:29:28.521217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:28.521217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:29:28.521217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:29:28.521217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:28.521217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:29:28.521217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:28.521960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:29:28.521960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:29:28.521960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:28.521960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:29:28.521960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:28.521960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:29:28.628391 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:29:28.629377 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:29:28.629589 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:29:28.630770 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:29:28.631141 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:29:28.631366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:28.632259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:29:28.632259 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:28.632259 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:29:28.632259 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:29:28.632259 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:28.632259 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:29:28.632259 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:28.632259 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:29:28.633894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:28.633894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:29:28.633894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:29:28.633894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:29:28.633894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:29:28.633894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:29:28.633894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:29:28.633894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:28.633894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:29:28.633894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:28.633894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:29:28.633894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:28.634526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:29:28.634526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:28.634526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:29:28.634526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:28.634526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:29:28.634526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:29:28.634526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:28.634526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:29:28.634526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:28.634526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:29:28.634526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:29:28.634526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:28.634526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:29:28.635139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:28.635139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:29:28.636325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:28.636325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:29:28.636325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:29:28.636325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:29:28.636325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:29:28.636325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:29:28.636325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:29:28.636325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:29:28.636325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:29:28.636325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:29:28.636325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:29:28.636325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:29:28.636325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:29:28.636325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:29:28.636325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:29:28.636325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:29:28.636810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:29:28.636810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:29:28.636810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:28.636810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:29:28.636810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:28.636810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:29:28.636810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:28.636810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:29:28.636810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:28.636810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:29:28.636810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:28.636810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:29:28.636810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:29:28.636810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:28.636810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:29:28.636810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:28.636810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:29:28.636810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:29:28.636810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:28.637363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:29:28.637363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:28.637363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:29:28.637363 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:29:28.637363 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:29:28.637363 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:29:28.637363 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:29:28.637875 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:29:28.638096 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:28.638857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:29:28.638857 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:28.638857 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:29:28.638857 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:29:28.638857 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:28.638857 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:29:28.638857 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:28.638857 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:29:28.639683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:28.639683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:29:28.639683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:29:28.639683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:29:28.639683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:28.639683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:29:28.639683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:28.639683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:29:28.639683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:28.639683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:29:28.639683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:28.639683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:29:28.639683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:28.639683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:29:28.639683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:29:28.640215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:28.640215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:29:28.640215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:28.640215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:29:28.640215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:29:28.640215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:28.640215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:29:28.640215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:28.640215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:29:28.640633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:28.640633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:29:28.640633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:29:28.640633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:29:28.640633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:29:28.640633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:29:28.640633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:29:28.640633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:29:28.640633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:29:28.640633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:29:28.640633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:28.640633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:29:28.641076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:28.641076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:29:28.641076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:28.641076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:29:28.641076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:28.641076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:29:28.641076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:28.641076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:29:28.641076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:29:28.641076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:28.641076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:29:28.641076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:28.641076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:29:28.641076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:29:28.641428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:28.641428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:29:28.641428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:28.641428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:29:28.642320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:28.642320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:29:28.642320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:29:28.642320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:29:28.642320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:29:28.642320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:29:28.642320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:29:28.642320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:29:28.642320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:29:28.642320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:29:28.642320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:29:28.642320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:29:28.642320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:29:28.642320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:29:28.642320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:29:28.642320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:29:28.642825 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:29:28.642825 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:29:28.642825 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:29:28.642825 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:29:28.642825 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:29:28.642825 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:28.642825 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:29:28.642825 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:28.642825 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:29:28.642825 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:28.642825 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:29:28.642825 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:28.642825 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:29:28.642825 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:28.642825 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:29:28.642825 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:29:28.642825 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:28.642825 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:29:28.643298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:28.643298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:29:28.643298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:29:28.643298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:28.643298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:29:28.643298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:28.643298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:29:28.643298 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:29:28.643298 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:29:28.643298 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:29:28.643579 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:29:28.643717 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:29:28.644105 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:28.644644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:29:28.644644 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:28.644644 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:29:28.644644 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:29:28.644644 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:28.644644 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:29:28.644644 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:28.644644 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:29:28.645590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:28.645590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:29:28.645590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:29:28.645590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:29:28.645590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:28.645590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:29:28.645590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:28.645590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:29:28.645590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:28.645590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:29:28.645590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:28.645590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:29:28.645590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:28.645590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:29:28.645590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:29:28.646077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:28.646077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:29:28.646077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:28.646077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:29:28.646077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:29:28.646077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:28.646077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:29:28.646077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:28.646077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:29:28.646585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:28.646585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:29:28.646585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:29:28.646585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:29:28.646585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:29:28.646585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:29:28.646585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:29:28.646585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:29:28.646585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:29:28.646585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:28.646585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:29:28.646995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:28.646995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:29:28.646995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:28.646995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:29:28.646995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:28.646995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:29:28.646995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:28.646995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:29:28.646995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:29:28.646995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:28.646995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:29:28.646995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:28.646995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:29:28.646995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:29:28.647360 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:28.647360 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:29:28.647360 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:28.647360 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:29:28.647716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:28.647716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:29:28.647716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:29:28.647716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:29:28.647716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:29:28.647716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:29:28.647716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:29:28.647716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:29:28.647716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:29:28.647716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:29:28.647716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:29:28.647716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:29:28.647716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:29:28.647716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:29:28.647716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:29:28.647716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:29:28.647716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:29:28.648241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:29:28.648241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:29:28.648241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:29:28.648241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:28.648241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:29:28.648241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:28.648241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:29:28.648241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:28.648241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:29:28.648241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:28.648241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:29:28.648241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:28.648241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:29:28.648241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:29:28.648241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:28.648241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:29:28.648241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:28.648241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:29:28.648241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:29:28.648832 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:28.648832 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:29:28.648832 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:28.648832 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:29:28.648832 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:29:28.648832 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:29:28.648832 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:29:28.648832 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:29:28.649129 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:29:28.649419 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:29:28.649589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:28.650188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:29:28.650188 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:28.650188 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:29:28.650188 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:29:28.650188 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:28.650188 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:29:28.650188 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:28.650188 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:29:28.651103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:28.651103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:29:28.651103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:29:28.651103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:29:28.651103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:29:28.651103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:29:28.651103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:29:28.651103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:28.651103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:29:28.651103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:28.651103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:29:28.651103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:28.651468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:29:28.651468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:28.651468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:29:28.651468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:28.651468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:29:28.651468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:29:28.651468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:28.651468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:29:28.651468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:28.651468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:29:28.651468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:29:28.651468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:28.651468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:29:28.651839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:28.651839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:29:28.652233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:28.652233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:29:28.652233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:29:28.652233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:29:28.652233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:29:28.652233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:29:28.652233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:29:28.652233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:29:28.652233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:29:28.652233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:29:28.652233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:29:28.652233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:29:28.652233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:29:28.652233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:29:28.652233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:29:28.652233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:29:28.652877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:29:28.652877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:29:28.652877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:28.652877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:29:28.652877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:28.652877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:29:28.652877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:28.652877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:29:28.652877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:28.652877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:29:28.652877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:28.652877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:29:28.652877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:29:28.652877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:28.652877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:29:28.652877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:28.652877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:29:28.652877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:29:28.652877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:28.653557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:29:28.653557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:28.653557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:29:33.476609 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:29:33.477376 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:29:33.477498 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:29:33.478520 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:29:33.531557 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:29:33.531557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:33.532507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:29:33.532507 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:33.532507 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:29:33.532507 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:29:33.532507 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:33.532507 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:29:33.532507 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:33.532507 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:29:33.534021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.055 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:33.534021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:29:33.534021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:29:33.534021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:29:33.534021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:29:33.534021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:29:33.534021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:29:33.534021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:33.534021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:29:33.534021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:33.534021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:29:33.534021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:33.534753 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:29:33.534753 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:33.534753 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:29:33.534753 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:33.534753 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:29:33.534753 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:29:33.534753 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:33.534753 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:29:33.534753 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:33.534753 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:29:33.534753 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:29:33.534753 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:33.534753 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:29:33.535372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:33.535372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:29:33.536111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:33.536111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:29:33.536111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:29:33.536111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:29:33.536111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:29:33.536111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:29:33.536111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:29:33.536111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:29:33.536111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:29:33.536111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:29:33.536111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:29:33.536111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:29:33.536111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:29:33.536111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:29:33.536111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:29:33.536111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:29:33.537902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:29:33.537902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:29:33.537902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:33.537902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:29:33.537902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:33.537902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:29:33.537902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:33.537902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:29:33.537902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:33.537902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:29:33.537902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:33.537902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:29:33.537902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:29:33.537902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:33.537902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:29:33.537902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:33.537902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:29:33.537902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:29:33.537902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:33.539553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:29:33.539553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:33.539553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:29:33.629704 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:29:33.631224 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:29:33.631436 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:29:33.632907 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:29:33.633200 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:29:33.633337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:33.634074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:29:33.634074 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:33.634074 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:29:33.634074 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:29:33.634074 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:33.634074 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:29:33.634074 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:33.634074 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:29:33.635888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:33.635888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:29:33.635888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:29:33.635888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:29:33.635888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:29:33.635888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:29:33.635888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:29:33.635888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:33.635888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:29:33.635888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:33.635888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:29:33.635888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:33.636427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:29:33.636427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:33.636427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:29:33.636427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:33.636427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:29:33.636427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:29:33.636427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:33.636427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:29:33.636427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:33.636427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:29:33.636427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:29:33.636427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:33.636427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:29:33.637060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:33.637060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:29:33.637594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:33.637594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:29:33.637594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:29:33.637594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:29:33.637594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:29:33.637594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:29:33.637594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:29:33.637594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:29:33.637594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:29:33.637594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:29:33.637594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:29:33.637594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:29:33.637594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:29:33.637594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:29:33.637594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:29:33.637594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:29:33.638070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:29:33.638070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:29:33.638070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:33.638070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:29:33.638070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:33.638070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:29:33.638070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:33.638070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:29:33.638070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:33.638070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:29:33.638070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:33.638070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:29:33.638070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:29:33.638070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:33.638070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:29:33.638070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:33.638070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:29:33.638070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:29:33.638070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:33.638605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:29:33.638605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:33.638605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:29:33.638605 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:29:33.638605 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:29:33.638605 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:29:33.638605 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:29:33.638964 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:29:33.639137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:33.639655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:29:33.639655 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:33.639655 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:29:33.639655 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:29:33.639655 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:33.639655 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:29:33.639655 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:33.639655 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:29:33.640427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:33.640427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:29:33.640427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:29:33.640427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:29:33.640427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:33.640427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:29:33.640427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:33.640427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:29:33.640427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:33.640427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:29:33.640427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:33.640427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:29:33.640427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:33.640427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:29:33.640427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:29:33.640866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:33.640866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:29:33.640866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:33.640866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:29:33.640866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:29:33.640866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:33.640866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:29:33.640866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:33.640866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:29:33.641320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:33.641320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:29:33.641320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:29:33.641320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:29:33.641320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:29:33.641320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:29:33.641320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:29:33.641320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:29:33.641320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:29:33.641320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:29:33.641320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:33.641320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:29:33.641891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:33.641891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:29:33.641891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:33.641891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:29:33.641891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:33.641891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:29:33.641891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:33.641891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:29:33.641891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:29:33.641891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:33.641891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:29:33.641891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:33.641891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:29:33.641891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:29:33.642263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:33.642263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:29:33.642263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:33.642263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:29:33.642395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:33.642395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:29:33.642395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:29:33.642395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:29:33.642395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:29:33.642395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:29:33.642395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:29:33.642395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:29:33.642395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:29:33.642395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:29:33.642395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:29:33.642395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:29:33.642395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:29:33.642395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:29:33.642395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:29:33.642395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:29:33.642875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:29:33.642875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:29:33.642875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:29:33.642875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:29:33.642875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:29:33.642875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:33.642875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:29:33.642875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:33.642875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:29:33.642875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:33.642875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:29:33.642875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:33.642875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:29:33.642875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:33.642875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:29:33.642875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:29:33.642875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:33.642875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:29:33.643366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:33.643366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:29:33.643366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:29:33.643366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:33.643366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:29:33.643366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:33.643366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:29:33.643366 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:29:33.643579 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:29:33.643579 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:29:33.643731 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:29:33.644158 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:29:33.644546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:33.645349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:29:33.645349 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:33.645349 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:29:33.645349 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:29:33.645349 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:33.645349 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:29:33.645349 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:33.645349 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:29:33.646140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:33.646140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:29:33.646140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:29:33.646140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:29:33.646140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:33.646140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:29:33.646140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:33.646140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:29:33.646140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:33.646140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:29:33.646140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:33.646140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:29:33.646140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:33.646140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:29:33.646140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:29:33.646653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:33.646653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:29:33.646653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:33.646653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:29:33.646653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:29:33.646653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:33.646653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:29:33.646653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:33.646653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:29:33.647088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:33.647088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:29:33.647088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:29:33.647088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:29:33.647088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:29:33.647088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:29:33.647088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:29:33.647088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:29:33.647088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:29:33.647088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:33.647088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:29:33.647480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:33.647480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:29:33.647480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:33.647480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:29:33.647480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:33.647480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:29:33.647480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:33.647480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:29:33.647480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:29:33.647480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:33.647480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:29:33.647480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:33.647480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:29:33.647480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:29:33.647971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:33.647971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:29:33.647971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:33.647971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:29:33.648436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:33.648436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:29:33.648436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:29:33.648436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:29:33.648436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:29:33.648436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:29:33.648436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:29:33.648436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:29:33.648436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:29:33.648436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:29:33.648436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:29:33.648436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:29:33.648436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:29:33.648436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:29:33.648436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:29:33.648436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:29:33.648436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:29:33.648997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:29:33.648997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:29:33.648997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:29:33.648997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:33.648997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:29:33.648997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:33.648997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:29:33.648997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:33.648997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:29:33.648997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:33.648997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:29:33.648997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:33.648997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:29:33.648997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:29:33.648997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:33.648997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:29:33.648997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:33.648997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:29:33.648997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:29:33.649897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:33.649897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:29:33.649897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:33.649897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:29:33.649897 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:29:33.650084 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:29:33.651389 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:29:33.654904 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:29:33.655264 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:29:33.655440 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:29:33.655565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:33.656072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:29:33.656072 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:33.656072 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:29:33.656072 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:29:33.656072 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:33.656072 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:29:33.656072 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:33.656072 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:29:33.656898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:33.656898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:29:33.656898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:29:33.656898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:29:33.656898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:29:33.656898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:29:33.656898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:29:33.656898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:33.656898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:29:33.656898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:33.656898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:29:33.656898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:33.657324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:29:33.657324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:33.657324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:29:33.657324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:33.657324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:29:33.657324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:29:33.657324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:33.657324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:29:33.657324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:33.657324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:29:33.657324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:29:33.657324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:33.657324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:29:33.657856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:33.657856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:29:33.659027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:33.659027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:29:33.659027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:29:33.659027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:29:33.659027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:29:33.659027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:29:33.659027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:29:33.659027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:29:33.659027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:29:33.659027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:29:33.659027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:29:33.659027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:29:33.659027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:29:33.659027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:29:33.659027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:29:33.659027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:29:33.659466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:29:33.659466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:29:33.659466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:33.659466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:29:33.659466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:33.659466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:29:33.659466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:33.659466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:29:33.659466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:33.659466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:29:33.659466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:33.659466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:29:33.659466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:29:33.659466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:33.659466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:29:33.659466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:33.659466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:29:33.659466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:29:33.659466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:33.660018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:29:33.660018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:33.660018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:29:38.478075 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:29:38.479020 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:29:38.479020 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:29:38.480422 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:29:38.515074 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:29:38.515944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:38.516490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:29:38.516490 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:38.516490 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:29:38.516490 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:29:38.516490 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:38.516490 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:29:38.516490 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:38.516490 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:29:38.517382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:38.517382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:29:38.517382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:29:38.517382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:29:38.517382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:29:38.517382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:29:38.517382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:29:38.517382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:38.517382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:29:38.517382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:38.517382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:29:38.517382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:38.518038 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:29:38.518038 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:38.518038 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:29:38.518038 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:38.518038 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:29:38.518038 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:29:38.518038 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:38.518038 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:29:38.518038 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:38.518038 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:29:38.518038 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:29:38.518038 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:38.518038 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:29:38.518628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:38.518628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:29:38.518628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:38.518628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:29:38.518628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:29:38.518628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:29:38.518628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:29:38.518628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:29:38.518628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:29:38.518628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:29:38.518628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:29:38.518628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:29:38.518628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:29:38.518628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:29:38.518628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:29:38.519431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:29:38.519431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:29:38.519431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:29:38.519431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:29:38.519431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:29:38.519431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:38.519431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:29:38.519431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:38.519431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:29:38.519431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:38.519431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:29:38.519431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:38.519431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:29:38.519431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:38.519431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:29:38.519431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:29:38.519431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:38.519431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:29:38.519431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:38.520480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:29:38.520480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:29:38.520480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:38.520480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:29:38.520480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:38.520480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:29:38.631300 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:29:38.632053 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:29:38.632211 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:29:38.633349 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:29:38.633774 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:29:38.634077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:38.635005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:29:38.635005 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:38.635005 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:29:38.635005 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:29:38.635005 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:38.635005 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:29:38.635005 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:38.635005 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:29:38.636180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:38.636180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:29:38.636180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:29:38.636180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:29:38.636180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:29:38.636180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:29:38.636180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:29:38.636180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:38.636180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:29:38.636180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:38.636180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:29:38.636180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:38.636795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:29:38.636795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:38.636795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:29:38.636795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:38.636795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:29:38.636795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:29:38.636795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:38.636795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:29:38.636795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:38.636795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:29:38.636795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:29:38.636795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:38.636795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:29:38.637369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:38.637369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:29:38.637369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:38.637369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:29:38.637369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:29:38.637369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:29:38.637369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:29:38.637369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:29:38.637369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:29:38.637369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:29:38.637369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:29:38.637369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:29:38.637369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:29:38.637369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:29:38.637369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:29:38.638126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:29:38.638126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:29:38.638126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:29:38.638126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:29:38.638126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:29:38.638126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:38.638126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:29:38.638126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:38.638126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:29:38.638126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:38.638126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:29:38.638126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:38.638126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:29:38.638126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:38.638126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:29:38.638126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:29:38.638126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:38.638126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:29:38.638126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:38.638710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:29:38.638710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:29:38.638710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:38.638710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:29:38.638710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:38.638710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:29:38.638710 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:29:38.639157 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:29:38.639351 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:29:38.639564 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:29:38.640128 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:29:38.640434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:38.641252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:29:38.641252 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:38.641252 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:29:38.641252 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:29:38.641252 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:38.641252 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:29:38.641252 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:38.641252 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:29:38.642097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:38.642097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:29:38.642097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:29:38.642097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:29:38.642097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:38.642097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:29:38.642097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:38.642097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:29:38.642097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:38.642097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:29:38.642097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:38.642097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:29:38.642097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:38.642097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:29:38.642097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:29:38.642998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:38.642998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:29:38.642998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:38.642998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:29:38.642998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:29:38.642998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:38.642998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:29:38.642998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:38.642998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:29:38.642998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:38.642998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:29:38.642998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:29:38.642998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:29:38.642998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:29:38.643730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:29:38.643730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:29:38.643730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:29:38.643730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:29:38.643730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:29:38.643730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:38.643730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:29:38.643730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:38.643730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:29:38.643730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:38.643730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:29:38.643730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:38.643730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:29:38.644394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:38.644394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:29:38.644394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:29:38.644394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:38.644394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:29:38.644394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:38.644394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:29:38.644394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:29:38.644394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:38.644394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:29:38.644394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:38.644394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:29:38.645538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:38.645538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:29:38.645538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:29:38.645538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:29:38.645538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:29:38.645538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:29:38.645538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:29:38.645538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:29:38.645538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:29:38.645538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:29:38.645538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:29:38.645538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:29:38.645538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:29:38.645538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:29:38.645538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:29:38.645538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:29:38.646346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:29:38.646346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:29:38.646346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:29:38.646346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:29:38.646346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:29:38.646346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:38.646346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:29:38.646346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:38.646346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:29:38.646346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:38.646346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:29:38.646346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:38.646346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:29:38.646346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:38.646346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:29:38.646346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:29:38.646346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:38.646346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:29:38.647206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:38.647206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:29:38.647206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:29:38.647206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:38.647206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:29:38.647206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:38.647206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:29:38.647206 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:29:38.647206 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:29:38.647206 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:29:38.647688 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:29:38.647688 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:29:38.647688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:38.647688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:29:38.647688 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:38.647688 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:29:38.647688 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:29:38.647688 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:38.647688 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:29:38.647688 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:38.647688 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:29:38.648481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:38.648481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:29:38.648481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:29:38.648481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:29:38.648481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:38.648481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:29:38.648481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:38.648481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:29:38.648481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:38.648481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:29:38.648481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:38.648481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:29:38.648481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:38.648481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:29:38.648481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:29:38.649185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:38.649185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:29:38.649185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:38.649185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:29:38.649185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:29:38.649185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:38.649185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:29:38.649185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:38.649185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:29:38.649598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:38.649598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:29:38.649598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:29:38.649598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:29:38.649598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:29:38.649598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:29:38.649598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:29:38.649598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:29:38.649598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:29:38.649598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:38.649598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:29:38.650176 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:38.650176 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:29:38.650176 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:38.650176 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:29:38.650176 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:38.650176 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:29:38.650176 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:38.650176 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:29:38.650176 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:29:38.650176 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:38.650176 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:29:38.650176 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:38.650176 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:29:38.650176 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:29:38.650525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:38.650525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:29:38.650525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:38.650525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:29:38.652002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:38.652002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:29:38.652002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:29:38.652002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:29:38.652002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:29:38.652002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:29:38.652002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:29:38.652002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:29:38.652002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:29:38.652002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:29:38.652002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:29:38.652002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:29:38.652002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:29:38.652002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:29:38.652002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:29:38.652002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:29:38.652002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:29:38.652359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:29:38.652359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:29:38.652359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:29:38.652359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:38.652359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:29:38.652359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:38.652359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:29:38.652359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:38.652359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:29:38.652359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:38.652359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:29:38.652359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:38.652359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:29:38.652359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:29:38.652359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:38.652359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:29:38.652359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:38.652359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:29:38.652359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:29:38.652692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:38.652692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:29:38.652692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:38.652692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:29:38.652692 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:29:38.652692 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:29:38.652692 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:29:38.652862 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:29:38.653359 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:29:38.653708 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:29:38.654025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:38.654754 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:29:38.654754 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:38.654754 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:29:38.654754 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:29:38.654754 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:38.654754 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:29:38.654754 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:38.654754 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:29:38.655687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:38.655687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:29:38.655687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:29:38.655687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:29:38.655687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:29:38.655687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:29:38.655687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:29:38.655687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:38.655687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:29:38.655687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:38.655687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:29:38.655687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:38.655944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:29:38.655944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:38.655944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:29:38.655944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:38.655944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:29:38.655944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:29:38.655944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:38.655944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:29:38.655944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:38.655944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:29:38.655944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:29:38.655944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:38.655944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:29:38.656174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:38.656174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:29:38.656771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:38.656771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:29:38.656771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:29:38.656771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:29:38.656771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:29:38.656771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:29:38.656771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:29:38.656771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:29:38.656771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:29:38.656771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:29:38.656771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:29:38.656771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:29:38.656771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:29:38.656771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:29:38.656771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:29:38.656771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:29:38.657076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:29:38.657076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:29:38.657076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:38.657076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:29:38.657076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:38.657076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:29:38.657076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:38.657076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:29:38.657076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:38.657076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:29:38.657076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:38.657076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:29:38.657076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:29:38.657076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:38.657076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:29:38.657076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:38.657076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:29:38.657076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:29:38.657076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:38.657418 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:29:38.657418 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:38.657418 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:29:43.478672 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:29:43.479244 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:29:43.479244 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:29:43.479990 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:29:43.518291 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:29:43.518705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:43.519775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:29:43.519775 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:43.519775 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:29:43.519775 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:29:43.519775 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:43.519775 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:29:43.519775 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:43.519775 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:29:43.520911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:43.520911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:29:43.520911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:29:43.520911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:29:43.520911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:29:43.520911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:29:43.520911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:29:43.520911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:43.520911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:29:43.520911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:43.520911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:29:43.520911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:43.521317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:29:43.521317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:43.521317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:29:43.521317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:43.521317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:29:43.521317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:29:43.521317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:43.521317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:29:43.521317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:43.521317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:29:43.521317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:29:43.521317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:43.521317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:29:43.521764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:43.521764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:29:43.522401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:43.522401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:29:43.522401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:29:43.522401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:29:43.522401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:29:43.522401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:29:43.522401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:29:43.522401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:29:43.522401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:29:43.522401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:29:43.522401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:29:43.522401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:29:43.522401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:29:43.522401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:29:43.522401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:29:43.522401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:29:43.522936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:29:43.522936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:29:43.522936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:43.522936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:29:43.522936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:43.522936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:29:43.522936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:43.522936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:29:43.522936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:43.522936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:29:43.522936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:43.522936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:29:43.522936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:29:43.522936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:43.522936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:29:43.522936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:43.522936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:29:43.522936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:29:43.522936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:43.523466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:29:43.523466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:43.523466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:29:43.632041 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:29:43.632413 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:29:43.632547 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:29:43.634555 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:29:43.634555 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:29:43.634555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:43.635532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:29:43.635532 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:43.635532 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:29:43.635532 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:29:43.635532 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:43.635532 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:29:43.635532 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:43.635532 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:29:43.636973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:43.636973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:29:43.636973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:29:43.636973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:29:43.636973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:29:43.636973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:29:43.636973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:29:43.636973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:43.636973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:29:43.636973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:43.636973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:29:43.636973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:43.637732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:29:43.637732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:43.637732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:29:43.637732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:43.637732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:29:43.637732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:29:43.637732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:43.637732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:29:43.637732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:43.637732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:29:43.637732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:29:43.637732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:43.637732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:29:43.638328 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:43.638328 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:29:43.639062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:43.639062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:29:43.639062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:29:43.639062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:29:43.639062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:29:43.639062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:29:43.639062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:29:43.639062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:29:43.639062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:29:43.639062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:29:43.639062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:29:43.639062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:29:43.639062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:29:43.639062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:29:43.639062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:29:43.639062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:29:43.639939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:29:43.639939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:29:43.639939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:43.639939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:29:43.639939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:43.639939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:29:43.639939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:43.639939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:29:43.639939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:43.639939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:29:43.639939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:43.639939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:29:43.639939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:29:43.639939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:43.639939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:29:43.639939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:43.639939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:29:43.639939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:29:43.639939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:43.640492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:29:43.640492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:43.640492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:29:43.640492 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:29:43.640492 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:29:43.640492 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:29:43.640680 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:29:43.641265 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:29:43.641374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:43.641965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:29:43.641965 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:43.641965 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:29:43.641965 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:29:43.641965 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:43.641965 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:29:43.641965 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:43.641965 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:29:43.642799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:43.642799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:29:43.642799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:29:43.642799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:29:43.642799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:43.642799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:29:43.642799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:43.642799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:29:43.642799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:43.642799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:29:43.642799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:43.642799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:29:43.642799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:43.642799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:29:43.642799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:29:43.643303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:43.643303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:29:43.643303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:43.643303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:29:43.643303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:29:43.643303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:43.643303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:29:43.643303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:43.643303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:29:43.643659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:43.643659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:29:43.643659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:29:43.643659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:29:43.643659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:29:43.643659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:29:43.643659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:29:43.643659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:29:43.643659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:29:43.643659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:29:43.643659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:43.643659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:29:43.644043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:43.644043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:29:43.644043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:43.644043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:29:43.644043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:43.644043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:29:43.644043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:43.644043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:29:43.644043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:29:43.644043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:43.644043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:29:43.644043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:43.644043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:29:43.644043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:29:43.644468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:43.644468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:29:43.644468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:43.644468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:29:43.644873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:43.644873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:29:43.644873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:29:43.644873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:29:43.644873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:29:43.644873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:29:43.644873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:29:43.644873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:29:43.644873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:29:43.644873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:29:43.644873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:29:43.644873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:29:43.644873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:29:43.644873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:29:43.644873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:29:43.644873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:29:43.645375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:29:43.645375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:29:43.645375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:29:43.645375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:29:43.645375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:29:43.645375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:43.645375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:29:43.645375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:43.645375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:29:43.645375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:43.645375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:29:43.645375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:43.645375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:29:43.645375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:43.645375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:29:43.645375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:29:43.645375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:43.645375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:29:43.645886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:43.645886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:29:43.645886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:29:43.645886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:43.645886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:29:43.645886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:43.645886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:29:43.646176 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:29:43.646443 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:29:43.646573 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:29:43.646754 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:29:43.647189 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:29:43.647490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:43.648092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:29:43.648092 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:43.648092 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:29:43.648092 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:29:43.648092 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:43.648092 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:29:43.648092 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:43.648092 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:29:43.649032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:43.649032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:29:43.649032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:29:43.649032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:29:43.649032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:43.649032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:29:43.649032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:43.649032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:29:43.649032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:43.649032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:29:43.649032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:43.649032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:29:43.649032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:43.649032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:29:43.649032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:29:43.649495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:43.649495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:29:43.649495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:43.649495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:29:43.649495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:29:43.649495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:43.649495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:29:43.649495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:43.649495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:29:43.650251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:43.650251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:29:43.650251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:29:43.650251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:29:43.650251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:29:43.650251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:29:43.650251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:29:43.650251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:29:43.650251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:29:43.650251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:43.650251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:29:43.650589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:43.650589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:29:43.650589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:43.650589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:29:43.650589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:43.650589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:29:43.650589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:43.650589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:29:43.650589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:29:43.650589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:43.650589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:29:43.650589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:43.650589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:29:43.650589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:29:43.650989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:43.650989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:29:43.650989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:43.650989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:29:43.651728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:43.651728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:29:43.651728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:29:43.651728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:29:43.651728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:29:43.651728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:29:43.651728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:29:43.651728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:29:43.651728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:29:43.651728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:29:43.651728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:29:43.651728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:29:43.651728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:29:43.651728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:29:43.651728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:29:43.651728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:29:43.651728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:29:43.652284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:29:43.652284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:29:43.652284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:29:43.652284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:43.652284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:29:43.652284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:43.652284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:29:43.652284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:43.652284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:29:43.652284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:43.652284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:29:43.652284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:43.652284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:29:43.652284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:29:43.652284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:43.652284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:29:43.652284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:43.652284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:29:43.652284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:29:43.652858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:43.652858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:29:43.652858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:43.652858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:29:43.652858 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:29:43.652858 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:29:43.653063 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:29:43.653063 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:29:43.653446 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:29:43.653726 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:29:43.653952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:43.654552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:29:43.654552 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:43.654552 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:29:43.654552 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:29:43.654552 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:43.654552 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:29:43.654552 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:43.654552 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:29:43.656460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:43.656460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:29:43.656460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:29:43.656460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:29:43.656460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:29:43.656460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:29:43.656460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:29:43.656460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:43.656460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:29:43.656460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:43.656460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:29:43.656460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:43.656834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:29:43.656834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:43.656834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:29:43.656834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:43.656834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:29:43.656834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:29:43.656834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:43.656834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:29:43.656834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:43.656834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:29:43.656834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:29:43.656834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:43.656834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:29:43.657220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:43.657220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:29:43.657527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:43.657527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:29:43.657527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:29:43.657527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:29:43.657527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:29:43.657527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:29:43.657527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:29:43.657527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:29:43.657527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:29:43.657527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:29:43.657527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:29:43.657527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:29:43.657527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:29:43.657527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:29:43.657527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:29:43.657527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:29:43.657965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:29:43.657965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:29:43.657965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:43.657965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:29:43.657965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:43.657965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:29:43.657965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:43.657965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:29:43.657965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:43.657965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:29:43.657965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:43.657965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:29:43.657965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:29:43.657965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:43.657965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:29:43.657965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:43.657965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:29:43.657965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:29:43.657965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:43.658506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:29:43.658506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:43.658506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:29:47.981788 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_for_inconsistencies' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:29:47.982945 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_for_inconsistencies {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:29:47.982945 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Checking Neutron and OVN revision consistency. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:29:47.992156 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Maintenance task: Synchronizing Neutron and OVN databases started {{(pid=106314) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:440}}
May 09 09:29:47.992344 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Maintenance task: Number of inconsistencies found at create/update: networks=4, routers=1, router_ports=1 {{(pid=106314) _log /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:419}}
May 09 09:29:47.992344 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Maintenance task: Fixing resource f8a38fcb-e8a2-421a-a340-bf7f2f5cc72b (type: networks) at create/update {{(pid=106314) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 09 09:29:48.023398 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-f8a38fcb-e8a2-421a-a340-bf7f2f5cc72b, resource={'id': 'f8a38fcb-e8a2-421a-a340-bf7f2f5cc72b', 'name': 'private', 'tenant_id': 'f4d54dfe001d43fbb344093a1ccabd6a', 'project_id': 'f4d54dfe001d43fbb344093a1ccabd6a', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['96e72a03-97b8-412a-b2b9-1bf048ce1066', 'bbd6957a-953f-4fc1-8761-5d93128564d9'], 'standard_attr_id': 12, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-05-09T09:04:34Z', 'updated_at': '2026-05-09T09:04:37Z', 'revision_number': 3, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 3730}, resource_type=networks, if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:29:48.023733 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-f8a38fcb-e8a2-421a-a340-bf7f2f5cc72b, col_values=(('external_ids', {'neutron:network_name': 'private', 'neutron:mtu': '1372', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:29:48.024335 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-78a32c7c-7637-4b83-b0ab-62aee8bb3038, external_ids_update=None, columns={'tag': 3730}, if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:29:48.024508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:48.025217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:29:48.025217 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:48.025217 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:29:48.025217 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:29:48.025217 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:48.025217 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:29:48.025217 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:48.025217 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:29:48.026453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Maintenance task: Failed to fix resource f8a38fcb-e8a2-421a-a340-bf7f2f5cc72b (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:48.026453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:29:48.026453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 09 09:29:48.026453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 09 09:29:48.026453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 09 09:29:48.026453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 09 09:29:48.026453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2332, in update_network
May 09 09:29:48.026453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:29:48.026453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:48.026453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:29:48.026453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:48.026894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:29:48.026894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:48.026894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:29:48.026894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:48.026894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:29:48.026894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:48.026894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:29:48.026894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:29:48.026894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:48.026894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:29:48.026894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:48.026894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:29:48.026894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:29:48.026894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:48.027607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:29:48.027607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:48.027607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:29:48.027607 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Maintenance task: Fixing resource d18a9fac-2ebf-4d1e-a86c-0f33fba8eeda (type: networks) at create/update {{(pid=106314) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 09 09:29:48.056789 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-d18a9fac-2ebf-4d1e-a86c-0f33fba8eeda, resource={'id': 'd18a9fac-2ebf-4d1e-a86c-0f33fba8eeda', 'name': 'public', 'tenant_id': 'f4d54dfe001d43fbb344093a1ccabd6a', 'project_id': 'f4d54dfe001d43fbb344093a1ccabd6a', 'admin_state_up': True, 'mtu': 1430, 'status': 'ACTIVE', 'subnets': ['088bbd0f-7e81-4a76-8399-24d15c984009', 'bbd6a8cc-80e6-4336-a70b-1c962c561590'], 'standard_attr_id': 18, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': True, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'is_default': True, 'tags': [], 'created_at': '2026-05-09T09:04:40Z', 'updated_at': '2026-05-09T09:04:50Z', 'revision_number': 3, 'provider:network_type': 'flat', 'provider:physical_network': 'public', 'provider:segmentation_id': None}, resource_type=networks, if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:29:48.057227 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-d18a9fac-2ebf-4d1e-a86c-0f33fba8eeda, col_values=(('external_ids', {'neutron:network_name': 'public', 'neutron:mtu': '1430', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'flat', 'neutron:provnet-physical-network': 'public'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false', 'fdb_age_threshold': '300', 'broadcast-arps-to-all-routers': 'true'})), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:29:48.057947 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-1163f525-70cd-4b84-ae9b-7d56378305e9, external_ids_update=None, columns={'tag': []}, if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:29:48.058441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:48.059520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:29:48.059520 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:48.059520 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:29:48.059520 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:29:48.059520 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:48.059520 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:29:48.059520 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:48.059520 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:29:48.060761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Maintenance task: Failed to fix resource d18a9fac-2ebf-4d1e-a86c-0f33fba8eeda (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:48.060761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:29:48.060761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 09 09:29:48.060761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 09 09:29:48.060761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 09 09:29:48.060761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 09 09:29:48.060761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2332, in update_network
May 09 09:29:48.060761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:29:48.060761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:48.060761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:29:48.060761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:48.061116 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:29:48.061116 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:48.061116 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:29:48.061116 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:48.061116 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:29:48.061116 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:48.061116 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:29:48.061116 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:29:48.061116 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:48.061116 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:29:48.061116 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:48.061116 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:29:48.061116 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:29:48.061116 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:48.061465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:29:48.061465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:48.061465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:29:48.061465 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Maintenance task: Fixing resource 417da935-2882-452f-8ba7-9076d68fcd2b (type: networks) at create/update {{(pid=106314) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 09 09:29:48.093652 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-417da935-2882-452f-8ba7-9076d68fcd2b, resource={'id': '417da935-2882-452f-8ba7-9076d68fcd2b', 'name': 'admin_net', 'tenant_id': '38e536cfc9c94b6cbf529caf55cc7739', 'project_id': '38e536cfc9c94b6cbf529caf55cc7739', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['d3c53d14-f490-4615-a986-3715a315f827'], 'standard_attr_id': 42, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-05-09T09:10:38Z', 'updated_at': '2026-05-09T09:10:39Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 32823}, resource_type=networks, if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:29:48.094014 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-417da935-2882-452f-8ba7-9076d68fcd2b, col_values=(('external_ids', {'neutron:network_name': 'admin_net', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:29:48.094471 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-5738ab2a-b471-447b-9292-7f8c6bc97168, external_ids_update=None, columns={'tag': 32823}, if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:29:48.094620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:48.095559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:29:48.095559 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:48.095559 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:29:48.095559 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:29:48.095559 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:48.095559 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:29:48.095559 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:48.095559 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:29:48.096952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Maintenance task: Failed to fix resource 417da935-2882-452f-8ba7-9076d68fcd2b (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:48.096952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:29:48.096952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 09 09:29:48.096952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 09 09:29:48.096952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 09 09:29:48.096952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 09 09:29:48.096952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2332, in update_network
May 09 09:29:48.096952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:29:48.096952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:48.096952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:29:48.096952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:48.097264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:29:48.097264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:48.097264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:29:48.097264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:48.097264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:29:48.097264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:48.097264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:29:48.097264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:29:48.097264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:48.097264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:29:48.097264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:48.097264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:29:48.097264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:29:48.097264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:48.097636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:29:48.097636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:48.097636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:29:48.097636 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Maintenance task: Fixing resource 433617e8-cdf8-414a-8cc6-63a7b5d0c678 (type: networks) at create/update {{(pid=106314) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 09 09:29:48.127834 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-433617e8-cdf8-414a-8cc6-63a7b5d0c678, resource={'id': '433617e8-cdf8-414a-8cc6-63a7b5d0c678', 'name': 'k8s-clusterapi-cluster-magnum-system-kube-whtb8', 'tenant_id': '3e013d45a44544c3b50eb0e83e4b1efc', 'project_id': '3e013d45a44544c3b50eb0e83e4b1efc', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['9187f065-9264-427a-8df8-672a6981191d'], 'standard_attr_id': 52, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-05-09T09:11:20Z', 'updated_at': '2026-05-09T09:11:20Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 32644}, resource_type=networks, if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:29:48.128285 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-433617e8-cdf8-414a-8cc6-63a7b5d0c678, col_values=(('external_ids', {'neutron:network_name': 'k8s-clusterapi-cluster-magnum-system-kube-whtb8', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:29:48.128866 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-8c3fe586-c182-47ba-9845-d7608113cb71, external_ids_update=None, columns={'tag': 32644}, if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:29:48.129056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:48.129978 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:29:48.129978 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:48.129978 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:29:48.129978 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:29:48.129978 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:48.129978 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:29:48.129978 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:48.129978 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:29:48.131473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Maintenance task: Failed to fix resource 433617e8-cdf8-414a-8cc6-63a7b5d0c678 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:48.131473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:29:48.131473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 09 09:29:48.131473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 09 09:29:48.131473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 09 09:29:48.131473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 09 09:29:48.131473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2332, in update_network
May 09 09:29:48.131473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:29:48.131473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:48.131473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:29:48.131473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:48.131961 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:29:48.131961 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:48.131961 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:29:48.131961 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:48.131961 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:29:48.131961 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:48.131961 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:29:48.131961 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:29:48.131961 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:48.131961 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:29:48.131961 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:48.131961 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:29:48.131961 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:29:48.131961 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:48.132976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:29:48.132976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:48.132976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:29:48.132976 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Maintenance task: Fixing resource cab10384-14c5-492c-8bd0-061f5838310c (type: routers) at create/update {{(pid=106314) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 09 09:29:48.209649 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-cab10384-14c5-492c-8bd0-061f5838310c, resource={'id': 'cab10384-14c5-492c-8bd0-061f5838310c', 'name': 'k8s-clusterapi-cluster-magnum-system-kube-whtb8', 'project_id': '3e013d45a44544c3b50eb0e83e4b1efc', 'admin_state_up': True, 'status': 'ACTIVE', 'enable_snat': True, 'external_gateway_info': {'network_id': 'd18a9fac-2ebf-4d1e-a86c-0f33fba8eeda', 'external_fixed_ips': [{'subnet_id': 'bbd6a8cc-80e6-4336-a70b-1c962c561590', 'ip_address': '172.24.5.116'}, {'subnet_id': '088bbd0f-7e81-4a76-8399-24d15c984009', 'ip_address': '2001:db8::1b7'}], 'enable_snat': True}, 'gw_port_id': 'af118f11-dc01-429a-b20a-e97b77b8d5ec', 'description': 'Created by cluster-api-provider-openstack cluster magnum-system-kube-whtb8', 'availability_zones': [], 'distributed': False, 'ha': True, 'ha_vr_id': 0, 'availability_zone_hints': [], 'enable_default_route_ecmp': False, 'enable_default_route_bfd': False, 'external_gateways': [{'network_id': 'd18a9fac-2ebf-4d1e-a86c-0f33fba8eeda', 'external_fixed_ips': [{'ip_address': '172.24.5.116', 'subnet_id': 'bbd6a8cc-80e6-4336-a70b-1c962c561590'}, {'ip_address': '2001:db8::1b7', 'subnet_id': '088bbd0f-7e81-4a76-8399-24d15c984009'}]}], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2026-05-09T09:11:21Z', 'updated_at': '2026-05-09T09:11:24Z', 'revision_number': 3, 'tenant_id': '3e013d45a44544c3b50eb0e83e4b1efc'}, resource_type=routers, if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:29:48.210188 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterCommand(_result=None, name=neutron-cab10384-14c5-492c-8bd0-061f5838310c, columns={'external_ids': {'neutron:router_name': 'k8s-clusterapi-cluster-magnum-system-kube-whtb8', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': ''}, 'enabled': True}, if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:29:48.210573 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): QoSDelCommand(_result=None, switch=neutron-d18a9fac-2ebf-4d1e-a86c-0f33fba8eeda, conditions=[('direction', '=', 'from-lport'), ('priority', '=', 2002), ('match', '=', 'inport == "af118f11-dc01-429a-b20a-e97b77b8d5ec"')], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:29:48.210853 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): QoSDelCommand(_result=None, switch=neutron-d18a9fac-2ebf-4d1e-a86c-0f33fba8eeda, conditions=[('direction', '=', 'to-lport'), ('priority', '=', 2002), ('match', '=', 'outport == "af118f11-dc01-429a-b20a-e97b77b8d5ec"')], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:29:48.211065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:48.212172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:29:48.212172 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:48.212172 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:29:48.212172 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:29:48.212172 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:48.212172 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:29:48.212172 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:48.212172 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:29:48.212419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_client [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Unable to update router cab10384-14c5-492c-8bd0-061f5838310c. Error: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:48.214233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Maintenance task: Failed to fix resource cab10384-14c5-492c-8bd0-061f5838310c (type: routers): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:48.214233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:29:48.214233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 09 09:29:48.214233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 09 09:29:48.214233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 09 09:29:48.214233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 09 09:29:48.214233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1635, in update_router
May 09 09:29:48.214233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with excutils.save_and_reraise_exception():
May 09 09:29:48.214233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 271, in __exit__
May 09 09:29:48.214233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.force_reraise()
May 09 09:29:48.214233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 233, in force_reraise
May 09 09:29:48.214562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise self.value
May 09 09:29:48.214562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1569, in update_router
May 09 09:29:48.214562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:29:48.214562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:48.214562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:29:48.214562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:48.214562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:29:48.214562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:48.214562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:29:48.214562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:48.214562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:29:48.214562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:48.214562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:29:48.214562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:29:48.214920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:48.214920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:29:48.214920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:48.214920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:29:48.214920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:29:48.214920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:48.214920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:29:48.214920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:48.214920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:29:48.214920 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Maintenance task: Fixing resource af118f11-dc01-429a-b20a-e97b77b8d5ec (type: router_ports) at create/update {{(pid=106314) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 09 09:29:48.331423 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=lrp-af118f11-dc01-429a-b20a-e97b77b8d5ec, resource={'id': 'af118f11-dc01-429a-b20a-e97b77b8d5ec', 'name': '', 'network_id': 'd18a9fac-2ebf-4d1e-a86c-0f33fba8eeda', 'project_id': '', 'mac_address': 'fa:16:3e:51:04:94', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'cab10384-14c5-492c-8bd0-061f5838310c', 'device_owner': 'network:router_gateway', 'standard_attr_id': 57, 'fixed_ips': [{'subnet_id': 'bbd6a8cc-80e6-4336-a70b-1c962c561590', 'ip_address': '172.24.5.116'}, {'subnet_id': '088bbd0f-7e81-4a76-8399-24d15c984009', 'ip_address': '2001:db8::1b7'}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'np0000173033', 'binding:vif_type': 'ovs', 'binding:vif_details': {'port_filter': True, 'connectivity': 'l2', 'ovs_create_tap': False, 'bridge_name': 'br-int', 'datapath_type': 'system', 'bound_drivers': {'0': 'ovn'}}, 'port_security_enabled': False, 'tags': [], 'created_at': '2026-05-09T09:11:21Z', 'updated_at': '2026-05-09T09:11:24Z', 'revision_number': 5, 'tenant_id': ''}, resource_type=router_ports, if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:29:48.331610 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterPortCommand(_result=None, name=lrp-af118f11-dc01-429a-b20a-e97b77b8d5ec, columns={'external_ids': {'neutron:revision_number': '5', 'neutron:subnet_ids': 'bbd6a8cc-80e6-4336-a70b-1c962c561590 088bbd0f-7e81-4a76-8399-24d15c984009', 'neutron:network_name': 'neutron-d18a9fac-2ebf-4d1e-a86c-0f33fba8eeda', 'neutron:is_ext_gw': 'True', 'neutron:router_name': 'neutron-cab10384-14c5-492c-8bd0-061f5838310c'}, 'options': {'reside-on-redirect-chassis': 'true', 'gateway_mtu': '1372'}, 'networks': ['172.24.5.116/24', '2001:db8::1b7/64'], 'ipv6_ra_configs': {}}, if_exists=False) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:29:48.331928 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLRouterPortInLSwitchPortCommand(_result=None, lswitch_port=af118f11-dc01-429a-b20a-e97b77b8d5ec, lrouter_port=lrp-af118f11-dc01-429a-b20a-e97b77b8d5ec, is_gw_port=True, if_exists=True, lsp_address=router) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:29:48.332200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:48.333173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:29:48.333173 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:48.333173 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:29:48.333173 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:29:48.333173 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:48.333173 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:29:48.333173 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:48.333173 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:29:48.335060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Maintenance task: Failed to fix resource af118f11-dc01-429a-b20a-e97b77b8d5ec (type: router_ports): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:48.335060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:29:48.335060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 09 09:29:48.335060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 09 09:29:48.335060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 09 09:29:48.335060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 09 09:29:48.335060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1976, in update_router_port
May 09 09:29:48.335060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:29:48.335060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:48.335060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:29:48.335060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:48.335598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:29:48.335598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:48.335598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:29:48.335598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:48.335598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:29:48.335598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:48.335598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:29:48.335598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:29:48.335598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:48.335598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:29:48.335598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:48.335598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:29:48.335598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:29:48.335598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:48.336278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:29:48.336278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:48.336278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:29:48.336278 np0000173033 neutron-ovn-maintenance-worker[106314]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_for_inconsistencies finished in 0.353 seconds
May 09 09:29:48.479367 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:29:48.479908 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:29:48.480113 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:29:48.481486 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:29:48.513870 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:29:48.514286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:48.515229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:29:48.515229 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:48.515229 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:29:48.515229 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:29:48.515229 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:48.515229 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:29:48.515229 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:48.515229 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:29:48.516775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:48.516775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:29:48.516775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:29:48.516775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:29:48.516775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:29:48.516775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:29:48.516775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:29:48.516775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:48.516775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:29:48.516775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:48.516775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:29:48.516775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:48.517450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:29:48.517450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:48.517450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:29:48.517450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:48.517450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:29:48.517450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:29:48.517450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:48.517450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:29:48.517450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:48.517450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:29:48.517450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:29:48.517450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:48.517450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:29:48.518032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:48.518032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:29:48.518032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:48.518032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:29:48.518032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:29:48.518032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:29:48.518032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:29:48.518032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:29:48.518032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:29:48.518032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:29:48.518032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:29:48.518032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:29:48.518032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:29:48.518032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:29:48.518032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:29:48.518667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:29:48.518667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:29:48.518667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:29:48.518667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:29:48.518667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:29:48.518667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:48.518667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:29:48.518667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:48.518667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:29:48.518667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:48.518667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:29:48.518667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:48.518667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:29:48.518667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:48.518667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:29:48.518667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:29:48.518667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:48.518667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:29:48.518667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:48.519520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:29:48.519520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:29:48.519520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:48.519520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:29:48.519520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:48.519520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:29:48.632666 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:29:48.633255 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:29:48.633460 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:29:48.634570 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:29:48.634819 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:29:48.635216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:48.635722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:29:48.635722 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:48.635722 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:29:48.635722 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:29:48.635722 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:48.635722 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:29:48.635722 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:48.635722 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:29:48.637866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:48.637866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:29:48.637866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:29:48.637866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:29:48.637866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:29:48.637866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:29:48.637866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:29:48.637866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:48.637866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:29:48.637866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:48.637866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:29:48.637866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:48.638652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:29:48.638652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:48.638652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:29:48.638652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:48.638652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:29:48.638652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:29:48.638652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:48.638652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:29:48.638652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:48.638652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:29:48.638652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:29:48.638652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:48.638652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:29:48.639269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:48.639269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:29:48.639558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:48.639558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:29:48.639558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:29:48.639558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:29:48.639558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:29:48.639558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:29:48.639558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:29:48.639558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:29:48.639558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:29:48.639558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:29:48.639558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:29:48.639558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:29:48.639558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:29:48.639558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:29:48.639558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:29:48.639558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:29:48.642136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:29:48.642136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:29:48.642136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:48.642136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:29:48.642136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:48.642136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:29:48.642136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:48.642136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:29:48.642136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:48.642136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:29:48.642136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:48.642136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:29:48.642136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:29:48.642136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:48.642136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:29:48.642136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:48.642136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:29:48.642136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:29:48.642136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:48.643225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:29:48.643225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:48.643225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:29:48.643225 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:29:48.643225 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:29:48.643225 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:29:48.643225 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:29:48.643225 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:29:48.643448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:48.643448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:29:48.643448 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:48.643448 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:29:48.643448 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:29:48.643448 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:48.643448 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:29:48.643448 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:48.643448 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:29:48.643448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:48.643448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:29:48.643448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:29:48.643448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:29:48.643448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:48.643448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:29:48.643448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:48.643448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:29:48.643910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:48.643910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:29:48.643910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:48.643910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:29:48.643910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:48.643910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:29:48.643910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:29:48.643910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:48.643910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:29:48.643910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:48.643910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:29:48.643910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:29:48.643910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:48.643910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:29:48.643910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:48.643910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:29:48.644291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:48.644291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:29:48.644291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:29:48.644291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:29:48.644291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:29:48.644291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:29:48.644291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:29:48.644291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:29:48.644291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:29:48.644291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:29:48.644291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:48.644291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:29:48.644612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:48.644612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:29:48.644612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:48.644612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:29:48.644612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:48.644612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:29:48.644612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:48.644612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:29:48.644612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:29:48.644612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:48.644612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:29:48.644612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:48.644612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:29:48.644612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:29:48.645045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:48.645045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:29:48.645045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:48.645045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:29:48.645370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:48.645370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:29:48.645370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:29:48.645370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:29:48.645370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:29:48.645370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:29:48.645370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:29:48.645370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:29:48.645370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:29:48.645370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:29:48.645370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:29:48.645370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:29:48.645370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:29:48.645370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:29:48.645370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:29:48.645370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:29:48.645891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:29:48.645891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:29:48.645891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:29:48.645891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:29:48.645891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:29:48.645891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:48.645891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:29:48.645891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:48.645891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:29:48.645891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:48.645891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:29:48.645891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:48.645891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:29:48.645891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:48.645891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:29:48.645891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:29:48.645891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:48.645891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:29:48.646338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:48.646338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:29:48.646338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:29:48.646338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:48.646338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:29:48.646338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:48.646338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:29:48.647442 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:29:48.647550 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:29:48.647667 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:29:48.647787 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:29:48.648091 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:29:48.648307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:48.648827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:29:48.648827 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:48.648827 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:29:48.648827 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:29:48.648827 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:48.648827 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:29:48.648827 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:48.648827 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:29:48.649409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:48.649409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:29:48.649409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:29:48.649409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:29:48.649409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:48.649409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:29:48.649409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:48.649409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:29:48.649409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:48.649409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:29:48.649409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:48.649409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:29:48.649409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:48.649409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:29:48.649409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:29:48.650104 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:48.650104 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:29:48.650104 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:48.650104 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:29:48.650104 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:29:48.650104 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:48.650104 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:29:48.650104 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:48.650104 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:29:48.650104 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:48.650104 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:29:48.650104 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:29:48.650104 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:29:48.650104 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:29:48.650789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:29:48.650789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:29:48.650789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:29:48.650789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:29:48.650789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:48.650789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:29:48.650789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:48.650789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:29:48.650789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:48.650789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:29:48.650789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:48.650789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:29:48.650789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:48.651359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:29:48.651359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:29:48.651359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:48.651359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:29:48.651359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:48.651359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:29:48.651359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:29:48.651359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:48.651359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:29:48.651359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:48.651359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:29:48.651359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:48.651939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:29:48.651939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:29:48.651939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:29:48.651939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:29:48.651939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:29:48.651939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:29:48.651939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:29:48.651939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:29:48.651939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:29:48.651939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:29:48.651939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:29:48.651939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:29:48.651939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:29:48.651939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:29:48.651939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:29:48.651939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:29:48.651939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:29:48.651939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:29:48.651939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:29:48.652953 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:48.652953 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:29:48.652953 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:48.652953 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:29:48.652953 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:48.652953 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:29:48.652953 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:48.652953 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:29:48.652953 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:48.652953 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:29:48.652953 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:29:48.652953 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:48.652953 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:29:48.652953 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:48.652953 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:29:48.652953 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:29:48.652953 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:48.652953 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:29:48.652953 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:48.652953 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:29:48.654219 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:29:48.654219 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:29:48.654219 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:29:48.654219 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:29:48.654219 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:29:48.654219 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:29:48.654550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:48.654610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:29:48.654610 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:48.654610 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:29:48.654610 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:29:48.654610 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:48.654610 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:29:48.654610 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:48.654610 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:29:48.655452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:48.655452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:29:48.655452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:29:48.655452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:29:48.655452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:29:48.655452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:29:48.655452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:29:48.655452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:48.655452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:29:48.655452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:48.655452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:29:48.655452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:48.655939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:29:48.655939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:48.655939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:29:48.655939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:48.655939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:29:48.655939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:29:48.655939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:48.655939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:29:48.655939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:48.655939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:29:48.655939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:29:48.655939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:48.655939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:29:48.656339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:48.656339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:29:48.656415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:48.656415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:29:48.656415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:29:48.656415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:29:48.656415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:29:48.656415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:29:48.656415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:29:48.656415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:29:48.656415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:29:48.656415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:29:48.656415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:29:48.656415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:29:48.656415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:29:48.656415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:29:48.656415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:29:48.656415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:29:48.656952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:29:48.656952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:29:48.656952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:48.656952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:29:48.656952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:48.656952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:29:48.656952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:48.656952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:29:48.656952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:48.656952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:29:48.656952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:48.656952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:29:48.656952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:29:48.656952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:48.656952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:29:48.656952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:48.656952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:29:48.656952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:29:48.656952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:48.657576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:29:48.657576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:48.657576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:29:53.480033 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:29:53.481384 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:29:53.481384 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:29:53.481384 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:29:53.513773 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:29:53.514235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:53.515193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:29:53.515193 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:53.515193 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:29:53.515193 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:29:53.515193 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:53.515193 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:29:53.515193 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:53.515193 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:29:53.516390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:53.516390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:29:53.516390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:29:53.516390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:29:53.516390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:29:53.516390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:29:53.516390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:29:53.516390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:53.516390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:29:53.516390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:53.516390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:29:53.516390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:53.517174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:29:53.517174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:53.517174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:29:53.517174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:53.517174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:29:53.517174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:29:53.517174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:53.517174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:29:53.517174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:53.517174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:29:53.517174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:29:53.517174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:53.517174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:29:53.517941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:53.517941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:29:53.517941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:53.517941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:29:53.517941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:29:53.517941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:29:53.517941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:29:53.517941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:29:53.517941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:29:53.517941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:29:53.517941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:29:53.517941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:29:53.517941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:29:53.517941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:29:53.517941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:29:53.518803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:29:53.518803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:29:53.518803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:29:53.518803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:29:53.518803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:29:53.518803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:53.518803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:29:53.518803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:53.518803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:29:53.518803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:53.518803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:29:53.518803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:53.518803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:29:53.518803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:53.518803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:29:53.518803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:29:53.518803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:53.518803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:29:53.518803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:53.520332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:29:53.520332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:29:53.520332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:53.520332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:29:53.520332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:53.520332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:29:53.633405 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:29:53.634211 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:29:53.634627 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:29:53.635726 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:29:53.636129 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:29:53.636410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:53.637340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:29:53.637340 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:53.637340 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:29:53.637340 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:29:53.637340 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:53.637340 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:29:53.637340 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:53.637340 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:29:53.639290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:53.639290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:29:53.639290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:29:53.639290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:29:53.639290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:29:53.639290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:29:53.639290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:29:53.639290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:53.639290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:29:53.639290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:53.639290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:29:53.639290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:53.639967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:29:53.639967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:53.639967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:29:53.639967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:53.639967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:29:53.639967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:29:53.639967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:53.639967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:29:53.639967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:53.639967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:29:53.639967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:29:53.639967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:53.639967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:29:53.640526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:53.640526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:29:53.642088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:53.642088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:29:53.642088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:29:53.642088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:29:53.642088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:29:53.642088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:29:53.642088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:29:53.642088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:29:53.642088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:29:53.642088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:29:53.642088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:29:53.642088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:29:53.642088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:29:53.642088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:29:53.642088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:29:53.642088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:29:53.642882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:29:53.642882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:29:53.642882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:53.642882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:29:53.642882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:53.642882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:29:53.642882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:53.642882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:29:53.642882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:53.642882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:29:53.642882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:53.642882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:29:53.642882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:29:53.642882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:53.642882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:29:53.642882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:53.642882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:29:53.642882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:29:53.642882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:53.643837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:29:53.643837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:53.643837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:29:53.643837 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:29:53.643837 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:29:53.643837 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:29:53.643837 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:29:53.644222 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:29:53.644222 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:53.644649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:29:53.644649 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:53.644649 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:29:53.644649 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:29:53.644649 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:53.644649 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:29:53.644649 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:53.644649 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:29:53.645538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:53.645538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:29:53.645538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:29:53.645538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:29:53.645538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:53.645538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:29:53.645538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:53.645538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:29:53.645538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:53.645538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:29:53.645538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:53.645538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:29:53.645538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:53.645538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:29:53.645538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:29:53.646425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:53.646425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:29:53.646425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:53.646425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:29:53.646425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:29:53.646425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:53.646425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:29:53.646425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:53.646425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:29:53.646892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:53.646892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:29:53.646892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:29:53.646892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:29:53.646892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:29:53.646892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:29:53.646892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:29:53.646892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:29:53.646892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:29:53.646892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:29:53.646892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:53.646892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:29:53.647264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:53.647264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:29:53.647264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:53.647264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:29:53.647264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:53.647264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:29:53.647264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:53.647264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:29:53.647264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:29:53.647264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:53.647264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:29:53.647264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:53.647264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:29:53.647264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:29:53.647630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:53.647630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:29:53.647630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:53.647630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:29:53.647870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:53.647870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:29:53.647870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:29:53.647870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:29:53.647870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:29:53.647870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:29:53.647870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:29:53.647870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:29:53.647870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:29:53.647870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:29:53.647870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:29:53.647870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:29:53.647870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:29:53.647870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:29:53.647870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:29:53.647870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:29:53.648337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:29:53.648337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:29:53.648337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:29:53.648337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:29:53.648337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:29:53.648337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:53.648337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:29:53.648337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:53.648337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:29:53.648337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:53.648337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:29:53.648337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:53.648337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:29:53.648337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:53.648337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:29:53.648337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:29:53.648337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:53.648337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:29:53.648874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:53.648874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:29:53.648874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:29:53.648874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:53.648874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:29:53.648874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:53.648874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:29:53.648874 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:29:53.648874 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:29:53.648874 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:29:53.649353 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:29:53.649353 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:29:53.649479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:53.650047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:29:53.650047 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:53.650047 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:29:53.650047 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:29:53.650047 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:53.650047 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:29:53.650047 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:53.650047 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:29:53.650788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:53.650788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:29:53.650788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:29:53.650788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:29:53.650788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:53.650788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:29:53.650788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:53.650788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:29:53.650788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:53.650788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:29:53.650788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:53.650788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:29:53.650788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:53.650788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:29:53.650788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:29:53.651225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:53.651225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:29:53.651225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:53.651225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:29:53.651225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:29:53.651225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:53.651225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:29:53.651225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:53.651225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:29:53.651669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:53.651669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:29:53.651669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:29:53.651669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:29:53.651669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:29:53.651669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:29:53.651669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:29:53.651669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:29:53.651669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:29:53.651669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:53.651669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:29:53.652101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:53.652101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:29:53.652101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:53.652101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:29:53.652101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:53.652101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:29:53.652101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:53.652101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:29:53.652101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:29:53.652101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:53.652101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:29:53.652101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:53.652101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:29:53.652101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:29:53.652459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:53.652459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:29:53.652459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:53.652459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:29:53.652690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:53.652690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:29:53.652690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:29:53.652690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:29:53.652690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:29:53.652690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:29:53.652690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:29:53.652690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:29:53.652690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:29:53.652690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:29:53.652690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:29:53.652690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:29:53.652690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:29:53.652690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:29:53.652690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:29:53.652690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:29:53.652690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:29:53.653403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:29:53.653403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:29:53.653403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:29:53.653403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:53.653403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:29:53.653403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:53.653403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:29:53.653403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:53.653403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:29:53.653403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:53.653403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:29:53.653403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:53.653403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:29:53.653403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:29:53.653403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:53.653403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:29:53.653403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:53.653403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:29:53.653403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:29:53.654017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:53.654017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:29:53.654017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:53.654017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:29:53.654017 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:29:53.654017 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:29:53.654017 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:29:53.654229 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:29:53.655487 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:29:53.655787 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:29:53.655960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:53.656576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:29:53.656576 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:53.656576 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:29:53.656576 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:29:53.656576 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:53.656576 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:29:53.656576 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:53.656576 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:29:53.657643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:53.657643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:29:53.657643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:29:53.657643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:29:53.657643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:29:53.657643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:29:53.657643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:29:53.657643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:53.657643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:29:53.657643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:53.657643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:29:53.657643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:53.657922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:29:53.657922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:53.657922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:29:53.657922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:53.657922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:29:53.657922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:29:53.657922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:53.657922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:29:53.657922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:53.657922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:29:53.657922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:29:53.657922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:53.657922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:29:53.658150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:53.658150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:29:53.659224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:53.659224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:29:53.659224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:29:53.659224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:29:53.659224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:29:53.659224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:29:53.659224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:29:53.659224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:29:53.659224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:29:53.659224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:29:53.659224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:29:53.659224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:29:53.659224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:29:53.659224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:29:53.659224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:29:53.659224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:29:53.659535 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:29:53.659535 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:29:53.659535 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:53.659535 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:29:53.659535 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:53.659535 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:29:53.659535 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:53.659535 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:29:53.659535 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:53.659535 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:29:53.659535 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:53.659535 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:29:53.659535 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:29:53.659535 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:53.659535 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:29:53.659535 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:53.659535 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:29:53.659535 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:29:53.659535 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:53.659915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:29:53.659915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:53.659915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:29:58.480574 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:29:58.481626 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:29:58.481626 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:29:58.482664 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:29:58.484989 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG dbcounter [-] [106314] Writing DB stats neutron:SELECT=434 {{(pid=106314) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 09 09:29:58.520954 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:29:58.521176 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:58.522161 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:29:58.522161 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:58.522161 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:29:58.522161 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:29:58.522161 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:58.522161 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:29:58.522161 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:58.522161 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:29:58.523591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:58.523591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:29:58.523591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:29:58.523591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:29:58.523591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:29:58.523591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:29:58.523591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:29:58.523591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:58.523591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:29:58.523591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:58.523591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:29:58.523591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:58.524122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:29:58.524122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:58.524122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:29:58.524122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:58.524122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:29:58.524122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:29:58.524122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:58.524122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:29:58.524122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:58.524122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:29:58.524122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:29:58.524122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:58.524122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:29:58.524525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:58.524525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:29:58.524623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:58.524623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:29:58.524623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:29:58.524623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:29:58.524623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:29:58.524623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:29:58.524623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:29:58.524623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:29:58.524623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:29:58.524623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:29:58.524623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:29:58.524623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:29:58.524623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:29:58.524623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:29:58.524623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:29:58.524623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:29:58.525342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:29:58.525342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:29:58.525342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:58.525342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:29:58.525342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:58.525342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:29:58.525342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:58.525342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:29:58.525342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:58.525342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:29:58.525342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:58.525342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:29:58.525342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:29:58.525342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:58.525342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:29:58.525342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:58.525342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:29:58.525342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:29:58.525342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:58.526529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:29:58.526529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:58.526529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:29:58.634207 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:29:58.634850 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:29:58.635046 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:29:58.635963 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:29:58.636206 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:29:58.636387 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:58.637185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:29:58.637185 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:58.637185 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:29:58.637185 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:29:58.637185 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:58.637185 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:29:58.637185 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:58.637185 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:29:58.638843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:58.638843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:29:58.638843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:29:58.638843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:29:58.638843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:29:58.638843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:29:58.638843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:29:58.638843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:58.638843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:29:58.638843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:58.638843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:29:58.638843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:58.640145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:29:58.640145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:58.640145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:29:58.640145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:58.640145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:29:58.640145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:29:58.640145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:58.640145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:29:58.640145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:58.640145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:29:58.640145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:29:58.640145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:58.640145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:29:58.640860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:58.640860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:29:58.640860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:58.640860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:29:58.640860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:29:58.640860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:29:58.640860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:29:58.640860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:29:58.640860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:29:58.640860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:29:58.640860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:29:58.640860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:29:58.640860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:29:58.640860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:29:58.640860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:29:58.641571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:29:58.641571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:29:58.641571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:29:58.641571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:29:58.641571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:29:58.641571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:58.641571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:29:58.641571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:58.641571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:29:58.641571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:58.641571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:29:58.641571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:58.641571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:29:58.641571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:58.641571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:29:58.641571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:29:58.641571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:58.641571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:29:58.641571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:58.642266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:29:58.642266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:29:58.642266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:58.642266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:29:58.642266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:58.642266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:29:58.642759 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:29:58.643042 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:29:58.643160 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:29:58.643288 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:29:58.643736 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:29:58.644096 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:58.644668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:29:58.644668 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:58.644668 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:29:58.644668 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:29:58.644668 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:58.644668 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:29:58.644668 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:58.644668 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:29:58.645526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:58.645526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:29:58.645526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:29:58.645526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:29:58.645526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:58.645526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:29:58.645526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:58.645526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:29:58.645526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:58.645526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:29:58.645526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:58.645526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:29:58.645526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:58.645526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:29:58.645526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:29:58.645903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:58.645903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:29:58.645903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:58.645903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:29:58.645903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:29:58.645903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:58.645903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:29:58.645903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:58.645903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:29:58.646548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:58.646548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:29:58.646548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:29:58.646548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:29:58.646548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:29:58.646548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:29:58.646548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:29:58.646548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:29:58.646548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:29:58.646548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:29:58.646548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:58.646548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:29:58.647290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:58.647290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:29:58.647290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:58.647290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:29:58.647290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:58.647290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:29:58.647290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:58.647290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:29:58.647290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:29:58.647290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:58.647290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:29:58.647290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:58.647290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:29:58.647290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:29:58.648054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:58.648054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:29:58.648054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:58.648054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:29:58.648054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:58.648054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:29:58.648054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:29:58.648054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:29:58.648054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:29:58.648054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:29:58.648054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:29:58.648054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:29:58.648054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:29:58.648054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:29:58.648407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:29:58.648407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:29:58.648407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:29:58.648407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:29:58.648407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:29:58.648407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:29:58.648407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:29:58.648407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:29:58.648407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:29:58.648407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:29:58.648407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:29:58.648407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:58.648407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:29:58.648407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:58.648407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:29:58.648407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:58.648407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:29:58.648407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:58.648407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:29:58.648899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:58.648899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:29:58.648899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:29:58.648899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:58.648899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:29:58.648899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:58.648899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:29:58.648899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:29:58.648899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:58.648899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:29:58.648899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:58.648899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:29:58.648899 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:29:58.649270 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:29:58.649270 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:29:58.649451 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:29:58.650070 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:29:58.650070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:58.650491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:29:58.650491 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:58.650491 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:29:58.650491 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:29:58.650491 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:58.650491 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:29:58.650491 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:58.650491 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:29:58.651709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:58.651709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:29:58.651709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:29:58.651709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:29:58.651709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:58.651709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:29:58.651709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:58.651709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:29:58.651709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:58.651709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:29:58.651709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:58.651709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:29:58.651709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:58.651709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:29:58.651709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:29:58.652818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:58.652818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:29:58.652818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:58.652818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:29:58.652818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:29:58.652818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:58.652818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:29:58.652818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:58.652818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:29:58.653372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:58.653372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:29:58.653372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:29:58.653372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:29:58.653372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:29:58.653372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:29:58.653372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:29:58.653372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:29:58.653372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:29:58.653372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:58.653372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:29:58.653670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:58.653670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:29:58.653670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:58.653670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:29:58.653670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:58.653670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:29:58.653670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:58.653670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:29:58.653670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:29:58.653670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:58.653670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:29:58.653670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:58.653670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:29:58.653670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:29:58.654029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:58.654029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:29:58.654029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:58.654029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:29:58.654659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:58.654659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:29:58.654659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:29:58.654659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:29:58.654659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:29:58.654659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:29:58.654659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:29:58.654659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:29:58.654659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:29:58.654659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:29:58.654659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:29:58.654659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:29:58.654659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:29:58.654659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:29:58.654659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:29:58.654659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:29:58.654659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:29:58.655149 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:29:58.655149 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:29:58.655149 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:29:58.655149 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:58.655149 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:29:58.655149 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:58.655149 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:29:58.655149 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:58.655149 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:29:58.655149 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:58.655149 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:29:58.655149 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:58.655149 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:29:58.655149 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:29:58.655149 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:58.655149 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:29:58.655149 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:58.655149 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:29:58.655149 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:29:58.655591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:58.655591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:29:58.655591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:58.655591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:29:58.655591 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:29:58.655591 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:29:58.655591 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:29:58.655591 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:29:58.655952 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:29:58.656469 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:29:58.656469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:58.656853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:29:58.656853 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:58.656853 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:29:58.656853 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:29:58.656853 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:58.656853 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:29:58.656853 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:58.656853 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:29:58.657713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:58.657713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:29:58.657713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:29:58.657713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:29:58.657713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:29:58.657713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:29:58.657713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:29:58.657713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:58.657713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:29:58.657713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:58.657713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:29:58.657713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:58.658458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:29:58.658458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:58.658458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:29:58.658458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:58.658458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:29:58.658458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:29:58.658458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:58.658458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:29:58.658458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:58.658458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:29:58.658458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:29:58.658458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:58.658458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:29:58.659031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:58.659031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:29:58.659031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:58.659031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:29:58.659031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:29:58.659031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:29:58.659031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:29:58.659031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:29:58.659031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:29:58.659031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:29:58.659031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:29:58.659031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:29:58.659031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:29:58.659031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:29:58.659031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:29:58.659413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:29:58.659413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:29:58.659413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:29:58.659413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:29:58.659413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:29:58.659413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:58.659413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:29:58.659413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:29:58.659413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:29:58.659413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:29:58.659413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:29:58.659413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:29:58.659413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:29:58.659413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:29:58.659413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:29:58.659413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:29:58.659413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:29:58.659413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:29:58.659413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:29:58.659875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:29:58.659875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:29:58.659875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:29:58.659875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:29:58.659875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:29:58.659875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:30:03.481327 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:30:03.482213 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:30:03.482213 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:30:03.482808 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:30:03.520641 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:30:03.521004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:03.521805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:30:03.521805 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:03.521805 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:30:03.521805 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:30:03.521805 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:03.521805 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:30:03.521805 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:03.521805 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:30:03.522555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:03.522555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:30:03.522555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:30:03.522555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:30:03.522555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:30:03.522555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:30:03.522555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:30:03.522555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:03.522555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:30:03.522555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:03.522555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:30:03.522555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:03.523034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:30:03.523034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:03.523034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:30:03.523034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:03.523034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:30:03.523034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:30:03.523034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:03.523034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:30:03.523034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:03.523034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:30:03.523034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:30:03.523034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:03.523034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:30:03.523583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:03.523583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:30:03.523583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:03.523583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:30:03.523583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:30:03.523583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:30:03.523583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:30:03.523583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:30:03.523583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:30:03.523583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:30:03.523583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:30:03.523583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:30:03.523583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:30:03.523583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:30:03.523583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:30:03.524325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:30:03.524325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:30:03.524325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:30:03.524325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:30:03.524325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:30:03.524325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:03.524325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:30:03.524325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:03.524325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:30:03.524325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:03.524325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:30:03.524325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:03.524325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:30:03.524325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:03.524325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:30:03.524325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:30:03.524325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:03.524325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:30:03.524325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:03.524934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:30:03.524934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:30:03.524934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:03.524934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:30:03.524934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:03.524934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:30:03.634967 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:30:03.635698 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:30:03.635856 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:30:03.636817 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:30:03.637141 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:30:03.637533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:03.638249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:30:03.638249 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:03.638249 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:30:03.638249 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:30:03.638249 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:03.638249 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:30:03.638249 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:03.638249 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:30:03.639625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:03.639625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:30:03.639625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:30:03.639625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:30:03.639625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:30:03.639625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:30:03.639625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:30:03.639625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:03.639625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:30:03.639625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:03.639625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:30:03.639625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:03.640047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:30:03.640047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:03.640047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:30:03.640047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:03.640047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:30:03.640047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:30:03.640047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:03.640047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:30:03.640047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:03.640047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:30:03.640047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:30:03.640047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:03.640047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:30:03.640545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:03.640545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:30:03.641268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:03.641268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:30:03.641268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:30:03.641268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:30:03.641268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:30:03.641268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:30:03.641268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:30:03.641268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:30:03.641268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:30:03.641268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:30:03.641268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:30:03.641268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:30:03.641268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:30:03.641268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:30:03.641268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:30:03.641268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:30:03.641891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:30:03.641891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:30:03.641891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:03.641891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:30:03.641891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:03.641891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:30:03.641891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:03.641891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:30:03.641891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:03.641891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:30:03.641891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:03.641891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:30:03.641891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:30:03.641891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:03.641891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:30:03.641891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:03.641891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:30:03.641891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:30:03.641891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:03.642478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:30:03.642478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:03.642478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:30:03.643182 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:30:03.643598 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:30:03.643795 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:30:03.643959 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:30:03.644537 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:30:03.645170 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:03.645438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:30:03.645438 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:03.645438 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:30:03.645438 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:30:03.645438 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:03.645438 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:30:03.645438 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:03.645438 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:30:03.646481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:03.646481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:30:03.646481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:30:03.646481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:30:03.646481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:03.646481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:30:03.646481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:03.646481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:30:03.646481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:03.646481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:30:03.646481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:03.646481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:30:03.646481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:03.646481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:30:03.646481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:30:03.647133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:03.647133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:30:03.647133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:03.647133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:30:03.647133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:30:03.647133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:03.647133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:30:03.647133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:03.647133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:30:03.647605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:03.647605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:30:03.647605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:30:03.647605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:30:03.647605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:30:03.647605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:30:03.647605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:30:03.647605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:30:03.647605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:30:03.647605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:30:03.647605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:03.647605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:30:03.648094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:03.648094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:30:03.648094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:03.648094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:30:03.648094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:03.648094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:30:03.648094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:03.648094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:30:03.648094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:30:03.648094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:03.648094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:30:03.648094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:03.648094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:30:03.648094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:30:03.648522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:03.648522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:30:03.648522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:03.648522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:30:03.649410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:03.649410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:30:03.649410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:30:03.649410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:30:03.649410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:30:03.649410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:30:03.649410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:30:03.649410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:30:03.649410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:30:03.649410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:30:03.649410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:30:03.649410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:30:03.649410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:30:03.649410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:30:03.649410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:30:03.649410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:30:03.649890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:30:03.649890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:30:03.649890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:30:03.649890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:30:03.649890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:30:03.649890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:03.649890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:30:03.649890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:03.649890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:30:03.649890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:03.649890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:30:03.649890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:03.649890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:30:03.649890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:03.649890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:30:03.649890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:30:03.649890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:03.649890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:30:03.650409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:03.650409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:30:03.650409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:30:03.650409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:03.650409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:30:03.650409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:03.650409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:30:03.650409 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:30:03.650409 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:30:03.650409 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:30:03.650722 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:30:03.651509 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:30:03.651762 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:03.652384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:30:03.652384 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:03.652384 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:30:03.652384 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:30:03.652384 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:03.652384 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:30:03.652384 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:03.652384 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:30:03.653075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:03.653075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:30:03.653075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:30:03.653075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:30:03.653075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:03.653075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:30:03.653075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:03.653075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:30:03.653075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:03.653075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:30:03.653075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:03.653075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:30:03.653075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:03.653075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:30:03.653075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:30:03.653522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:03.653522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:30:03.653522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:03.653522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:30:03.653522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:30:03.653522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:03.653522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:30:03.653522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:03.653522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:30:03.653904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:03.653904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:30:03.653904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:30:03.653904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:30:03.653904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:30:03.653904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:30:03.653904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:30:03.653904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:30:03.653904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:30:03.653904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:03.653904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:30:03.654273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:03.654273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:30:03.654273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:03.654273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:30:03.654273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:03.654273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:30:03.654273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:03.654273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:30:03.654273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:30:03.654273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:03.654273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:30:03.654273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:03.654273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:30:03.654273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:30:03.654690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:03.654690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:30:03.654690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:03.654690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:30:03.654690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:03.654690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:30:03.654690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:30:03.654690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:30:03.654690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:30:03.654690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:30:03.654690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:30:03.654690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:30:03.654690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:30:03.654690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:30:03.655711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:30:03.655711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:30:03.655711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:30:03.655711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:30:03.655711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:30:03.655711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:30:03.655711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:30:03.655711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:30:03.655711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:30:03.655711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:30:03.655711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:03.655711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:30:03.655711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:03.655711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:30:03.655711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:03.655711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:30:03.655711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:03.655711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:30:03.655711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:03.656365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:30:03.656365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:30:03.656365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:03.656365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:30:03.656365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:03.656365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:30:03.656365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:30:03.656365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:03.656365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:30:03.656365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:03.656365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:30:03.656365 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:30:03.656365 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:30:03.656738 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:30:03.656738 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:30:03.656738 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:30:03.656738 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:30:03.656738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:03.656738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:30:03.656738 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:03.656738 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:30:03.656738 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:30:03.657129 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:03.657129 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:30:03.657129 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:03.657129 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:30:03.657240 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:03.657240 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:30:03.657240 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:30:03.657240 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:30:03.657240 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:30:03.657240 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:30:03.657240 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:30:03.657240 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:03.657240 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:30:03.657240 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:03.657240 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:30:03.657240 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:03.657578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:30:03.657578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:03.657578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:30:03.657578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:03.657578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:30:03.657578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:30:03.657578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:03.657578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:30:03.657578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:03.657578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:30:03.657578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:30:03.657578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:03.657578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:30:03.659277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:03.659277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:30:03.659277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:03.659277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:30:03.659277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:30:03.659277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:30:03.659277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:30:03.659277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:30:03.659277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:30:03.659277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:30:03.659277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:30:03.659277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:30:03.659277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:30:03.659277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:30:03.659277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:30:03.659654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:30:03.659654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:30:03.659654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:30:03.659654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:30:03.659654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:30:03.659654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:03.659654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:30:03.659654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:03.659654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:30:03.659654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:03.659654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:30:03.659654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:03.659654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:30:03.659654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:03.659654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:30:03.659654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:30:03.659654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:03.659654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:30:03.659654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:03.660182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:30:03.660182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:30:03.660182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:03.660182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:30:03.660182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:03.660182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:30:08.481961 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:30:08.482997 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:30:08.483318 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:30:08.486138 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:30:08.525409 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:30:08.525935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:08.526714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:30:08.526714 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:08.526714 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:30:08.526714 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:30:08.526714 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:08.526714 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:30:08.526714 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:08.526714 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:30:08.529302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.044 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:08.529302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:30:08.529302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:30:08.529302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:30:08.529302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:30:08.529302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:30:08.529302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:30:08.529302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:08.529302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:30:08.529302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:08.529302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:30:08.529302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:08.529698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:30:08.529698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:08.529698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:30:08.529698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:08.529698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:30:08.529698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:30:08.529698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:08.529698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:30:08.529698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:08.529698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:30:08.529698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:30:08.529698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:08.529698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:30:08.530177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:08.530177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:30:08.530177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:08.530177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:30:08.530177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:30:08.530177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:30:08.530177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:30:08.530177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:30:08.530177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:30:08.530177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:30:08.530177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:30:08.530177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:30:08.530177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:30:08.530177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:30:08.530177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:30:08.530634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:30:08.530634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:30:08.530634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:30:08.530634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:30:08.530634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:30:08.530634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:08.530634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:30:08.530634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:08.530634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:30:08.530634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:08.530634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:30:08.530634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:08.530634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:30:08.530634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:08.530634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:30:08.530634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:30:08.530634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:08.530634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:30:08.530634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:08.531224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:30:08.531224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:30:08.531224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:08.531224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:30:08.531224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:08.531224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:30:08.636053 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:30:08.637890 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:30:08.637890 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:30:08.639110 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:30:08.639504 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:30:08.639785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:08.640827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:30:08.640827 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:08.640827 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:30:08.640827 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:30:08.640827 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:08.640827 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:30:08.640827 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:08.640827 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:30:08.642830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:08.642830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:30:08.642830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:30:08.642830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:30:08.642830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:30:08.642830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:30:08.642830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:30:08.642830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:08.642830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:30:08.642830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:08.642830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:30:08.642830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:08.643833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:30:08.643833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:08.643833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:30:08.643833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:08.643833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:30:08.643833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:30:08.643833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:08.643833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:30:08.643833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:08.643833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:30:08.643833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:30:08.643833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:08.643833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:30:08.644922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:08.644922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:30:08.644922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:08.644922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:30:08.644922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:30:08.644922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:30:08.644922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:30:08.644922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:30:08.644922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:30:08.644922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:30:08.644922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:30:08.644922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:30:08.644922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:30:08.644922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:30:08.644922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:30:08.645768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:30:08.645768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:30:08.645768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:30:08.645768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:30:08.645768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:30:08.645768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:08.645768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:30:08.645768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:08.645768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:30:08.645768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:08.645768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:30:08.645768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:08.645768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:30:08.645768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:08.645768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:30:08.645768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:30:08.645768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:08.645768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:30:08.645768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:08.646528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:30:08.646528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:30:08.646528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:08.646528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:30:08.646528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:08.646528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:30:08.646528 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:30:08.646528 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:30:08.646528 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:30:08.646528 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:30:08.646836 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:30:08.646836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:08.646836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:30:08.646836 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:08.646836 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:30:08.646836 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:30:08.646836 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:08.646836 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:30:08.646836 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:08.646836 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:30:08.647710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:08.647710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:30:08.647710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:30:08.647710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:30:08.647710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:08.647710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:30:08.647710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:08.647710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:30:08.647710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:08.647710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:30:08.647710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:08.647710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:30:08.647710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:08.647710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:30:08.647710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:30:08.648192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:08.648192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:30:08.648192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:08.648192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:30:08.648192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:30:08.648192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:08.648192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:30:08.648192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:08.648192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:30:08.648693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:08.648693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:30:08.648693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:30:08.648693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:30:08.648693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:30:08.648693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:30:08.648693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:30:08.648693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:30:08.648693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:30:08.648693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:30:08.648693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:08.648693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:30:08.649138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:08.649138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:30:08.649138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:08.649138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:30:08.649138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:08.649138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:30:08.649138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:08.649138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:30:08.649138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:30:08.649138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:08.649138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:30:08.649138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:08.649138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:30:08.649138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:30:08.649559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:08.649559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:30:08.649559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:08.649559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:30:08.649947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:08.649947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:30:08.649947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:30:08.649947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:30:08.649947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:30:08.649947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:30:08.649947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:30:08.649947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:30:08.649947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:30:08.649947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:30:08.649947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:30:08.649947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:30:08.649947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:30:08.649947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:30:08.649947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:30:08.649947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:30:08.650401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:30:08.650401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:30:08.650401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:30:08.650401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:30:08.650401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:30:08.650401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:08.650401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:30:08.650401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:08.650401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:30:08.650401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:08.650401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:30:08.650401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:08.650401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:30:08.650401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:08.650401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:30:08.650401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:30:08.650401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:08.650401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:30:08.651020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:08.651020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:30:08.651020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:30:08.651020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:08.651020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:30:08.651020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:08.651020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:30:08.651020 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:30:08.651020 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:30:08.651020 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:30:08.651394 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:30:08.651394 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:30:08.651394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:08.651857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:30:08.651857 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:08.651857 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:30:08.651857 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:30:08.651857 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:08.651857 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:30:08.651857 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:08.651857 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:30:08.652593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:08.652593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:30:08.652593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:30:08.652593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:30:08.652593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:08.652593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:30:08.652593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:08.652593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:30:08.652593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:08.652593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:30:08.652593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:08.652593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:30:08.652593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:08.652593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:30:08.652593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:30:08.652995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:08.652995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:30:08.652995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:08.652995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:30:08.652995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:30:08.652995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:08.652995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:30:08.652995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:08.652995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:30:08.653357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:08.653357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:30:08.653357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:30:08.653357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:30:08.653357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:30:08.653357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:30:08.653357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:30:08.653357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:30:08.653357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:30:08.653357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:08.653357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:30:08.653700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:08.653700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:30:08.653700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:08.653700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:30:08.653700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:08.653700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:30:08.653700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:08.653700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:30:08.653700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:30:08.653700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:08.653700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:30:08.653700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:08.653700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:30:08.653700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:30:08.654084 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:08.654084 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:30:08.654084 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:08.654084 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:30:08.654192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:08.654192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:30:08.654192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:30:08.654192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:30:08.654192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:30:08.654192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:30:08.654192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:30:08.654192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:30:08.654192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:30:08.654192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:30:08.654192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:30:08.654192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:30:08.654192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:30:08.654192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:30:08.654192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:30:08.654192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:30:08.654192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:30:08.654616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:30:08.654616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:30:08.654616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:30:08.654616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:08.654616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:30:08.654616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:08.654616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:30:08.654616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:08.654616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:30:08.654616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:08.654616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:30:08.654616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:08.654616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:30:08.654616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:30:08.654616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:08.654616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:30:08.654616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:08.654616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:30:08.654616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:30:08.655137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:08.655137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:30:08.655137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:08.655137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:30:08.655511 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:30:08.655854 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:30:08.655940 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:30:08.656114 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:30:08.657148 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:30:08.657517 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:30:08.657759 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:08.658409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:30:08.658409 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:08.658409 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:30:08.658409 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:30:08.658409 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:08.658409 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:30:08.658409 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:08.658409 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:30:08.659881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:08.659881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:30:08.659881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:30:08.659881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:30:08.659881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:30:08.659881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:30:08.659881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:30:08.659881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:08.659881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:30:08.659881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:08.659881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:30:08.659881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:08.660230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:30:08.660230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:08.660230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:30:08.660230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:08.660230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:30:08.660230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:30:08.660230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:08.660230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:30:08.660230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:08.660230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:30:08.660230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:30:08.660230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:08.660230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:30:08.660647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:08.660647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:30:08.660647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:08.660647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:30:08.660647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:30:08.660647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:30:08.660647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:30:08.660647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:30:08.660647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:30:08.660647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:30:08.660647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:30:08.660647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:30:08.660647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:30:08.660647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:30:08.660647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:30:08.661188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:30:08.661188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:30:08.661188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:30:08.661188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:30:08.661188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:30:08.661188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:08.661188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:30:08.661188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:08.661188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:30:08.661188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:08.661188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:30:08.661188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:08.661188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:30:08.661188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:08.661188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:30:08.661188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:30:08.661188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:08.661188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:30:08.661188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:08.661970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:30:08.661970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:30:08.661970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:08.661970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:30:08.661970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:08.661970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:30:13.483125 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:30:13.484109 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:30:13.484109 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:30:13.484629 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:30:13.518242 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:30:13.518242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:13.518242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:30:13.518242 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:13.518242 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:30:13.518242 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:30:13.518242 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:13.518242 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:30:13.518242 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:13.518242 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:30:13.519135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:13.519135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:30:13.519135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:30:13.519135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:30:13.519135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:30:13.519135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:30:13.519135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:30:13.519135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:13.519135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:30:13.519135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:13.519135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:30:13.519135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:13.519820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:30:13.519820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:13.519820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:30:13.519820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:13.519820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:30:13.519820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:30:13.519820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:13.519820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:30:13.519820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:13.519820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:30:13.519820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:30:13.519820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:13.519820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:30:13.520511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:13.520511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:30:13.520511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:13.520511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:30:13.520511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:30:13.520511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:30:13.520511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:30:13.520511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:30:13.520511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:30:13.520511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:30:13.520511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:30:13.520511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:30:13.520511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:30:13.520511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:30:13.520511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:30:13.521381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:30:13.521381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:30:13.521381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:30:13.521381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:30:13.521381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:30:13.521381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:13.521381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:30:13.521381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:13.521381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:30:13.521381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:13.521381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:30:13.521381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:13.521381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:30:13.521381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:13.521381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:30:13.521381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:30:13.521381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:13.521381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:30:13.521381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:13.523691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:30:13.523691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:30:13.523691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:13.523691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:30:13.523691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:13.523691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:30:13.637117 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:30:13.637571 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:30:13.637660 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:30:13.638551 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:30:13.638854 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:30:13.639080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:13.639707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:30:13.639707 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:13.639707 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:30:13.639707 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:30:13.639707 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:13.639707 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:30:13.639707 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:13.639707 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:30:13.640471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:13.640471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:30:13.640471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:30:13.640471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:30:13.640471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:30:13.640471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:30:13.640471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:30:13.640471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:13.640471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:30:13.640471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:13.640471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:30:13.640471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:13.640852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:30:13.640852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:13.640852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:30:13.640852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:13.640852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:30:13.640852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:30:13.640852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:13.640852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:30:13.640852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:13.640852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:30:13.640852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:30:13.640852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:13.640852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:30:13.641228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:13.641228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:30:13.641228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:13.641228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:30:13.641228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:30:13.641228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:30:13.641228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:30:13.641228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:30:13.641228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:30:13.641228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:30:13.641228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:30:13.641228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:30:13.641228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:30:13.641228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:30:13.641228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:30:13.641671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:30:13.641671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:30:13.641671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:30:13.641671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:30:13.641671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:30:13.641671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:13.641671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:30:13.641671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:13.641671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:30:13.641671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:13.641671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:30:13.641671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:13.641671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:30:13.641671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:13.641671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:30:13.641671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:30:13.641671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:13.641671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:30:13.641671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:13.642276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:30:13.642276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:30:13.642276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:13.642276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:30:13.642276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:13.642276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:30:13.644657 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:30:13.644895 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:30:13.644960 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:30:13.645052 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:30:13.645912 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:30:13.646088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:13.646637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:30:13.646637 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:13.646637 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:30:13.646637 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:30:13.646637 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:13.646637 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:30:13.646637 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:13.646637 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:30:13.647195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:13.647195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:30:13.647195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:30:13.647195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:30:13.647195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:13.647195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:30:13.647195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:13.647195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:30:13.647195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:13.647195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:30:13.647195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:13.647195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:30:13.647195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:13.647195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:30:13.647195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:30:13.647665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:13.647665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:30:13.647665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:13.647665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:30:13.647665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:30:13.647665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:13.647665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:30:13.647665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:13.647665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:30:13.648070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:13.648070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:30:13.648070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:30:13.648070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:30:13.648070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:30:13.648070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:30:13.648070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:30:13.648070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:30:13.648070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:30:13.648070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:30:13.648070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:13.648070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:30:13.648456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:13.648456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:30:13.648456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:13.648456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:30:13.648456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:13.648456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:30:13.648456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:13.648456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:30:13.648456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:30:13.648456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:13.648456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:30:13.648456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:13.648456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:30:13.648456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:30:13.648984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:13.648984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:30:13.648984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:13.648984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:30:13.648984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:13.648984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:30:13.648984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:30:13.648984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:30:13.648984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:30:13.648984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:30:13.648984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:30:13.648984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:30:13.648984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:30:13.648984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:30:13.649407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:30:13.649407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:30:13.649407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:30:13.649407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:30:13.649407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:30:13.649407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:30:13.649407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:30:13.649407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:30:13.649407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:30:13.649407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:30:13.649407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:30:13.649407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:13.649407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:30:13.649407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:13.649407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:30:13.649407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:13.649407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:30:13.649407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:13.649407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:30:13.650018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:13.650018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:30:13.650018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:30:13.650018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:13.650018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:30:13.650018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:13.650018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:30:13.650018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:30:13.650018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:13.650018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:30:13.650018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:13.650018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:30:13.650696 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:30:13.650975 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:30:13.651044 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:30:13.651130 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:30:13.651558 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:30:13.651894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:13.652560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:30:13.652560 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:13.652560 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:30:13.652560 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:30:13.652560 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:13.652560 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:30:13.652560 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:13.652560 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:30:13.653097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:13.653097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:30:13.653097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:30:13.653097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:30:13.653097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:13.653097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:30:13.653097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:13.653097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:30:13.653097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:13.653097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:30:13.653097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:13.653097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:30:13.653097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:13.653097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:30:13.653097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:30:13.653564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:13.653564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:30:13.653564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:13.653564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:30:13.653564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:30:13.653564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:13.653564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:30:13.653564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:13.653564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:30:13.653929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:13.653929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:30:13.653929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:30:13.653929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:30:13.653929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:30:13.653929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:30:13.653929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:30:13.653929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:30:13.653929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:30:13.653929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:13.653929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:30:13.654348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:13.654348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:30:13.654348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:13.654348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:30:13.654348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:13.654348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:30:13.654348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:13.654348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:30:13.654348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:30:13.654348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:13.654348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:30:13.654348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:13.654348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:30:13.654348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:30:13.654784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:13.654784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:30:13.654784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:13.654784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:30:13.654784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:13.654784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:30:13.654784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:30:13.654784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:30:13.654784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:30:13.654784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:30:13.654784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:30:13.654784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:30:13.654784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:30:13.654784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:30:13.655277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:30:13.655277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:30:13.655277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:30:13.655277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:30:13.655277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:30:13.655277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:30:13.655277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:30:13.655277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:30:13.655277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:30:13.655277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:30:13.655277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:13.655277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:30:13.655277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:13.655277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:30:13.655277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:13.655277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:30:13.655277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:13.655277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:30:13.655277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:13.655970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:30:13.655970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:30:13.655970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:13.655970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:30:13.655970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:13.655970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:30:13.655970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:30:13.655970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:13.655970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:30:13.655970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:13.655970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:30:13.655970 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:30:13.656481 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:30:13.656481 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:30:13.656481 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:30:13.656799 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:30:13.657146 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:30:13.657391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:13.658081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:30:13.658081 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:13.658081 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:30:13.658081 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:30:13.658081 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:13.658081 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:30:13.658081 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:13.658081 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:30:13.658715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:13.658715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:30:13.658715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:30:13.658715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:30:13.658715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:30:13.658715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:30:13.658715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:30:13.658715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:13.658715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:30:13.658715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:13.658715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:30:13.658715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:13.659136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:30:13.659136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:13.659136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:30:13.659136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:13.659136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:30:13.659136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:30:13.659136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:13.659136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:30:13.659136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:13.659136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:30:13.659136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:30:13.659136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:13.659136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:30:13.659488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:13.659488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:30:13.659488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:13.659488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:30:13.659488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:30:13.659488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:30:13.659488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:30:13.659488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:30:13.659488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:30:13.659488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:30:13.659488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:30:13.659488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:30:13.659488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:30:13.659488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:30:13.659488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:30:13.659943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:30:13.659943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:30:13.659943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:30:13.659943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:30:13.659943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:30:13.659943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:13.659943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:30:13.659943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:13.659943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:30:13.659943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:13.659943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:30:13.659943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:13.659943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:30:13.659943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:13.659943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:30:13.659943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:30:13.659943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:13.659943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:30:13.659943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:13.660472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:30:13.660472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:30:13.660472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:13.660472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:30:13.660472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:13.660472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:30:18.483824 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:30:18.484379 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:30:18.484525 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:30:18.485900 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:30:18.526264 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:30:18.526801 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:18.528035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:30:18.528035 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:18.528035 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:30:18.528035 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:30:18.528035 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:18.528035 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:30:18.528035 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:18.528035 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:30:18.529020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.043 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:18.529020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:30:18.529020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:30:18.529020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:30:18.529020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:30:18.529020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:30:18.529020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:30:18.529020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:18.529020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:30:18.529020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:18.529020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:30:18.529020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:18.530389 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:30:18.530389 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:18.530389 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:30:18.530389 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:18.530389 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:30:18.530389 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:30:18.530389 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:18.530389 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:30:18.530389 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:18.530389 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:30:18.530389 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:30:18.530389 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:18.530389 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:30:18.531339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:18.531339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:30:18.531339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:18.531339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:30:18.531339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:30:18.531339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:30:18.531339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:30:18.531339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:30:18.531339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:30:18.531339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:30:18.531339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:30:18.531339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:30:18.531339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:30:18.531339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:30:18.531339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:30:18.532130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:30:18.532130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:30:18.532130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:30:18.532130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:30:18.532130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:30:18.532130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:18.532130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:30:18.532130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:18.532130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:30:18.532130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:18.532130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:30:18.532130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:18.532130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:30:18.532130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:18.532130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:30:18.532130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:30:18.532130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:18.532130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:30:18.532130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:18.533160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:30:18.533160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:30:18.533160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:18.533160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:30:18.533160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:18.533160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:30:18.637836 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:30:18.638263 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:30:18.638391 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:30:18.639781 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:30:18.640251 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:30:18.640593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:18.641585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:30:18.641585 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:18.641585 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:30:18.641585 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:30:18.641585 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:18.641585 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:30:18.641585 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:18.641585 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:30:18.643461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:18.643461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:30:18.643461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:30:18.643461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:30:18.643461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:30:18.643461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:30:18.643461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:30:18.643461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:18.643461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:30:18.643461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:18.643461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:30:18.643461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:18.643945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:30:18.643945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:18.643945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:30:18.643945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:18.643945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:30:18.643945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:30:18.643945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:18.643945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:30:18.643945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:18.643945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:30:18.643945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:30:18.643945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:18.643945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:30:18.644290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:18.644290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:30:18.645392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:18.645392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:30:18.645392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:30:18.645392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:30:18.645392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:30:18.645392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:30:18.645392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:30:18.645392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:30:18.645392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:30:18.645392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:30:18.645392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:30:18.645392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:30:18.645392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:30:18.645392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:30:18.645392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:30:18.645392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:30:18.646015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:30:18.646015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:30:18.646015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:18.646015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:30:18.646015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:18.646015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:30:18.646015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:18.646015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:30:18.646015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:18.646015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:30:18.646015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:18.646015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:30:18.646015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:30:18.646015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:18.646015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:30:18.646015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:18.646015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:30:18.646015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:30:18.646015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:18.647188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:30:18.647188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:18.647188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:30:18.647188 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:30:18.647188 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:30:18.647188 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:30:18.647188 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:30:18.647188 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:30:18.647824 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:18.648409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:30:18.648409 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:18.648409 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:30:18.648409 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:30:18.648409 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:18.648409 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:30:18.648409 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:18.648409 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:30:18.649196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:18.649196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:30:18.649196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:30:18.649196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:30:18.649196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:18.649196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:30:18.649196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:18.649196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:30:18.649196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:18.649196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:30:18.649196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:18.649196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:30:18.649196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:18.649196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:30:18.649196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:30:18.649665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:18.649665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:30:18.649665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:18.649665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:30:18.649665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:30:18.649665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:18.649665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:30:18.649665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:18.649665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:30:18.650109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:18.650109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:30:18.650109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:30:18.650109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:30:18.650109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:30:18.650109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:30:18.650109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:30:18.650109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:30:18.650109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:30:18.650109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:30:18.650109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:18.650109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:30:18.651029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:18.651029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:30:18.651029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:18.651029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:30:18.651029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:18.651029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:30:18.651029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:18.651029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:30:18.651029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:30:18.651029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:18.651029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:30:18.651029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:18.651029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:30:18.651029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:30:18.651878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:18.651878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:30:18.651878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:18.651878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:30:18.651878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:18.651878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:30:18.651878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:30:18.651878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:30:18.651878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:30:18.651878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:30:18.651878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:30:18.651878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:30:18.651878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:30:18.651878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:30:18.652779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:30:18.652779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:30:18.652779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:30:18.652779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:30:18.652779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:30:18.652779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:30:18.652779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:30:18.652779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:30:18.652779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:30:18.652779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:30:18.652779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:30:18.652779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:18.652779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:30:18.652779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:18.652779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:30:18.652779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:18.652779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:30:18.652779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:18.652779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:30:18.657927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:18.657927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:30:18.657927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:30:18.657927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:18.657927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:30:18.657927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:18.657927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:30:18.657927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:30:18.657927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:18.657927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:30:18.657927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:18.657927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:30:18.657927 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:30:18.657927 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:30:18.658465 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:30:18.658465 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:30:18.658465 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:30:18.658465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:18.658465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:30:18.658465 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:18.658465 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:30:18.658465 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:30:18.658465 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:18.658465 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:30:18.658465 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:18.658465 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:30:18.658995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:18.658995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:30:18.658995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:30:18.658995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:30:18.658995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:18.658995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:30:18.658995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:18.658995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:30:18.658995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:18.658995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:30:18.658995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:18.658995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:30:18.658995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:18.658995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:30:18.658995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:30:18.659608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:18.659608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:30:18.659608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:18.659608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:30:18.659608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:30:18.659608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:18.659608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:30:18.659608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:18.659608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:30:18.659608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:18.659608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:30:18.659608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:30:18.659608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:30:18.659608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:30:18.660010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:30:18.660010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:30:18.660010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:30:18.660010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:30:18.660010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:18.660010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:30:18.660010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:18.660010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:30:18.660010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:18.660010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:30:18.660010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:18.660010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:30:18.660010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:18.660254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:30:18.660254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:30:18.660254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:18.660254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:30:18.660254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:18.660254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:30:18.660254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:30:18.660254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:18.660254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:30:18.660254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:18.660254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:30:18.660254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:18.660483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:30:18.660483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:30:18.660483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:30:18.660483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:30:18.660483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:30:18.660483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:30:18.660483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:30:18.660483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:30:18.660483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:30:18.660483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:30:18.660483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:30:18.660483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:30:18.660483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:30:18.660483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:30:18.660483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:30:18.660483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:30:18.660483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:30:18.660483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:30:18.660483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:30:18.661692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:18.661692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:30:18.661692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:18.661692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:30:18.661692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:18.661692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:30:18.661692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:18.661692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:30:18.661692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:18.661692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:30:18.661692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:30:18.661692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:18.661692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:30:18.661692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:18.661692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:30:18.661692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:30:18.661692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:18.661692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:30:18.661692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:18.661692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:30:18.662228 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:30:18.662228 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:30:18.662228 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:30:18.662228 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:30:18.662228 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:30:18.662228 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:30:18.662553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:18.664843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:30:18.664843 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:18.664843 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:30:18.664843 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:30:18.664843 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:18.664843 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:30:18.664843 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:18.664843 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:30:18.664843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:18.664843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:30:18.664843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:30:18.664843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:30:18.664843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:30:18.664843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:30:18.664843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:30:18.665510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:18.665510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:30:18.665510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:18.665510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:30:18.665510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:18.665510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:30:18.665510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:18.665510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:30:18.665510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:18.665510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:30:18.665510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:30:18.665510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:18.665510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:30:18.665510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:18.665808 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:30:18.665808 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:30:18.665808 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:18.665808 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:30:18.665808 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:18.665808 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:30:18.666393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:18.666393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:30:18.666393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:30:18.666393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:30:18.666393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:30:18.666393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:30:18.666393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:30:18.666393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:30:18.666393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:30:18.666393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:30:18.666393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:30:18.666393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:30:18.666393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:30:18.666393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:30:18.666393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:30:18.666393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:30:18.666890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:30:18.666890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:30:18.666890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:18.666890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:30:18.666890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:18.666890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:30:18.666890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:18.666890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:30:18.666890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:18.666890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:30:18.666890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:18.666890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:30:18.666890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:30:18.666890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:18.666890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:30:18.666890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:18.666890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:30:18.666890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:30:18.666890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:18.667423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:30:18.667423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:18.667423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:30:23.484533 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:30:23.485646 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:30:23.485646 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:30:23.487085 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:30:23.520244 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:30:23.520449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:23.520992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:30:23.520992 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:23.520992 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:30:23.520992 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:30:23.520992 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:23.520992 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:30:23.520992 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:23.520992 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:30:23.521741 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:23.521741 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:30:23.521741 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:30:23.521741 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:30:23.521741 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:30:23.521741 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:30:23.521741 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:30:23.521741 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:23.521741 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:30:23.521741 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:23.521741 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:30:23.521741 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:23.522204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:30:23.522204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:23.522204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:30:23.522204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:23.522204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:30:23.522204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:30:23.522204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:23.522204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:30:23.522204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:23.522204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:30:23.522204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:30:23.522204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:23.522204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:30:23.522665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:23.522665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:30:23.522838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:23.522838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:30:23.522838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:30:23.522838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:30:23.522838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:30:23.522838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:30:23.522838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:30:23.522838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:30:23.522838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:30:23.522838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:30:23.522838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:30:23.522838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:30:23.522838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:30:23.522838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:30:23.522838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:30:23.522838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:30:23.523245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:30:23.523245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:30:23.523245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:23.523245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:30:23.523245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:23.523245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:30:23.523245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:23.523245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:30:23.523245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:23.523245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:30:23.523245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:23.523245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:30:23.523245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:30:23.523245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:23.523245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:30:23.523245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:23.523245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:30:23.523245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:30:23.523245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:23.523641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:30:23.523641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:23.523641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:30:23.638331 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:30:23.638829 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:30:23.638964 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:30:23.639651 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:30:23.639907 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:30:23.640109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:23.640886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:30:23.640886 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:23.640886 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:30:23.640886 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:30:23.640886 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:23.640886 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:30:23.640886 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:23.640886 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:30:23.642112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:23.642112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:30:23.642112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:30:23.642112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:30:23.642112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:30:23.642112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:30:23.642112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:30:23.642112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:23.642112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:30:23.642112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:23.642112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:30:23.642112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:23.642941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:30:23.642941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:23.642941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:30:23.642941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:23.642941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:30:23.642941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:30:23.642941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:23.642941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:30:23.642941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:23.642941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:30:23.642941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:30:23.642941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:23.642941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:30:23.643305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:23.643305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:30:23.643519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:23.643519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:30:23.643519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:30:23.643519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:30:23.643519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:30:23.643519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:30:23.643519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:30:23.643519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:30:23.643519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:30:23.643519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:30:23.643519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:30:23.643519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:30:23.643519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:30:23.643519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:30:23.643519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:30:23.643519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:30:23.644159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:30:23.644159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:30:23.644159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:23.644159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:30:23.644159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:23.644159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:30:23.644159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:23.644159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:30:23.644159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:23.644159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:30:23.644159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:23.644159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:30:23.644159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:30:23.644159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:23.644159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:30:23.644159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:23.644159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:30:23.644159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:30:23.644159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:23.644795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:30:23.644795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:23.644795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:30:23.646590 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:30:23.646898 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:30:23.647025 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:30:23.647154 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:30:23.647763 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:30:23.647999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:23.648679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:30:23.648679 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:23.648679 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:30:23.648679 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:30:23.648679 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:23.648679 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:30:23.648679 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:23.648679 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:30:23.649500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:23.649500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:30:23.649500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:30:23.649500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:30:23.649500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:23.649500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:30:23.649500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:23.649500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:30:23.649500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:23.649500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:30:23.649500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:23.649500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:30:23.649500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:23.649500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:30:23.649500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:30:23.650018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:23.650018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:30:23.650018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:23.650018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:30:23.650018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:30:23.650018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:23.650018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:30:23.650018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:23.650018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:30:23.650512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:23.650512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:30:23.650512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:30:23.650512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:30:23.650512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:30:23.650512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:30:23.650512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:30:23.650512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:30:23.650512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:30:23.650512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:30:23.650512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:23.650512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:30:23.650916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:23.650916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:30:23.650916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:23.650916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:30:23.650916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:23.650916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:30:23.650916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:23.650916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:30:23.650916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:30:23.650916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:23.650916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:30:23.650916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:23.650916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:30:23.650916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:30:23.651287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:23.651287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:30:23.651287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:23.651287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:30:23.651704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:23.651704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:30:23.651704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:30:23.651704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:30:23.651704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:30:23.651704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:30:23.651704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:30:23.651704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:30:23.651704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:30:23.651704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:30:23.651704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:30:23.651704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:30:23.651704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:30:23.651704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:30:23.651704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:30:23.651704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:30:23.652338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:30:23.652338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:30:23.652338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:30:23.652338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:30:23.652338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:30:23.652338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:23.652338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:30:23.652338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:23.652338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:30:23.652338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:23.652338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:30:23.652338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:23.652338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:30:23.652338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:23.652338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:30:23.652338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:30:23.652338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:23.652338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:30:23.652897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:23.652897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:30:23.652897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:30:23.652897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:23.652897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:30:23.652897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:23.652897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:30:23.653326 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:30:23.653379 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:30:23.653574 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:30:23.653783 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:30:23.654414 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:30:23.654908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:23.656547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:30:23.656547 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:23.656547 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:30:23.656547 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:30:23.656547 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:23.656547 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:30:23.656547 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:23.656547 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:30:23.657674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:23.657674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:30:23.657674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:30:23.657674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:30:23.657674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:23.657674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:30:23.657674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:23.657674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:30:23.657674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:23.657674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:30:23.657674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:23.657674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:30:23.657674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:23.657674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:30:23.657674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:30:23.658128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:23.658128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:30:23.658128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:23.658128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:30:23.658128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:30:23.658128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:23.658128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:30:23.658128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:23.658128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:30:23.658973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:23.658973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:30:23.658973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:30:23.658973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:30:23.658973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:30:23.658973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:30:23.658973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:30:23.658973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:30:23.658973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:30:23.658973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:23.658973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:30:23.659316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:23.659316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:30:23.659316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:23.659316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:30:23.659316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:23.659316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:30:23.659316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:23.659316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:30:23.659316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:30:23.659316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:23.659316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:30:23.659316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:23.659316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:30:23.659316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:30:23.659699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:23.659699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:30:23.659699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:23.659699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:30:23.660613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:23.660613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:30:23.660613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:30:23.660613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:30:23.660613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:30:23.660613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:30:23.660613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:30:23.660613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:30:23.660613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:30:23.660613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:30:23.660613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:30:23.660613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:30:23.660613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:30:23.660613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:30:23.660613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:30:23.660613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:30:23.660613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:30:23.661131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:30:23.661131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:30:23.661131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:30:23.661131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:23.661131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:30:23.661131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:23.661131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:30:23.661131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:23.661131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:30:23.661131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:23.661131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:30:23.661131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:23.661131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:30:23.661131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:30:23.661131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:23.661131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:30:23.661131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:23.661131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:30:23.661131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:30:23.661633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:23.661633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:30:23.661633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:23.661633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:30:23.661633 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:30:23.661633 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:30:23.661633 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:30:23.661891 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:30:23.662155 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:30:23.662374 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:30:23.662529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:23.663059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:30:23.663059 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:23.663059 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:30:23.663059 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:30:23.663059 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:23.663059 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:30:23.663059 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:23.663059 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:30:23.663911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:23.663911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:30:23.663911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:30:23.663911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:30:23.663911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:30:23.663911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:30:23.663911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:30:23.663911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:23.663911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:30:23.663911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:23.663911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:30:23.663911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:23.664358 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:30:23.664358 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:23.664358 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:30:23.664358 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:23.664358 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:30:23.664358 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:30:23.664358 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:23.664358 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:30:23.664358 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:23.664358 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:30:23.664358 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:30:23.664358 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:23.664358 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:30:23.664737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:23.664737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:30:23.664737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:23.664737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:30:23.664737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:30:23.664737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:30:23.664737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:30:23.664737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:30:23.664737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:30:23.664737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:30:23.664737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:30:23.664737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:30:23.664737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:30:23.664737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:30:23.664737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:30:23.665214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:30:23.665214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:30:23.665214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:30:23.665214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:30:23.665214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:30:23.665214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:23.665214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:30:23.665214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:23.665214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:30:23.665214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:23.665214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:30:23.665214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:23.665214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:30:23.665214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:23.665214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:30:23.665214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:30:23.665214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:23.665214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:30:23.665214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:23.665765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:30:23.665765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:30:23.665765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:23.665765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:30:23.665765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:23.665765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:30:28.485646 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:30:28.486682 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:30:28.486682 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:30:28.487142 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:30:28.528381 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:30:28.528687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:28.529586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:30:28.529586 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:28.529586 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:30:28.529586 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:30:28.529586 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:28.529586 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:30:28.529586 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:28.529586 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:30:28.530657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.043 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:28.530657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:30:28.530657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:30:28.530657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:30:28.530657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:30:28.530657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:30:28.530657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:30:28.530657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:28.530657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:30:28.530657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:28.530657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:30:28.530657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:28.531353 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:30:28.531353 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:28.531353 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:30:28.531353 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:28.531353 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:30:28.531353 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:30:28.531353 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:28.531353 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:30:28.531353 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:28.531353 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:30:28.531353 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:30:28.531353 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:28.531353 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:30:28.532135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:28.532135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:30:28.532135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:28.532135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:30:28.532135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:30:28.532135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:30:28.532135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:30:28.532135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:30:28.532135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:30:28.532135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:30:28.532135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:30:28.532135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:30:28.532135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:30:28.532135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:30:28.532135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:30:28.532920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:30:28.532920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:30:28.532920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:30:28.532920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:30:28.532920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:30:28.532920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:28.532920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:30:28.532920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:28.532920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:30:28.532920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:28.532920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:30:28.532920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:28.532920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:30:28.532920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:28.532920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:30:28.532920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:30:28.532920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:28.532920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:30:28.532920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:28.533989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:30:28.533989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:30:28.533989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:28.533989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:30:28.533989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:28.533989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:30:28.639185 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:30:28.640236 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:30:28.640486 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:30:28.641793 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:30:28.642198 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:30:28.642517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:28.643478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:30:28.643478 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:28.643478 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:30:28.643478 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:30:28.643478 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:28.643478 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:30:28.643478 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:28.643478 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:30:28.645159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:28.645159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:30:28.645159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:30:28.645159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:30:28.645159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:30:28.645159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:30:28.645159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:30:28.645159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:28.645159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:30:28.645159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:28.645159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:30:28.645159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:28.646094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:30:28.646094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:28.646094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:30:28.646094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:28.646094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:30:28.646094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:30:28.646094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:28.646094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:30:28.646094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:28.646094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:30:28.646094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:30:28.646094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:28.646094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:30:28.646592 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:28.646592 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:30:28.647236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:28.647236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:30:28.647236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:30:28.647236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:30:28.647236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:30:28.647236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:30:28.647236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:30:28.647236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:30:28.647236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:30:28.647236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:30:28.647236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:30:28.647236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:30:28.647236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:30:28.647236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:30:28.647236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:30:28.647236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:30:28.647784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:30:28.647784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:30:28.647784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:28.647784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:30:28.647784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:28.647784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:30:28.647784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:28.647784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:30:28.647784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:28.647784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:30:28.647784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:28.647784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:30:28.647784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:30:28.647784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:28.647784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:30:28.647784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:28.647784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:30:28.647784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:30:28.647784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:28.648371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:30:28.648371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:28.648371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:30:28.648790 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:30:28.648998 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:30:28.649062 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:30:28.649160 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:30:28.649597 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:30:28.649838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:28.650321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:30:28.650321 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:28.650321 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:30:28.650321 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:30:28.650321 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:28.650321 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:30:28.650321 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:28.650321 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:30:28.650880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:28.650880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:30:28.650880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:30:28.650880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:30:28.650880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:28.650880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:30:28.650880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:28.650880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:30:28.650880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:28.650880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:30:28.650880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:28.650880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:30:28.650880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:28.650880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:30:28.650880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:30:28.651450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:28.651450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:30:28.651450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:28.651450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:30:28.651450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:30:28.651450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:28.651450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:30:28.651450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:28.651450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:30:28.651927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:28.651927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:30:28.651927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:30:28.651927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:30:28.651927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:30:28.651927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:30:28.651927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:30:28.651927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:30:28.651927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:30:28.651927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:30:28.651927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:28.651927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:30:28.652355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:28.652355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:30:28.652355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:28.652355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:30:28.652355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:28.652355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:30:28.652355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:28.652355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:30:28.652355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:30:28.652355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:28.652355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:30:28.652355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:28.652355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:30:28.652355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:30:28.652864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:28.652864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:30:28.652864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:28.652864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:30:28.652864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:28.652864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:30:28.652864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:30:28.652864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:30:28.652864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:30:28.652864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:30:28.652864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:30:28.652864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:30:28.652864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:30:28.652864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:30:28.653797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:30:28.653797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:30:28.653797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:30:28.653797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:30:28.653797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:30:28.653797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:30:28.653797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:30:28.653797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:30:28.653797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:30:28.653797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:30:28.653797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:30:28.653797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:28.653797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:30:28.653797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:28.653797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:30:28.653797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:28.653797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:30:28.653797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:28.653797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:30:28.654873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:28.654873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:30:28.654873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:30:28.654873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:28.654873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:30:28.654873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:28.654873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:30:28.654873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:30:28.654873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:28.654873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:30:28.654873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:28.654873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:30:28.654873 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:30:28.654873 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:30:28.655399 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:30:28.655399 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:30:28.655399 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:30:28.655399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:28.655399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:30:28.655399 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:28.655399 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:30:28.655399 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:30:28.655399 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:28.655399 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:30:28.655399 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:28.655399 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:30:28.656012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:28.656012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:30:28.656012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:30:28.656012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:30:28.656012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:28.656012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:30:28.656012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:28.656012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:30:28.656012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:28.656012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:30:28.656012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:28.656012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:30:28.656012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:28.656012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:30:28.656012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:30:28.656509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:28.656509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:30:28.656509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:28.656509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:30:28.656509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:30:28.656509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:28.656509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:30:28.656509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:28.656509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:30:28.656509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:28.656509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:30:28.656509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:30:28.656509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:30:28.656509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:30:28.657088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:30:28.657088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:30:28.657088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:30:28.657088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:30:28.657088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:28.657088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:30:28.657088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:28.657088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:30:28.657088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:28.657088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:30:28.657088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:28.657088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:30:28.657088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:28.657600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:30:28.657600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:30:28.657600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:28.657600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:30:28.657600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:28.657600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:30:28.657600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:30:28.657600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:28.657600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:30:28.657600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:28.657600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:30:28.657600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:28.657966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:30:28.657966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:30:28.657966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:30:28.657966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:30:28.657966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:30:28.657966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:30:28.657966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:30:28.657966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:30:28.657966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:30:28.657966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:30:28.657966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:30:28.657966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:30:28.657966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:30:28.657966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:30:28.657966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:30:28.657966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:30:28.657966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:30:28.657966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:30:28.657966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:30:28.658384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:28.658384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:30:28.658384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:28.658384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:30:28.658384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:28.658384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:30:28.658384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:28.658384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:30:28.658384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:28.658384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:30:28.658384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:30:28.658384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:28.658384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:30:28.658384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:28.658384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:30:28.658384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:30:28.658384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:28.658384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:30:28.658384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:28.658384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:30:28.661599 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:30:28.661834 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:30:28.661945 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:30:28.662108 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:30:28.662575 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:30:28.663993 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:30:28.664217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:28.667412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:30:28.667412 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:28.667412 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:30:28.667412 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:30:28.667412 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:28.667412 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:30:28.667412 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:28.667412 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:30:28.667412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:28.667412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:30:28.667412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:30:28.667412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:30:28.667412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:30:28.667412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:30:28.667412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:30:28.667915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:28.667915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:30:28.667915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:28.667915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:30:28.667915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:28.667915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:30:28.667915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:28.667915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:30:28.667915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:28.667915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:30:28.667915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:30:28.667915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:28.667915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:30:28.667915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:28.668334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:30:28.668334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:30:28.668334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:28.668334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:30:28.668334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:28.668334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:30:28.668334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:28.668334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:30:28.668334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:30:28.668334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:30:28.668334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:30:28.668334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:30:28.668334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:30:28.668334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:30:28.668756 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:30:28.668756 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:30:28.668756 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:30:28.668756 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:30:28.668756 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:30:28.668756 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:30:28.668756 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:30:28.668756 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:30:28.668756 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:30:28.668756 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:30:28.668756 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:28.668756 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:30:28.668756 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:28.668756 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:30:28.668756 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:28.668756 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:30:28.668756 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:28.668756 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:30:28.668756 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:28.668756 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:30:28.669289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:30:28.669289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:28.669289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:30:28.669289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:28.669289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:30:28.669289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:30:28.669289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:28.669289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:30:28.669289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:28.669289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:30:33.486011 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:30:33.486556 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:30:33.486604 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:30:33.489523 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:30:33.523656 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:30:33.523877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:33.524340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:30:33.524340 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:33.524340 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:30:33.524340 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:30:33.524340 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:33.524340 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:30:33.524340 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:33.524340 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:30:33.525051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:33.525051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:30:33.525051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:30:33.525051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:30:33.525051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:30:33.525051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:30:33.525051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:30:33.525051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:33.525051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:30:33.525051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:33.525051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:30:33.525051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:33.525684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:30:33.525684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:33.525684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:30:33.525684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:33.525684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:30:33.525684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:30:33.525684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:33.525684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:30:33.525684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:33.525684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:30:33.525684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:30:33.525684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:33.525684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:30:33.526334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:33.526334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:30:33.526334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:33.526334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:30:33.526334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:30:33.526334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:30:33.526334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:30:33.526334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:30:33.526334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:30:33.526334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:30:33.526334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:30:33.526334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:30:33.526334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:30:33.526334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:30:33.526334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:30:33.527118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:30:33.527118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:30:33.527118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:30:33.527118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:30:33.527118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:30:33.527118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:33.527118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:30:33.527118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:33.527118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:30:33.527118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:33.527118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:30:33.527118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:33.527118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:30:33.527118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:33.527118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:30:33.527118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:30:33.527118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:33.527118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:30:33.527118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:33.527974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:30:33.527974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:30:33.527974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:33.527974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:30:33.527974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:33.527974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:30:33.639940 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:30:33.640503 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:30:33.640625 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:30:33.641305 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:30:33.641497 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:30:33.641652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:33.642282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:30:33.642282 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:33.642282 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:30:33.642282 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:30:33.642282 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:33.642282 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:30:33.642282 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:33.642282 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:30:33.642971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:33.642971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:30:33.642971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:30:33.642971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:30:33.642971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:30:33.642971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:30:33.642971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:30:33.642971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:33.642971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:30:33.642971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:33.642971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:30:33.642971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:33.643307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:30:33.643307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:33.643307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:30:33.643307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:33.643307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:30:33.643307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:30:33.643307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:33.643307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:30:33.643307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:33.643307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:30:33.643307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:30:33.643307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:33.643307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:30:33.643549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:33.643549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:30:33.643721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:33.643721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:30:33.643721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:30:33.643721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:30:33.643721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:30:33.643721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:30:33.643721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:30:33.643721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:30:33.643721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:30:33.643721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:30:33.643721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:30:33.643721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:30:33.643721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:30:33.643721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:30:33.643721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:30:33.643721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:30:33.644113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:30:33.644113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:30:33.644113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:33.644113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:30:33.644113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:33.644113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:30:33.644113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:33.644113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:30:33.644113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:33.644113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:30:33.644113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:33.644113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:30:33.644113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:30:33.644113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:33.644113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:30:33.644113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:33.644113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:30:33.644113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:30:33.644113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:33.644513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:30:33.644513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:33.644513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:30:33.648997 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:30:33.649149 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:30:33.649217 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:30:33.649314 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:30:33.649567 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:30:33.649714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:33.650097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:30:33.650097 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:33.650097 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:30:33.650097 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:30:33.650097 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:33.650097 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:30:33.650097 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:33.650097 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:30:33.651256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:33.651256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:30:33.651256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:30:33.651256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:30:33.651256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:33.651256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:30:33.651256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:33.651256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:30:33.651256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:33.651256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:30:33.651256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:33.651256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:30:33.651256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:33.651256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:30:33.651256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:30:33.651896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:33.651896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:30:33.651896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:33.651896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:30:33.651896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:30:33.651896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:33.651896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:30:33.651896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:33.651896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:30:33.652609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:33.652609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:30:33.652609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:30:33.652609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:30:33.652609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:30:33.652609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:30:33.652609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:30:33.652609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:30:33.652609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:30:33.652609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:30:33.652609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:33.652609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:30:33.653122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:33.653122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:30:33.653122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:33.653122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:30:33.653122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:33.653122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:30:33.653122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:33.653122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:30:33.653122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:30:33.653122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:33.653122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:30:33.653122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:33.653122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:30:33.653122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:30:33.653659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:33.653659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:30:33.653659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:33.653659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:30:33.655010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:33.655010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:30:33.655010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:30:33.655010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:30:33.655010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:30:33.655010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:30:33.655010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:30:33.655010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:30:33.655010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:30:33.655010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:30:33.655010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:30:33.655010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:30:33.655010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:30:33.655010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:30:33.655010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:30:33.655010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:30:33.655620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:30:33.655620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:30:33.655620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:30:33.655620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:30:33.655620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:30:33.655620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:33.655620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:30:33.655620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:33.655620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:30:33.655620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:33.655620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:30:33.655620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:33.655620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:30:33.655620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:33.655620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:30:33.655620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:30:33.655620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:33.655620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:30:33.656180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:33.656180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:30:33.656180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:30:33.656180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:33.656180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:30:33.656180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:33.656180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:30:33.657156 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:30:33.657627 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:30:33.657765 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:30:33.657910 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:30:33.658470 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:30:33.658708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:33.659420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:30:33.659420 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:33.659420 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:30:33.659420 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:30:33.659420 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:33.659420 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:30:33.659420 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:33.659420 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:30:33.660890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:33.660890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:30:33.660890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:30:33.660890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:30:33.660890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:33.660890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:30:33.660890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:33.660890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:30:33.660890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:33.660890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:30:33.660890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:33.660890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:30:33.660890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:33.660890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:30:33.660890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:30:33.661378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:33.661378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:30:33.661378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:33.661378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:30:33.661378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:30:33.661378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:33.661378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:30:33.661378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:33.661378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:30:33.661378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:33.661378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:30:33.661378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:30:33.661378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:30:33.661378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:30:33.661664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:30:33.661664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:30:33.661664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:30:33.661664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:30:33.661664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:33.661664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:30:33.661664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:33.661664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:30:33.661664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:33.661664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:30:33.661664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:33.661664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:30:33.661664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:33.661915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:30:33.661915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:30:33.661915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:33.661915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:30:33.661915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:33.661915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:30:33.661915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:30:33.661915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:33.661915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:30:33.661915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:33.661915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:30:33.662838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:33.662838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:30:33.662838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:30:33.662838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:30:33.662838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:30:33.662838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:30:33.662838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:30:33.662838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:30:33.662838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:30:33.662838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:30:33.662838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:30:33.662838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:30:33.662838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:30:33.662838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:30:33.662838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:30:33.662838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:30:33.662838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:30:33.663210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:30:33.663210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:30:33.663210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:30:33.663210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:33.663210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:30:33.663210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:33.663210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:30:33.663210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:33.663210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:30:33.663210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:33.663210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:30:33.663210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:33.663210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:30:33.663210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:30:33.663210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:33.663210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:30:33.663210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:33.663210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:30:33.663210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:30:33.663564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:33.663564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:30:33.663564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:33.663564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:30:33.663564 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:30:33.663709 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:30:33.664033 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:30:33.664241 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:30:33.664561 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:30:33.664782 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:30:33.664949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:33.665433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:30:33.665433 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:33.665433 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:30:33.665433 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:30:33.665433 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:33.665433 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:30:33.665433 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:33.665433 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:30:33.666552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:33.666552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:30:33.666552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:30:33.666552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:30:33.666552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:30:33.666552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:30:33.666552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:30:33.666552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:33.666552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:30:33.666552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:33.666552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:30:33.666552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:33.666856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:30:33.666856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:33.666856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:30:33.666856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:33.666856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:30:33.666856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:30:33.666856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:33.666856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:30:33.666856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:33.666856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:30:33.666856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:30:33.666856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:33.666856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:30:33.667137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:33.667137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:30:33.667543 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:33.667543 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:30:33.667543 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:30:33.667543 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:30:33.667543 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:30:33.667543 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:30:33.667543 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:30:33.667543 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:30:33.667543 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:30:33.667543 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:30:33.667543 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:30:33.667543 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:30:33.667543 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:30:33.667543 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:30:33.667543 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:30:33.667543 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:30:33.668115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:30:33.668115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:30:33.668115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:33.668115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:30:33.668115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:33.668115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:30:33.668115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:33.668115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:30:33.668115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:33.668115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:30:33.668115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:33.668115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:30:33.668115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:30:33.668115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:33.668115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:30:33.668115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:33.668115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:30:33.668115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:30:33.668115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:33.668812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:30:33.668812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:33.668812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:30:38.486646 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:30:38.487430 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:30:38.487430 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:30:38.488992 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:30:38.529559 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:30:38.529962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:38.531315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:30:38.531315 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:38.531315 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:30:38.531315 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:30:38.531315 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:38.531315 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:30:38.531315 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:38.531315 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:30:38.532355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.044 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:38.532355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:30:38.532355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:30:38.532355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:30:38.532355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:30:38.532355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:30:38.532355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:30:38.532355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:38.532355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:30:38.532355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:38.532355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:30:38.532355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:38.532804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:30:38.532804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:38.532804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:30:38.532804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:38.532804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:30:38.532804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:30:38.532804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:38.532804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:30:38.532804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:38.532804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:30:38.532804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:30:38.532804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:38.532804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:30:38.533178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:38.533178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:30:38.533779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:38.533779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:30:38.533779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:30:38.533779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:30:38.533779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:30:38.533779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:30:38.533779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:30:38.533779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:30:38.533779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:30:38.533779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:30:38.533779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:30:38.533779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:30:38.533779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:30:38.533779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:30:38.533779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:30:38.533779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:30:38.534481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:30:38.534481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:30:38.534481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:38.534481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:30:38.534481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:38.534481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:30:38.534481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:38.534481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:30:38.534481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:38.534481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:30:38.534481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:38.534481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:30:38.534481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:30:38.534481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:38.534481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:30:38.534481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:38.534481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:30:38.534481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:30:38.534481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:38.535324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:30:38.535324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:38.535324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:30:38.640995 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:30:38.641547 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:30:38.641769 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:30:38.643116 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:30:38.643510 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:30:38.643890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:38.644944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:30:38.644944 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:38.644944 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:30:38.644944 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:30:38.644944 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:38.644944 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:30:38.644944 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:38.644944 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:30:38.646630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:38.646630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:30:38.646630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:30:38.646630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:30:38.646630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:30:38.646630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:30:38.646630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:30:38.646630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:38.646630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:30:38.646630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:38.646630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:30:38.646630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:38.647252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:30:38.647252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:38.647252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:30:38.647252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:38.647252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:30:38.647252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:30:38.647252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:38.647252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:30:38.647252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:38.647252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:30:38.647252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:30:38.647252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:38.647252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:30:38.647854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:38.647854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:30:38.648490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:38.648490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:30:38.648490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:30:38.648490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:30:38.648490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:30:38.648490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:30:38.648490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:30:38.648490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:30:38.648490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:30:38.648490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:30:38.648490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:30:38.648490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:30:38.648490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:30:38.648490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:30:38.648490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:30:38.648490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:30:38.649258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:30:38.649258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:30:38.649258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:38.649258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:30:38.649258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:38.649258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:30:38.649258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:38.649258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:30:38.649258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:38.649258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:30:38.649258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:38.649258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:30:38.649258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:30:38.649258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:38.649258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:30:38.649258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:38.649258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:30:38.649258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:30:38.649258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:38.650044 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:30:38.650044 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:38.650044 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:30:38.650044 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:30:38.650044 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:30:38.651084 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:30:38.651084 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:30:38.651289 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:30:38.651653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:38.652766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:30:38.652766 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:38.652766 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:30:38.652766 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:30:38.652766 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:38.652766 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:30:38.652766 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:38.652766 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:30:38.653760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:38.653760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:30:38.653760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:30:38.653760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:30:38.653760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:38.653760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:30:38.653760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:38.653760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:30:38.653760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:38.653760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:30:38.653760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:38.653760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:30:38.653760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:38.653760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:30:38.653760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:30:38.655014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:38.655014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:30:38.655014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:38.655014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:30:38.655014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:30:38.655014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:38.655014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:30:38.655014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:38.655014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:30:38.655014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:38.655014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:30:38.655014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:30:38.655014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:30:38.655014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:30:38.655915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:30:38.655915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:30:38.655915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:30:38.655915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:30:38.655915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:30:38.655915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:38.655915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:30:38.655915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:38.655915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:30:38.655915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:38.655915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:30:38.655915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:38.655915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:30:38.657289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:38.657289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:30:38.657289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:30:38.657289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:38.657289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:30:38.657289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:38.657289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:30:38.657289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:30:38.657289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:38.657289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:30:38.657289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:38.657289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:30:38.659692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:38.659692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:30:38.659692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:30:38.659692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:30:38.659692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:30:38.659692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:30:38.659692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:30:38.659692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:30:38.659692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:30:38.659692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:30:38.659692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:30:38.659692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:30:38.659692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:30:38.659692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:30:38.659692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:30:38.659692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:30:38.660270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:30:38.660270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:30:38.660270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:30:38.660270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:30:38.660270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:30:38.660270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:38.660270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:30:38.660270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:38.660270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:30:38.660270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:38.660270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:30:38.660270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:38.660270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:30:38.660270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:38.660270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:30:38.660270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:30:38.660270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:38.660270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:30:38.660822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:38.660822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:30:38.660822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:30:38.660822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:38.660822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:30:38.660822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:38.660822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:30:38.660822 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:30:38.660822 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:30:38.660822 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:30:38.661024 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:30:38.661024 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:30:38.661024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:38.661024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:30:38.661024 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:38.661024 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:30:38.661024 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:30:38.661024 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:38.661024 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:30:38.661024 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:38.661024 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:30:38.661024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:38.661265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:30:38.661265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:30:38.661265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:30:38.661265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:38.661265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:30:38.661265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:38.661265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:30:38.661265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:38.661265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:30:38.661265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:38.661265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:30:38.661265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:38.661265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:30:38.661265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:30:38.661265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:38.661265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:30:38.661265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:38.661558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:30:38.661558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:30:38.661558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:38.661558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:30:38.661558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:38.661558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:30:38.661558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:38.661558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:30:38.661558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:30:38.661558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:30:38.661558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:30:38.661558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:30:38.661558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:30:38.661824 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:30:38.661824 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:30:38.661824 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:38.661824 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:30:38.661824 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:38.661824 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:30:38.661824 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:38.661824 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:30:38.661824 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:38.661824 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:30:38.661824 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:38.661824 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:30:38.661824 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:30:38.662052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:38.662052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:30:38.662052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:38.662052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:30:38.662052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:30:38.662052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:38.662052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:30:38.662052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:38.662052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:30:38.662213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:38.662213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:30:38.662213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:30:38.662213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:30:38.662213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:30:38.662213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:30:38.662213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:30:38.662213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:30:38.662213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:30:38.662213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:30:38.662213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:30:38.662213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:30:38.662213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:30:38.662213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:30:38.662213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:30:38.662213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:30:38.662213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:30:38.662542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:30:38.662542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:30:38.662542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:30:38.662542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:38.662542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:30:38.662542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:38.662542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:30:38.662542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:38.662542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:30:38.662542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:38.662542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:30:38.662542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:38.662542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:30:38.662542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:30:38.662542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:38.662542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:30:38.662542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:38.662542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:30:38.662542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:30:38.664089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:38.664089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:30:38.664089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:38.664089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:30:38.664089 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:30:38.664089 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:30:38.664089 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:30:38.664260 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:30:38.664729 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:30:38.665154 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:30:38.665154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:38.665674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:30:38.665674 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:38.665674 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:30:38.665674 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:30:38.665674 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:38.665674 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:30:38.665674 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:38.665674 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:30:38.667350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:38.667350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:30:38.667350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:30:38.667350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:30:38.667350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:30:38.667350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:30:38.667350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:30:38.667350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:38.667350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:30:38.667350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:38.667350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:30:38.667350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:38.667645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:30:38.667645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:38.667645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:30:38.667645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:38.667645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:30:38.667645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:30:38.667645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:38.667645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:30:38.667645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:38.667645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:30:38.667645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:30:38.667645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:38.667645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:30:38.667914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:38.667914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:30:38.667914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:38.667914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:30:38.667914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:30:38.667914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:30:38.667914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:30:38.667914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:30:38.667914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:30:38.667914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:30:38.667914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:30:38.667914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:30:38.667914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:30:38.667914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:30:38.667914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:30:38.668184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:30:38.668184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:30:38.668184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:30:38.668184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:30:38.668184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:30:38.668184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:38.668184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:30:38.668184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:38.668184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:30:38.668184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:38.668184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:30:38.668184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:38.668184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:30:38.668184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:38.668184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:30:38.668184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:30:38.668184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:38.668184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:30:38.668184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:38.668520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:30:38.668520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:30:38.668520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:38.668520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:30:38.668520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:38.668520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:30:43.487254 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:30:43.487937 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:30:43.487937 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:30:43.489156 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:30:43.524667 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:30:43.525207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:43.526382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:30:43.526382 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:43.526382 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:30:43.526382 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:30:43.526382 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:43.526382 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:30:43.526382 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:43.526382 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:30:43.527525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:43.527525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:30:43.527525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:30:43.527525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:30:43.527525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:30:43.527525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:30:43.527525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:30:43.527525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:43.527525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:30:43.527525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:43.527525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:30:43.527525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:43.528271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:30:43.528271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:43.528271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:30:43.528271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:43.528271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:30:43.528271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:30:43.528271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:43.528271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:30:43.528271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:43.528271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:30:43.528271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:30:43.528271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:43.528271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:30:43.528971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:43.528971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:30:43.528971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:43.528971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:30:43.528971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:30:43.528971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:30:43.528971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:30:43.528971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:30:43.528971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:30:43.528971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:30:43.528971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:30:43.528971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:30:43.528971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:30:43.528971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:30:43.528971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:30:43.529864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:30:43.529864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:30:43.529864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:30:43.529864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:30:43.529864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:30:43.529864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:43.529864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:30:43.529864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:43.529864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:30:43.529864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:43.529864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:30:43.529864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:43.529864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:30:43.529864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:43.529864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:30:43.529864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:30:43.529864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:43.529864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:30:43.529864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:43.531665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:30:43.531665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:30:43.531665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:43.531665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:30:43.531665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:43.531665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:30:43.641732 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:30:43.642361 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:30:43.642695 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:30:43.643778 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:30:43.644074 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:30:43.644210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:43.645237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:30:43.645237 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:43.645237 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:30:43.645237 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:30:43.645237 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:43.645237 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:30:43.645237 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:43.645237 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:30:43.647372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:43.647372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:30:43.647372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:30:43.647372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:30:43.647372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:30:43.647372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:30:43.647372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:30:43.647372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:43.647372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:30:43.647372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:43.647372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:30:43.647372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:43.648018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:30:43.648018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:43.648018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:30:43.648018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:43.648018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:30:43.648018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:30:43.648018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:43.648018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:30:43.648018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:43.648018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:30:43.648018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:30:43.648018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:43.648018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:30:43.648560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:43.648560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:30:43.649782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:43.649782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:30:43.649782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:30:43.649782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:30:43.649782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:30:43.649782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:30:43.649782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:30:43.649782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:30:43.649782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:30:43.649782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:30:43.649782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:30:43.649782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:30:43.649782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:30:43.649782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:30:43.649782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:30:43.649782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:30:43.650561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:30:43.650561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:30:43.650561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:43.650561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:30:43.650561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:43.650561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:30:43.650561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:43.650561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:30:43.650561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:43.650561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:30:43.650561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:43.650561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:30:43.650561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:30:43.650561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:43.650561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:30:43.650561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:43.650561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:30:43.650561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:30:43.650561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:43.651395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:30:43.651395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:43.651395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:30:43.651395 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:30:43.651395 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:30:43.651395 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:30:43.651395 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:30:43.651395 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:30:43.651836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:43.652204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:30:43.652204 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:43.652204 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:30:43.652204 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:30:43.652204 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:43.652204 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:30:43.652204 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:43.652204 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:30:43.653361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:43.653361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:30:43.653361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:30:43.653361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:30:43.653361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:43.653361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:30:43.653361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:43.653361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:30:43.653361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:43.653361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:30:43.653361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:43.653361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:30:43.653361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:43.653361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:30:43.653361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:30:43.654184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:43.654184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:30:43.654184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:43.654184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:30:43.654184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:30:43.654184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:43.654184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:30:43.654184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:43.654184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:30:43.654617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:43.654617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:30:43.654617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:30:43.654617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:30:43.654617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:30:43.654617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:30:43.654617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:30:43.654617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:30:43.654617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:30:43.654617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:30:43.654617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:43.654617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:30:43.655237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:43.655237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:30:43.655237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:43.655237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:30:43.655237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:43.655237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:30:43.655237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:43.655237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:30:43.655237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:30:43.655237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:43.655237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:30:43.655237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:43.655237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:30:43.655237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:30:43.655849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:43.655849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:30:43.655849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:43.655849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:30:43.655849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:43.655849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:30:43.655849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:30:43.655849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:30:43.655849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:30:43.655849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:30:43.655849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:30:43.655849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:30:43.655849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:30:43.655849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:30:43.656668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:30:43.656668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:30:43.656668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:30:43.656668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:30:43.656668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:30:43.656668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:30:43.656668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:30:43.656668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:30:43.656668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:30:43.656668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:30:43.656668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:30:43.656668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:43.656668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:30:43.656668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:43.656668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:30:43.656668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:43.656668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:30:43.656668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:43.656668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:30:43.657642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:43.657642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:30:43.657642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:30:43.657642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:43.657642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:30:43.657642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:43.657642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:30:43.657642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:30:43.657642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:43.657642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:30:43.657642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:43.657642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:30:43.657983 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:30:43.658705 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:30:43.658839 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:30:43.658989 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:30:43.659759 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:30:43.660231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:43.661195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:30:43.661195 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:43.661195 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:30:43.661195 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:30:43.661195 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:43.661195 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:30:43.661195 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:43.661195 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:30:43.662054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:43.662054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:30:43.662054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:30:43.662054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:30:43.662054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:43.662054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:30:43.662054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:43.662054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:30:43.662054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:43.662054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:30:43.662054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:43.662054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:30:43.662054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:43.662054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:30:43.662054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:30:43.662493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:43.662493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:30:43.662493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:43.662493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:30:43.662493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:30:43.662493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:43.662493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:30:43.662493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:43.662493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:30:43.662946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:43.662946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:30:43.662946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:30:43.662946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:30:43.662946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:30:43.662946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:30:43.662946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:30:43.662946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:30:43.662946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:30:43.662946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:43.662946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:30:43.663286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:43.663286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:30:43.663286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:43.663286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:30:43.663286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:43.663286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:30:43.663286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:43.663286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:30:43.663286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:30:43.663286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:43.663286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:30:43.663286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:43.663286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:30:43.663286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:30:43.663659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:43.663659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:30:43.663659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:43.663659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:30:43.664023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:43.664023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:30:43.664023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:30:43.664023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:30:43.664023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:30:43.664023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:30:43.664023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:30:43.664023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:30:43.664023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:30:43.664023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:30:43.664023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:30:43.664023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:30:43.664023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:30:43.664023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:30:43.664023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:30:43.664023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:30:43.664023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:30:43.664494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:30:43.664494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:30:43.664494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:30:43.664494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:43.664494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:30:43.664494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:43.664494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:30:43.664494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:43.664494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:30:43.664494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:43.664494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:30:43.664494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:43.664494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:30:43.664494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:30:43.664494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:43.664494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:30:43.664494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:43.664494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:30:43.664494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:30:43.665071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:43.665071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:30:43.665071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:43.665071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:30:43.665071 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:30:43.665071 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:30:43.665071 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:30:43.665071 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:30:43.665543 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:30:43.665843 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:30:43.665998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:43.666522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:30:43.666522 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:43.666522 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:30:43.666522 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:30:43.666522 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:43.666522 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:30:43.666522 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:43.666522 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:30:43.667399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:43.667399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:30:43.667399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:30:43.667399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:30:43.667399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:30:43.667399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:30:43.667399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:30:43.667399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:43.667399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:30:43.667399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:43.667399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:30:43.667399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:43.667778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:30:43.667778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:43.667778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:30:43.667778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:43.667778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:30:43.667778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:30:43.667778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:43.667778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:30:43.667778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:43.667778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:30:43.667778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:30:43.667778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:43.667778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:30:43.668117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:43.668117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:30:43.668372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:43.668372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:30:43.668372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:30:43.668372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:30:43.668372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:30:43.668372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:30:43.668372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:30:43.668372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:30:43.668372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:30:43.668372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:30:43.668372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:30:43.668372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:30:43.668372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:30:43.668372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:30:43.668372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:30:43.668372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:30:43.668892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:30:43.668892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:30:43.668892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:43.668892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:30:43.668892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:43.668892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:30:43.668892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:43.668892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:30:43.668892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:43.668892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:30:43.668892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:43.668892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:30:43.668892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:30:43.668892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:43.668892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:30:43.668892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:43.668892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:30:43.668892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:30:43.668892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:43.669452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:30:43.669452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:43.669452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:30:48.487785 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:30:48.488728 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:30:48.488728 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:30:48.490161 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:30:48.521703 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:30:48.522204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:48.523195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:30:48.523195 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:48.523195 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:30:48.523195 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:30:48.523195 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:48.523195 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:30:48.523195 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:48.523195 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:30:48.524025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:48.524025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:30:48.524025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:30:48.524025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:30:48.524025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:30:48.524025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:30:48.524025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:30:48.524025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:48.524025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:30:48.524025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:48.524025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:30:48.524025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:48.524703 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:30:48.524703 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:48.524703 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:30:48.524703 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:48.524703 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:30:48.524703 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:30:48.524703 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:48.524703 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:30:48.524703 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:48.524703 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:30:48.524703 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:30:48.524703 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:48.524703 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:30:48.525431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:48.525431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:30:48.525431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:48.525431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:30:48.525431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:30:48.525431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:30:48.525431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:30:48.525431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:30:48.525431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:30:48.525431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:30:48.525431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:30:48.525431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:30:48.525431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:30:48.525431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:30:48.525431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:30:48.526392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:30:48.526392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:30:48.526392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:30:48.526392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:30:48.526392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:30:48.526392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:48.526392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:30:48.526392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:48.526392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:30:48.526392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:48.526392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:30:48.526392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:48.526392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:30:48.526392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:48.526392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:30:48.526392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:30:48.526392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:48.526392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:30:48.526392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:48.527549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:30:48.527549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:30:48.527549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:48.527549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:30:48.527549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:48.527549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:30:48.642512 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:30:48.642876 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:30:48.643069 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:30:48.644337 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:30:48.644701 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:30:48.644992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:48.645910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:30:48.645910 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:48.645910 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:30:48.645910 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:30:48.645910 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:48.645910 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:30:48.645910 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:48.645910 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:30:48.647516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:48.647516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:30:48.647516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:30:48.647516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:30:48.647516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:30:48.647516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:30:48.647516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:30:48.647516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:48.647516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:30:48.647516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:48.647516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:30:48.647516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:48.648206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:30:48.648206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:48.648206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:30:48.648206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:48.648206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:30:48.648206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:30:48.648206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:48.648206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:30:48.648206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:48.648206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:30:48.648206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:30:48.648206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:48.648206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:30:48.648850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:48.648850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:30:48.649205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:48.649205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:30:48.649205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:30:48.649205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:30:48.649205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:30:48.649205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:30:48.649205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:30:48.649205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:30:48.649205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:30:48.649205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:30:48.649205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:30:48.649205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:30:48.649205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:30:48.649205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:30:48.649205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:30:48.649205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:30:48.650026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:30:48.650026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:30:48.650026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:48.650026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:30:48.650026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:48.650026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:30:48.650026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:48.650026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:30:48.650026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:48.650026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:30:48.650026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:48.650026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:30:48.650026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:30:48.650026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:48.650026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:30:48.650026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:48.650026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:30:48.650026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:30:48.650026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:48.650907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:30:48.650907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:48.650907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:30:48.650907 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:30:48.650907 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:30:48.650907 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:30:48.650907 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:30:48.651469 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:30:48.651767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:48.652339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:30:48.652339 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:48.652339 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:30:48.652339 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:30:48.652339 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:48.652339 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:30:48.652339 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:48.652339 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:30:48.653092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:48.653092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:30:48.653092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:30:48.653092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:30:48.653092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:48.653092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:30:48.653092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:48.653092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:30:48.653092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:48.653092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:30:48.653092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:48.653092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:30:48.653092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:48.653092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:30:48.653092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:30:48.653568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:48.653568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:30:48.653568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:48.653568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:30:48.653568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:30:48.653568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:48.653568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:30:48.653568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:48.653568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:30:48.653885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:48.653885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:30:48.653885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:30:48.653885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:30:48.653885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:30:48.653885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:30:48.653885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:30:48.653885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:30:48.653885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:30:48.653885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:30:48.653885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:48.653885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:30:48.654240 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:48.654240 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:30:48.654240 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:48.654240 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:30:48.654240 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:48.654240 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:30:48.654240 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:48.654240 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:30:48.654240 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:30:48.654240 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:48.654240 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:30:48.654240 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:48.654240 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:30:48.654240 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:30:48.654632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:48.654632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:30:48.654632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:48.654632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:30:48.654907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:48.654907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:30:48.654907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:30:48.654907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:30:48.654907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:30:48.654907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:30:48.654907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:30:48.654907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:30:48.654907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:30:48.654907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:30:48.654907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:30:48.654907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:30:48.654907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:30:48.654907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:30:48.654907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:30:48.654907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:30:48.655408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:30:48.655408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:30:48.655408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:30:48.655408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:30:48.655408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:30:48.655408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:48.655408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:30:48.655408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:48.655408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:30:48.655408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:48.655408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:30:48.655408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:48.655408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:30:48.655408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:48.655408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:30:48.655408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:30:48.655408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:48.655408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:30:48.655884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:48.655884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:30:48.655884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:30:48.655884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:48.655884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:30:48.655884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:48.655884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:30:48.658739 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:30:48.658996 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:30:48.659085 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:30:48.659240 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:30:48.659702 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:30:48.660027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:48.660721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:30:48.660721 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:48.660721 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:30:48.660721 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:30:48.660721 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:48.660721 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:30:48.660721 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:48.660721 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:30:48.661644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:48.661644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:30:48.661644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:30:48.661644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:30:48.661644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:48.661644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:30:48.661644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:48.661644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:30:48.661644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:48.661644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:30:48.661644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:48.661644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:30:48.661644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:48.661644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:30:48.661644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:30:48.662125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:48.662125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:30:48.662125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:48.662125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:30:48.662125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:30:48.662125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:48.662125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:30:48.662125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:48.662125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:30:48.662610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:48.662610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:30:48.662610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:30:48.662610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:30:48.662610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:30:48.662610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:30:48.662610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:30:48.662610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:30:48.662610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:30:48.662610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:48.662610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:30:48.662992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:48.662992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:30:48.662992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:48.662992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:30:48.662992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:48.662992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:30:48.662992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:48.662992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:30:48.662992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:30:48.662992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:48.662992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:30:48.662992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:48.662992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:30:48.662992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:30:48.663526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:48.663526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:30:48.663526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:48.663526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:30:48.663821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:48.663821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:30:48.663821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:30:48.663821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:30:48.663821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:30:48.663821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:30:48.663821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:30:48.663821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:30:48.663821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:30:48.663821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:30:48.663821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:30:48.663821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:30:48.663821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:30:48.663821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:30:48.663821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:30:48.663821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:30:48.663821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:30:48.664341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:30:48.664341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:30:48.664341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:30:48.664341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:48.664341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:30:48.664341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:48.664341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:30:48.664341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:48.664341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:30:48.664341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:48.664341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:30:48.664341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:48.664341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:30:48.664341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:30:48.664341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:48.664341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:30:48.664341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:48.664341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:30:48.664341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:30:48.665020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:48.665020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:30:48.665020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:48.665020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:30:48.665020 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:30:48.665222 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:30:48.665222 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:30:48.665432 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:30:48.665735 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:30:48.665994 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:30:48.666150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:48.666631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:30:48.666631 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:48.666631 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:30:48.666631 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:30:48.666631 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:48.666631 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:30:48.666631 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:48.666631 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:30:48.667558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:48.667558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:30:48.667558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:30:48.667558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:30:48.667558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:30:48.667558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:30:48.667558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:30:48.667558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:48.667558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:30:48.667558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:48.667558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:30:48.667558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:48.668047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:30:48.668047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:48.668047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:30:48.668047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:48.668047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:30:48.668047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:30:48.668047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:48.668047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:30:48.668047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:48.668047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:30:48.668047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:30:48.668047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:48.668047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:30:48.668439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:48.668439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:30:48.668941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:48.668941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:30:48.668941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:30:48.668941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:30:48.668941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:30:48.668941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:30:48.668941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:30:48.668941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:30:48.668941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:30:48.668941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:30:48.668941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:30:48.668941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:30:48.668941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:30:48.668941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:30:48.668941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:30:48.668941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:30:48.669616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:30:48.669616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:30:48.669616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:48.669616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:30:48.669616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:48.669616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:30:48.669616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:48.669616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:30:48.669616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:48.669616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:30:48.669616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:48.669616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:30:48.669616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:30:48.669616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:48.669616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:30:48.669616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:48.669616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:30:48.669616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:30:48.669616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:48.670314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:30:48.670314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:48.670314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:30:53.488453 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:30:53.488907 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:30:53.489524 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:30:53.490227 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:30:53.527798 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:30:53.528335 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:53.532661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:30:53.532661 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:53.532661 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:30:53.532661 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:30:53.532661 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:53.532661 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:30:53.532661 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:53.532661 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:30:53.532661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:53.532661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:30:53.532661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:30:53.532661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:30:53.532661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:30:53.532661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:30:53.532661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:30:53.533299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:53.533299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:30:53.533299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:53.533299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:30:53.533299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:53.533299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:30:53.533299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:53.533299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:30:53.533299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:53.533299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:30:53.533299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:30:53.533299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:53.533299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:30:53.533299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:53.533803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:30:53.533803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:30:53.533803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:53.533803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:30:53.533803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:53.533803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:30:53.533803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:53.533803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:30:53.533803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:30:53.533803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:30:53.533803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:30:53.533803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:30:53.533803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:30:53.533803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:30:53.534322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:30:53.534322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:30:53.534322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:30:53.534322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:30:53.534322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:30:53.534322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:30:53.534322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:30:53.534322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:30:53.534322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:30:53.534322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:30:53.534322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:53.534322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:30:53.534322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:53.534322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:30:53.534322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:53.534322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:30:53.534322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:53.534322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:30:53.534322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:53.534322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:30:53.534322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:30:53.535051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:53.535051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:30:53.535051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:53.535051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:30:53.535051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:30:53.535051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:53.535051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:30:53.535051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:53.535051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:30:53.643230 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:30:53.643578 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:30:53.643658 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:30:53.644460 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:30:53.644690 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:30:53.645038 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:53.645884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:30:53.645884 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:53.645884 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:30:53.645884 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:30:53.645884 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:53.645884 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:30:53.645884 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:53.645884 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:30:53.646559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:53.646559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:30:53.646559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:30:53.646559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:30:53.646559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:30:53.646559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:30:53.646559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:30:53.646559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:53.646559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:30:53.646559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:53.646559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:30:53.646559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:53.647130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:30:53.647130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:53.647130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:30:53.647130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:53.647130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:30:53.647130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:30:53.647130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:53.647130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:30:53.647130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:53.647130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:30:53.647130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:30:53.647130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:53.647130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:30:53.647526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:53.647526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:30:53.647526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:53.647526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:30:53.647526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:30:53.647526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:30:53.647526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:30:53.647526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:30:53.647526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:30:53.647526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:30:53.647526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:30:53.647526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:30:53.647526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:30:53.647526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:30:53.647526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:30:53.648080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:30:53.648080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:30:53.648080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:30:53.648080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:30:53.648080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:30:53.648080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:53.648080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:30:53.648080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:53.648080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:30:53.648080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:53.648080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:30:53.648080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:53.648080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:30:53.648080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:53.648080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:30:53.648080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:30:53.648080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:53.648080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:30:53.648080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:53.648600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:30:53.648600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:30:53.648600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:53.648600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:30:53.648600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:53.648600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:30:53.652648 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:30:53.653083 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:30:53.653083 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:30:53.653180 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:30:53.653800 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:30:53.654051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:53.654623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:30:53.654623 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:53.654623 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:30:53.654623 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:30:53.654623 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:53.654623 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:30:53.654623 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:53.654623 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:30:53.655963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:53.655963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:30:53.655963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:30:53.655963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:30:53.655963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:53.655963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:30:53.655963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:53.655963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:30:53.655963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:53.655963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:30:53.655963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:53.655963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:30:53.655963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:53.655963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:30:53.655963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:30:53.656985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:53.656985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:30:53.656985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:53.656985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:30:53.656985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:30:53.656985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:53.656985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:30:53.656985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:53.656985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:30:53.656985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:53.656985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:30:53.656985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:30:53.656985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:30:53.656985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:30:53.657852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:30:53.657852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:30:53.657852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:30:53.657852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:30:53.657852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:30:53.657852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:53.657852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:30:53.657852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:53.657852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:30:53.657852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:53.657852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:30:53.657852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:53.657852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:30:53.658422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:53.658422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:30:53.658422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:30:53.658422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:53.658422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:30:53.658422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:53.658422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:30:53.658422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:30:53.658422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:53.658422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:30:53.658422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:53.658422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:30:53.661393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:53.661393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:30:53.661393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:30:53.661393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:30:53.661393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:30:53.661393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:30:53.661393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:30:53.661393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:30:53.661393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:30:53.661393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:30:53.661393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:30:53.661393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:30:53.661393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:30:53.661393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:30:53.661393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:30:53.661393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:30:53.662031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:30:53.662031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:30:53.662031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:30:53.662031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:30:53.662031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:30:53.662031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:53.662031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:30:53.662031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:53.662031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:30:53.662031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:53.662031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:30:53.662031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:53.662031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:30:53.662031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:53.662031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:30:53.662031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:30:53.662031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:53.662031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:30:53.662645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:53.662645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:30:53.662645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:30:53.662645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:53.662645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:30:53.662645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:53.662645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:30:53.662645 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:30:53.662645 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:30:53.662645 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:30:53.664065 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:30:53.664065 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:30:53.664065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:53.664065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:30:53.664065 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:53.664065 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:30:53.664065 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:30:53.664065 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:53.664065 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:30:53.664065 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:53.664065 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:30:53.664065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:53.665030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:30:53.665030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:30:53.665030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:30:53.665030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:53.665030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:30:53.665030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:53.665030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:30:53.665030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:53.665030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:30:53.665030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:53.665030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:30:53.665030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:53.665030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:30:53.665030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:30:53.665030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:53.665030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:30:53.665030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:53.666320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:30:53.666320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:30:53.666320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:53.666320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:30:53.666320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:53.666320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:30:53.666320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:53.666320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:30:53.666320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:30:53.666320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:30:53.666320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:30:53.666320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:30:53.666320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:30:53.667544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:30:53.667544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:30:53.667544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:53.667544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:30:53.667544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:53.667544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:30:53.667544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:53.667544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:30:53.667544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:53.667544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:30:53.667544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:53.667544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:30:53.667544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:30:53.668668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:53.668668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:30:53.668668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:53.668668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:30:53.668668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:30:53.668668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:53.668668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:30:53.668668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:53.668668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:30:53.668668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:53.668668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:30:53.668668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:30:53.669145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:30:53.669145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:30:53.669145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:30:53.669145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:30:53.669145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:30:53.669145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:30:53.669145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:30:53.669145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:30:53.669145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:30:53.669145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:30:53.669145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:30:53.669145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:30:53.669145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:30:53.669145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:30:53.669145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:30:53.669145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:30:53.669145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:30:53.669145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:53.669145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:30:53.669584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:53.669584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:30:53.669584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:53.669584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:30:53.669584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:53.669584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:30:53.669584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:53.669584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:30:53.669584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:30:53.669584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:53.669584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:30:53.669584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:53.669584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:30:53.669584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:30:53.669584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:53.669584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:30:53.669584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:53.669584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:30:53.669942 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:30:53.669942 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:30:53.669942 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:30:53.669942 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:30:53.669942 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:30:53.669942 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:30:53.670085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:53.670085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:30:53.670085 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:53.670085 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:30:53.670085 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:30:53.670085 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:53.670085 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:30:53.670085 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:53.670085 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:30:53.670427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:53.670427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:30:53.670427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:30:53.670427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:30:53.670427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:30:53.670427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:30:53.670427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:30:53.670427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:53.670427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:30:53.670427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:53.670427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:30:53.670427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:53.670673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:30:53.670673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:53.670673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:30:53.670673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:53.670673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:30:53.670673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:30:53.670673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:53.670673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:30:53.670673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:53.670673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:30:53.670673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:30:53.670673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:53.670673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:30:53.670939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:53.670939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:30:53.671133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:53.671133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:30:53.671133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:30:53.671133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:30:53.671133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:30:53.671133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:30:53.671133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:30:53.671133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:30:53.671133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:30:53.671133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:30:53.671133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:30:53.671133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:30:53.671133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:30:53.671133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:30:53.671133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:30:53.671133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:30:53.671474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:30:53.671474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:30:53.671474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:53.671474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:30:53.671474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:53.671474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:30:53.671474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:53.671474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:30:53.671474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:53.671474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:30:53.671474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:53.671474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:30:53.671474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:30:53.671474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:53.671474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:30:53.671474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:53.671474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:30:53.671474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:30:53.671474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:53.671852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:30:53.671852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:53.671852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:30:58.489182 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:30:58.490376 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:30:58.490376 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:30:58.490773 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:30:58.493019 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG dbcounter [-] [106314] Writing DB stats neutron:SELECT=240 {{(pid=106314) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 09 09:30:58.526482 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:30:58.526736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:58.527773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:30:58.527773 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:58.527773 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:30:58.527773 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:30:58.527773 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:58.527773 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:30:58.527773 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:58.527773 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:30:58.528832 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:58.528832 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:30:58.528832 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:30:58.528832 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:30:58.528832 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:30:58.528832 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:30:58.528832 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:30:58.528832 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:58.528832 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:30:58.528832 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:58.528832 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:30:58.528832 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:58.529773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:30:58.529773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:58.529773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:30:58.529773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:58.529773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:30:58.529773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:30:58.529773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:58.529773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:30:58.529773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:58.529773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:30:58.529773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:30:58.529773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:58.529773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:30:58.530497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:58.530497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:30:58.530621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:58.530621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:30:58.530621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:30:58.530621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:30:58.530621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:30:58.530621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:30:58.530621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:30:58.530621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:30:58.530621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:30:58.530621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:30:58.530621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:30:58.530621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:30:58.530621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:30:58.530621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:30:58.530621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:30:58.530621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:30:58.531616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:30:58.531616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:30:58.531616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:58.531616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:30:58.531616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:58.531616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:30:58.531616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:58.531616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:30:58.531616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:58.531616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:30:58.531616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:58.531616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:30:58.531616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:30:58.531616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:58.531616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:30:58.531616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:58.531616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:30:58.531616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:30:58.531616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:58.532547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:30:58.532547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:58.532547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:30:58.643882 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:30:58.644477 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:30:58.644721 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:30:58.645977 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:30:58.646421 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:30:58.646688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:58.647654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:30:58.647654 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:58.647654 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:30:58.647654 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:30:58.647654 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:58.647654 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:30:58.647654 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:58.647654 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:30:58.649405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:58.649405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:30:58.649405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:30:58.649405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:30:58.649405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:30:58.649405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:30:58.649405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:30:58.649405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:58.649405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:30:58.649405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:58.649405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:30:58.649405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:58.650432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:30:58.650432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:58.650432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:30:58.650432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:58.650432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:30:58.650432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:30:58.650432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:58.650432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:30:58.650432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:58.650432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:30:58.650432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:30:58.650432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:58.650432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:30:58.651302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:58.651302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:30:58.651302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:58.651302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:30:58.651302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:30:58.651302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:30:58.651302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:30:58.651302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:30:58.651302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:30:58.651302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:30:58.651302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:30:58.651302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:30:58.651302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:30:58.651302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:30:58.651302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:30:58.652424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:30:58.652424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:30:58.652424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:30:58.652424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:30:58.652424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:30:58.652424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:58.652424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:30:58.652424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:58.652424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:30:58.652424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:58.652424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:30:58.652424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:58.652424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:30:58.652424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:58.652424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:30:58.652424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:30:58.652424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:58.652424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:30:58.652424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:58.653700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:30:58.653700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:30:58.653700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:58.653700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:30:58.653700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:58.653700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:30:58.653700 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:30:58.653700 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:30:58.653700 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:30:58.654267 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:30:58.654416 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:30:58.654721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:58.655459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:30:58.655459 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:58.655459 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:30:58.655459 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:30:58.655459 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:58.655459 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:30:58.655459 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:58.655459 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:30:58.656197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:58.656197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:30:58.656197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:30:58.656197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:30:58.656197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:58.656197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:30:58.656197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:58.656197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:30:58.656197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:58.656197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:30:58.656197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:58.656197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:30:58.656197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:58.656197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:30:58.656197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:30:58.656690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:58.656690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:30:58.656690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:58.656690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:30:58.656690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:30:58.656690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:58.656690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:30:58.656690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:58.656690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:30:58.657058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:58.657058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:30:58.657058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:30:58.657058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:30:58.657058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:30:58.657058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:30:58.657058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:30:58.657058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:30:58.657058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:30:58.657058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:30:58.657058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:58.657058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:30:58.657453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:58.657453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:30:58.657453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:58.657453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:30:58.657453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:58.657453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:30:58.657453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:58.657453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:30:58.657453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:30:58.657453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:58.657453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:30:58.657453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:58.657453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:30:58.657453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:30:58.657879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:58.657879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:30:58.657879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:58.657879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:30:58.658237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:58.658237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:30:58.658237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:30:58.658237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:30:58.658237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:30:58.658237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:30:58.658237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:30:58.658237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:30:58.658237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:30:58.658237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:30:58.658237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:30:58.658237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:30:58.658237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:30:58.658237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:30:58.658237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:30:58.658237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:30:58.658699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:30:58.658699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:30:58.658699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:30:58.658699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:30:58.658699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:30:58.658699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:58.658699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:30:58.658699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:58.658699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:30:58.658699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:58.658699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:30:58.658699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:58.658699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:30:58.658699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:58.658699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:30:58.658699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:30:58.658699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:58.658699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:30:58.659337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:58.659337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:30:58.659337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:30:58.659337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:58.659337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:30:58.659337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:58.659337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:30:58.661478 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:30:58.662363 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:30:58.662731 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:30:58.663132 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:30:58.663723 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:30:58.664199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:58.665239 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:30:58.665239 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:58.665239 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:30:58.665239 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:30:58.665239 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:58.665239 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:30:58.665239 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:58.665239 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:30:58.666237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:58.666237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:30:58.666237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:30:58.666237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:30:58.666237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:58.666237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:30:58.666237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:58.666237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:30:58.666237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:58.666237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:30:58.666237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:58.666237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:30:58.666237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:58.666237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:30:58.666237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:30:58.666593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:58.666593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:30:58.666593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:58.666593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:30:58.666593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:30:58.666593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:58.666593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:30:58.666593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:58.666593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:30:58.667179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:58.667179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:30:58.667179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:30:58.667179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:30:58.667179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:30:58.667179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:30:58.667179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:30:58.667179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:30:58.667179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:30:58.667179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:58.667179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:30:58.667426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:58.667426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:30:58.667426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:58.667426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:30:58.667426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:58.667426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:30:58.667426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:58.667426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:30:58.667426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:30:58.667426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:58.667426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:30:58.667426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:58.667426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:30:58.667426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:30:58.667667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:58.667667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:30:58.667667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:58.667667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:30:58.668525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:58.668525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:30:58.668525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:30:58.668525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:30:58.668525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:30:58.668525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:30:58.668525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:30:58.668525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:30:58.668525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:30:58.668525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:30:58.668525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:30:58.668525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:30:58.668525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:30:58.668525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:30:58.668525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:30:58.668525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:30:58.668525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:30:58.668991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:30:58.668991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:30:58.668991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:30:58.668991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:58.668991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:30:58.668991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:58.668991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:30:58.668991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:58.668991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:30:58.668991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:58.668991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:30:58.668991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:58.668991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:30:58.668991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:30:58.668991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:58.668991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:30:58.668991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:58.668991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:30:58.668991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:30:58.669382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:58.669382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:30:58.669382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:58.669382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:30:58.669875 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:30:58.670294 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:30:58.670417 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:30:58.670595 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:30:58.670906 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:30:58.671225 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:30:58.671367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:58.671822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:30:58.671822 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:58.671822 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:30:58.671822 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:30:58.671822 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:58.671822 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:30:58.671822 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:58.671822 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:30:58.672692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:58.672692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:30:58.672692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:30:58.672692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:30:58.672692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:30:58.672692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:30:58.672692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:30:58.672692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:58.672692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:30:58.672692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:58.672692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:30:58.672692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:58.673064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:30:58.673064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:58.673064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:30:58.673064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:58.673064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:30:58.673064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:30:58.673064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:58.673064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:30:58.673064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:58.673064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:30:58.673064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:30:58.673064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:58.673064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:30:58.673396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:58.673396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:30:58.674067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:58.674067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:30:58.674067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:30:58.674067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:30:58.674067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:30:58.674067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:30:58.674067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:30:58.674067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:30:58.674067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:30:58.674067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:30:58.674067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:30:58.674067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:30:58.674067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:30:58.674067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:30:58.674067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:30:58.674067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:30:58.674576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:30:58.674576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:30:58.674576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:58.674576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:30:58.674576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:30:58.674576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:30:58.674576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:30:58.674576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:30:58.674576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:30:58.674576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:30:58.674576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:30:58.674576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:30:58.674576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:30:58.674576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:30:58.674576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:30:58.674576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:30:58.674576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:30:58.674576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:30:58.674576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:30:58.675379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:30:58.675379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:30:58.675379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:31:03.489891 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:31:03.490780 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:31:03.490780 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:31:03.490987 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:31:03.521466 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:31:03.521928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:03.522862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:31:03.522862 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:03.522862 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:31:03.522862 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:31:03.522862 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:03.522862 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:31:03.522862 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:03.522862 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:31:03.524299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.033 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:03.524299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:31:03.524299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:31:03.524299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:31:03.524299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:31:03.524299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:31:03.524299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:31:03.524299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:03.524299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:31:03.524299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:03.524299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:31:03.524299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:03.524809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:31:03.524809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:03.524809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:31:03.524809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:03.524809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:31:03.524809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:31:03.524809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:03.524809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:31:03.524809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:03.524809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:31:03.524809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:31:03.524809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:03.524809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:31:03.525472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:03.525472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:31:03.526137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:03.526137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:31:03.526137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:31:03.526137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:31:03.526137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:31:03.526137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:31:03.526137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:31:03.526137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:31:03.526137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:31:03.526137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:31:03.526137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:31:03.526137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:31:03.526137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:31:03.526137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:31:03.526137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:31:03.526137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:31:03.526718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:31:03.526718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:31:03.526718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:03.526718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:31:03.526718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:03.526718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:31:03.526718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:03.526718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:31:03.526718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:03.526718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:31:03.526718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:03.526718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:31:03.526718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:31:03.526718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:03.526718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:31:03.526718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:03.526718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:31:03.526718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:31:03.526718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:03.527329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:31:03.527329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:03.527329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:31:03.644775 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:31:03.645406 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:31:03.645617 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:31:03.646887 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:31:03.647231 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:31:03.647489 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:03.648444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:31:03.648444 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:03.648444 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:31:03.648444 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:31:03.648444 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:03.648444 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:31:03.648444 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:03.648444 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:31:03.650162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:03.650162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:31:03.650162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:31:03.650162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:31:03.650162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:31:03.650162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:31:03.650162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:31:03.650162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:03.650162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:31:03.650162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:03.650162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:31:03.650162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:03.650648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:31:03.650648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:03.650648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:31:03.650648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:03.650648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:31:03.650648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:31:03.650648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:03.650648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:31:03.650648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:03.650648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:31:03.650648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:31:03.650648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:03.650648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:31:03.651089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:03.651089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:31:03.651973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:03.651973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:31:03.651973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:31:03.651973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:31:03.651973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:31:03.651973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:31:03.651973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:31:03.651973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:31:03.651973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:31:03.651973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:31:03.651973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:31:03.651973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:31:03.651973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:31:03.651973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:31:03.651973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:31:03.651973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:31:03.653092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:31:03.653092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:31:03.653092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:03.653092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:31:03.653092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:03.653092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:31:03.653092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:03.653092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:31:03.653092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:03.653092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:31:03.653092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:03.653092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:31:03.653092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:31:03.653092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:03.653092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:31:03.653092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:03.653092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:31:03.653092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:31:03.653092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:03.654651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:31:03.654651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:03.654651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:31:03.654651 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:31:03.654651 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:31:03.654651 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:31:03.655127 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:31:03.656299 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:31:03.656736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:03.657847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:31:03.657847 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:03.657847 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:31:03.657847 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:31:03.657847 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:03.657847 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:31:03.657847 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:03.657847 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:31:03.658660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:03.658660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:31:03.658660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:31:03.658660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:31:03.658660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:03.658660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:31:03.658660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:03.658660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:31:03.658660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:03.658660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:31:03.658660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:03.658660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:31:03.658660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:03.658660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:31:03.658660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:31:03.659454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:03.659454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:31:03.659454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:03.659454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:31:03.659454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:31:03.659454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:03.659454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:31:03.659454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:03.659454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:31:03.659817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:03.659817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:31:03.659817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:31:03.659817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:31:03.659817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:31:03.659817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:31:03.659817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:31:03.659817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:31:03.659817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:31:03.659817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:31:03.659817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:03.659817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:31:03.660268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:03.660268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:31:03.660268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:03.660268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:31:03.660268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:03.660268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:31:03.660268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:03.660268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:31:03.660268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:31:03.660268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:03.660268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:31:03.660268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:03.660268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:31:03.660268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:31:03.660713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:03.660713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:31:03.660713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:03.660713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:31:03.660713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:03.660713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:31:03.660713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:31:03.660713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:31:03.660713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:31:03.660713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:31:03.660713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:31:03.660713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:31:03.660713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:31:03.660713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:31:03.661226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:31:03.661226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:31:03.661226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:31:03.661226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:31:03.661226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:31:03.661226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:31:03.661226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:31:03.661226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:31:03.661226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:31:03.661226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:31:03.661226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:31:03.661226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:03.661226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:31:03.661226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:03.661226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:31:03.661226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:03.661226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:31:03.661226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:03.661226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:31:03.661958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:03.661958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:31:03.661958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:31:03.661958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:03.661958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:31:03.661958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:03.661958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:31:03.661958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:31:03.661958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:03.661958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:31:03.661958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:03.661958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:31:03.661958 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:31:03.663410 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:31:03.664213 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:31:03.664378 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:31:03.664932 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:31:03.665219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:03.665865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:31:03.665865 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:03.665865 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:31:03.665865 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:31:03.665865 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:03.665865 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:31:03.665865 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:03.665865 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:31:03.666563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:03.666563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:31:03.666563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:31:03.666563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:31:03.666563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:03.666563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:31:03.666563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:03.666563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:31:03.666563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:03.666563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:31:03.666563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:03.666563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:31:03.666563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:03.666563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:31:03.666563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:31:03.667028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:03.667028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:31:03.667028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:03.667028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:31:03.667028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:31:03.667028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:03.667028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:31:03.667028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:03.667028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:31:03.667385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:03.667385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:31:03.667385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:31:03.667385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:31:03.667385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:31:03.667385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:31:03.667385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:31:03.667385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:31:03.667385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:31:03.667385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:03.667385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:31:03.667732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:03.667732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:31:03.667732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:03.667732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:31:03.667732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:03.667732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:31:03.667732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:03.667732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:31:03.667732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:31:03.667732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:03.667732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:31:03.667732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:03.667732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:31:03.667732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:31:03.668152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:03.668152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:31:03.668152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:03.668152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:31:03.668152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:03.668152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:31:03.668152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:31:03.668152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:31:03.668152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:31:03.668152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:31:03.668152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:31:03.668152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:31:03.668152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:31:03.668152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:31:03.668561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:31:03.668561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:31:03.668561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:31:03.668561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:31:03.668561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:31:03.668561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:31:03.668561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:31:03.668561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:31:03.668561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:31:03.668561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:31:03.668561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:03.668561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:31:03.668561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:03.668561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:31:03.668561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:03.668561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:31:03.668561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:03.668561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:31:03.668561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:03.669123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:31:03.669123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:31:03.669123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:03.669123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:31:03.669123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:03.669123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:31:03.669123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:31:03.669123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:03.669123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:31:03.669123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:03.669123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:31:03.670451 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:31:03.670714 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:31:03.670847 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:31:03.670970 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:31:03.671436 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:31:03.671868 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:31:03.672126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:03.672921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:31:03.672921 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:03.672921 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:31:03.672921 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:31:03.672921 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:03.672921 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:31:03.672921 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:03.672921 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:31:03.673935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:03.673935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:31:03.673935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:31:03.673935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:31:03.673935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:31:03.673935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:31:03.673935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:31:03.673935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:03.673935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:31:03.673935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:03.673935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:31:03.673935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:03.674311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:31:03.674311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:03.674311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:31:03.674311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:03.674311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:31:03.674311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:31:03.674311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:03.674311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:31:03.674311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:03.674311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:31:03.674311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:31:03.674311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:03.674311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:31:03.674710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:03.674710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:31:03.674982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:03.674982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:31:03.674982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:31:03.674982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:31:03.674982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:31:03.674982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:31:03.674982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:31:03.674982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:31:03.674982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:31:03.674982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:31:03.674982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:31:03.674982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:31:03.674982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:31:03.674982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:31:03.674982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:31:03.674982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:31:03.675559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:31:03.675559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:31:03.675559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:03.675559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:31:03.675559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:03.675559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:31:03.675559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:03.675559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:31:03.675559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:03.675559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:31:03.675559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:03.675559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:31:03.675559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:31:03.675559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:03.675559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:31:03.675559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:03.675559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:31:03.675559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:31:03.675559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:03.676391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:31:03.676391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:03.676391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:31:08.490358 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:31:08.491425 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:31:08.491425 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:31:08.492564 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:31:08.526785 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:31:08.527011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:08.528553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:31:08.528553 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:08.528553 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:31:08.528553 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:31:08.528553 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:08.528553 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:31:08.528553 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:08.528553 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:31:08.528553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:08.528553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:31:08.528553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:31:08.528553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:31:08.528553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:31:08.528553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:31:08.528553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:31:08.529701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:08.529701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:31:08.529701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:08.529701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:31:08.529701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:08.529701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:31:08.529701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:08.529701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:31:08.529701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:08.529701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:31:08.529701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:31:08.529701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:08.529701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:31:08.529701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:08.530184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:31:08.530184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:31:08.530184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:08.530184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:31:08.530184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:08.530184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:31:08.530184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:08.530184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:31:08.530184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:31:08.530184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:31:08.530184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:31:08.530184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:31:08.530184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:31:08.530184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:31:08.530465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:31:08.530465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:31:08.530465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:31:08.530465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:31:08.530465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:31:08.530465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:31:08.530465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:31:08.530465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:31:08.530465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:31:08.530465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:31:08.530465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:08.530465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:31:08.530465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:08.530465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:31:08.530465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:08.530465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:31:08.530465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:08.530465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:31:08.530465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:08.530465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:31:08.530465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:31:08.530982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:08.530982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:31:08.530982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:08.530982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:31:08.530982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:31:08.530982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:08.530982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:31:08.530982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:08.530982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:31:08.645417 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:31:08.646182 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:31:08.646473 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:31:08.647897 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:31:08.648353 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:31:08.648674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:08.649626 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:31:08.649626 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:08.649626 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:31:08.649626 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:31:08.649626 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:08.649626 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:31:08.649626 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:08.649626 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:31:08.650848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:08.650848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:31:08.650848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:31:08.650848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:31:08.650848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:31:08.650848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:31:08.650848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:31:08.650848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:08.650848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:31:08.650848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:08.650848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:31:08.650848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:08.651198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:31:08.651198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:08.651198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:31:08.651198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:08.651198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:31:08.651198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:31:08.651198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:08.651198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:31:08.651198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:08.651198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:31:08.651198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:31:08.651198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:08.651198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:31:08.651622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:08.651622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:31:08.652238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:08.652238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:31:08.652238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:31:08.652238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:31:08.652238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:31:08.652238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:31:08.652238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:31:08.652238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:31:08.652238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:31:08.652238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:31:08.652238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:31:08.652238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:31:08.652238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:31:08.652238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:31:08.652238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:31:08.652238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:31:08.652728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:31:08.652728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:31:08.652728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:08.652728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:31:08.652728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:08.652728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:31:08.652728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:08.652728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:31:08.652728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:08.652728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:31:08.652728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:08.652728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:31:08.652728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:31:08.652728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:08.652728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:31:08.652728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:08.652728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:31:08.652728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:31:08.652728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:08.653257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:31:08.653257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:08.653257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:31:08.654820 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:31:08.654820 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:31:08.654917 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:31:08.655055 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:31:08.655455 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:31:08.655686 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:08.656438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:31:08.656438 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:08.656438 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:31:08.656438 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:31:08.656438 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:08.656438 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:31:08.656438 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:08.656438 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:31:08.657385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:08.657385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:31:08.657385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:31:08.657385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:31:08.657385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:08.657385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:31:08.657385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:08.657385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:31:08.657385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:08.657385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:31:08.657385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:08.657385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:31:08.657385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:08.657385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:31:08.657385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:31:08.658195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:08.658195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:31:08.658195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:08.658195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:31:08.658195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:31:08.658195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:08.658195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:31:08.658195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:08.658195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:31:08.658195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:08.658195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:31:08.658195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:31:08.658195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:31:08.658195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:31:08.658555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:31:08.658555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:31:08.658555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:31:08.658555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:31:08.658555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:31:08.658555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:08.658555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:31:08.658555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:08.658555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:31:08.658555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:08.658555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:31:08.658555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:08.658555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:31:08.658921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:08.658921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:31:08.658921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:31:08.658921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:08.658921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:31:08.658921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:08.658921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:31:08.658921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:31:08.658921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:08.658921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:31:08.658921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:08.658921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:31:08.659971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:08.659971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:31:08.659971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:31:08.659971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:31:08.659971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:31:08.659971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:31:08.659971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:31:08.659971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:31:08.659971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:31:08.659971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:31:08.659971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:31:08.659971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:31:08.659971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:31:08.659971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:31:08.659971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:31:08.659971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:31:08.661976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:31:08.661976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:31:08.661976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:31:08.661976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:31:08.661976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:31:08.661976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:08.661976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:31:08.661976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:08.661976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:31:08.661976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:08.661976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:31:08.661976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:08.661976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:31:08.661976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:08.661976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:31:08.661976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:31:08.661976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:08.661976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:31:08.662445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:08.662445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:31:08.662445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:31:08.662445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:08.662445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:31:08.662445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:08.662445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:31:08.662636 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:31:08.663041 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:31:08.663041 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:31:08.663672 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:31:08.664380 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:31:08.664823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:08.667284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:31:08.667284 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:08.667284 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:31:08.667284 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:31:08.667284 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:08.667284 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:31:08.667284 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:08.667284 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:31:08.668511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:08.668511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:31:08.668511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:31:08.668511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:31:08.668511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:08.668511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:31:08.668511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:08.668511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:31:08.668511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:08.668511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:31:08.668511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:08.668511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:31:08.668511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:08.668511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:31:08.668511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:31:08.668976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:08.668976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:31:08.668976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:08.668976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:31:08.668976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:31:08.668976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:08.668976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:31:08.668976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:08.668976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:31:08.670609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:08.670609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:31:08.670609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:31:08.670609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:31:08.670609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:31:08.670609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:31:08.670609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:31:08.670609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:31:08.670609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:31:08.670609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:08.670609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:31:08.670991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:08.670991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:31:08.670991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:08.670991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:31:08.670991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:08.670991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:31:08.670991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:08.670991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:31:08.670991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:31:08.670991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:08.670991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:31:08.670991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:08.670991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:31:08.670991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:31:08.671353 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:08.671353 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:31:08.671353 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:08.671353 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:31:08.671353 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:08.671353 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:31:08.671353 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:31:08.671353 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:31:08.671353 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:31:08.671353 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:31:08.671353 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:31:08.671353 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:31:08.671353 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:31:08.671353 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:31:08.671733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:31:08.671733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:31:08.671733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:31:08.671733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:31:08.671733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:31:08.671733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:31:08.671733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:31:08.671733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:31:08.671733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:31:08.671733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:31:08.671733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:08.671733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:31:08.671733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:08.671733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:31:08.671733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:08.671733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:31:08.671733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:08.671733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:31:08.671733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:08.672345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:31:08.672345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:31:08.672345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:08.672345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:31:08.672345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:08.672345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:31:08.672345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:31:08.672345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:08.672345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:31:08.672345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:08.672345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:31:08.672345 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:31:08.672345 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:31:08.672690 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:31:08.672690 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:31:08.672690 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:31:08.672690 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:31:08.672690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:08.672851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:31:08.672851 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:08.672851 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:31:08.672851 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:31:08.672851 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:08.672851 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:31:08.672851 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:08.672851 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:31:08.674913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:08.674913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:31:08.674913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:31:08.674913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:31:08.674913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:31:08.674913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:31:08.674913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:31:08.674913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:08.674913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:31:08.674913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:08.674913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:31:08.674913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:08.675768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:31:08.675768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:08.675768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:31:08.675768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:08.675768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:31:08.675768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:31:08.675768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:08.675768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:31:08.675768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:08.675768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:31:08.675768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:31:08.675768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:08.675768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:31:08.676224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:08.676224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:31:08.676224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:08.676224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:31:08.676224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:31:08.676224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:31:08.676224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:31:08.676224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:31:08.676224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:31:08.676224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:31:08.676224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:31:08.676224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:31:08.676224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:31:08.676224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:31:08.676224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:31:08.676807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:31:08.676807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:31:08.676807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:31:08.676807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:31:08.676807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:31:08.676807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:08.676807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:31:08.676807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:08.676807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:31:08.676807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:08.676807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:31:08.676807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:08.676807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:31:08.676807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:08.676807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:31:08.676807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:31:08.676807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:08.676807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:31:08.676807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:08.677431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:31:08.677431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:31:08.677431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:08.677431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:31:08.677431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:08.677431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:31:13.491518 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:31:13.492302 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:31:13.492302 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:31:13.493149 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:31:13.523272 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:31:13.523545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:13.524596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:31:13.524596 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:13.524596 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:31:13.524596 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:31:13.524596 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:13.524596 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:31:13.524596 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:13.524596 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:31:13.526346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.033 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:13.526346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:31:13.526346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:31:13.526346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:31:13.526346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:31:13.526346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:31:13.526346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:31:13.526346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:13.526346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:31:13.526346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:13.526346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:31:13.526346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:13.526952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:31:13.526952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:13.526952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:31:13.526952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:13.526952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:31:13.526952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:31:13.526952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:13.526952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:31:13.526952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:13.526952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:31:13.526952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:31:13.526952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:13.526952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:31:13.527436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:13.527436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:31:13.528190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:13.528190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:31:13.528190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:31:13.528190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:31:13.528190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:31:13.528190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:31:13.528190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:31:13.528190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:31:13.528190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:31:13.528190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:31:13.528190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:31:13.528190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:31:13.528190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:31:13.528190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:31:13.528190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:31:13.528190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:31:13.528883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:31:13.528883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:31:13.528883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:13.528883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:31:13.528883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:13.528883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:31:13.528883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:13.528883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:31:13.528883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:13.528883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:31:13.528883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:13.528883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:31:13.528883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:31:13.528883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:13.528883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:31:13.528883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:13.528883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:31:13.528883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:31:13.528883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:13.529626 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:31:13.529626 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:13.529626 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:31:13.646003 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:31:13.646589 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:31:13.646773 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:31:13.647905 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:31:13.648594 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:31:13.649663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:13.650628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:31:13.650628 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:13.650628 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:31:13.650628 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:31:13.650628 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:13.650628 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:31:13.650628 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:13.650628 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:31:13.652067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:13.652067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:31:13.652067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:31:13.652067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:31:13.652067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:31:13.652067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:31:13.652067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:31:13.652067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:13.652067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:31:13.652067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:13.652067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:31:13.652067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:13.652864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:31:13.652864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:13.652864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:31:13.652864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:13.652864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:31:13.652864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:31:13.652864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:13.652864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:31:13.652864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:13.652864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:31:13.652864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:31:13.652864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:13.652864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:31:13.653608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:13.653608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:31:13.653608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:13.653608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:31:13.653608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:31:13.653608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:31:13.653608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:31:13.653608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:31:13.653608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:31:13.653608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:31:13.653608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:31:13.653608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:31:13.653608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:31:13.653608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:31:13.653608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:31:13.654498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:31:13.654498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:31:13.654498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:31:13.654498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:31:13.654498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:31:13.654498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:13.654498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:31:13.654498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:13.654498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:31:13.654498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:13.654498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:31:13.654498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:13.654498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:31:13.654498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:13.654498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:31:13.654498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:31:13.654498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:13.654498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:31:13.654498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:13.655653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:31:13.655653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:31:13.655653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:13.655653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:31:13.655653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:13.655653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:31:13.655653 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:31:13.655653 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:31:13.655653 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:31:13.655653 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:31:13.657188 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:31:13.657188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:13.657188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:31:13.657188 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:13.657188 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:31:13.657188 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:31:13.657188 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:13.657188 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:31:13.657188 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:13.657188 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:31:13.657785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:13.657785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:31:13.657785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:31:13.657785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:31:13.657785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:13.657785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:31:13.657785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:13.657785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:31:13.657785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:13.657785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:31:13.657785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:13.657785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:31:13.657785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:13.657785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:31:13.657785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:31:13.658701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:13.658701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:31:13.658701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:13.658701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:31:13.658701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:31:13.658701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:13.658701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:31:13.658701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:13.658701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:31:13.658701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:13.658701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:31:13.658701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:31:13.658701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:31:13.658701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:31:13.659728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:31:13.659728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:31:13.659728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:31:13.659728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:31:13.659728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:31:13.659728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:13.659728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:31:13.659728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:13.659728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:31:13.659728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:13.659728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:31:13.659728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:13.659728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:31:13.660652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:13.660652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:31:13.660652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:31:13.660652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:13.660652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:31:13.660652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:13.660652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:31:13.660652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:31:13.660652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:13.660652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:31:13.660652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:13.660652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:31:13.661165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:13.661165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:31:13.661165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:31:13.661165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:31:13.661165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:31:13.661165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:31:13.661165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:31:13.661165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:31:13.661165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:31:13.661165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:31:13.661165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:31:13.661165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:31:13.661165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:31:13.661165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:31:13.661165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:31:13.661165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:31:13.661618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:31:13.661618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:31:13.661618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:31:13.661618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:31:13.661618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:31:13.661618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:13.661618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:31:13.661618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:13.661618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:31:13.661618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:13.661618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:31:13.661618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:13.661618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:31:13.661618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:13.661618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:31:13.661618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:31:13.661618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:13.661618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:31:13.662065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:13.662065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:31:13.662065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:31:13.662065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:13.662065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:31:13.662065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:13.662065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:31:13.662951 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:31:13.663285 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:31:13.663414 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:31:13.663559 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:31:13.664129 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:31:13.664601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:13.665034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:31:13.665034 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:13.665034 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:31:13.665034 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:31:13.665034 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:13.665034 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:31:13.665034 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:13.665034 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:31:13.666111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:13.666111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:31:13.666111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:31:13.666111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:31:13.666111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:13.666111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:31:13.666111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:13.666111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:31:13.666111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:13.666111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:31:13.666111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:13.666111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:31:13.666111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:13.666111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:31:13.666111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:31:13.666393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:13.666393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:31:13.666393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:13.666393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:31:13.666393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:31:13.666393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:13.666393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:31:13.666393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:13.666393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:31:13.667229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:13.667229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:31:13.667229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:31:13.667229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:31:13.667229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:31:13.667229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:31:13.667229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:31:13.667229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:31:13.667229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:31:13.667229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:13.667229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:31:13.667472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:13.667472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:31:13.667472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:13.667472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:31:13.667472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:13.667472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:31:13.667472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:13.667472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:31:13.667472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:31:13.667472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:13.667472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:31:13.667472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:13.667472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:31:13.667472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:31:13.667725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:13.667725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:31:13.667725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:13.667725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:31:13.668491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:13.668491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:31:13.668491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:31:13.668491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:31:13.668491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:31:13.668491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:31:13.668491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:31:13.668491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:31:13.668491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:31:13.668491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:31:13.668491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:31:13.668491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:31:13.668491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:31:13.668491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:31:13.668491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:31:13.668491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:31:13.668491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:31:13.668818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:31:13.668818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:31:13.668818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:31:13.668818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:13.668818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:31:13.668818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:13.668818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:31:13.668818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:13.668818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:31:13.668818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:13.668818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:31:13.668818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:13.668818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:31:13.668818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:31:13.668818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:13.668818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:31:13.668818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:13.668818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:31:13.668818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:31:13.669214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:13.669214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:31:13.669214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:13.669214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:31:13.671359 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:31:13.671784 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:31:13.671938 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:31:13.672216 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:31:13.673664 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:31:13.673927 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:31:13.674725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:13.675310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:31:13.675310 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:13.675310 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:31:13.675310 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:31:13.675310 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:13.675310 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:31:13.675310 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:13.675310 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:31:13.676348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:13.676348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:31:13.676348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:31:13.676348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:31:13.676348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:31:13.676348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:31:13.676348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:31:13.676348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:13.676348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:31:13.676348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:13.676348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:31:13.676348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:13.676773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:31:13.676773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:13.676773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:31:13.676773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:13.676773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:31:13.676773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:31:13.676773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:13.676773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:31:13.676773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:13.676773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:31:13.676773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:31:13.676773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:13.676773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:31:13.677212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:13.677212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:31:13.677429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:13.677429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:31:13.677429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:31:13.677429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:31:13.677429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:31:13.677429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:31:13.677429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:31:13.677429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:31:13.677429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:31:13.677429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:31:13.677429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:31:13.677429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:31:13.677429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:31:13.677429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:31:13.677429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:31:13.677429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:31:13.677951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:31:13.677951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:31:13.677951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:13.677951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:31:13.677951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:13.677951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:31:13.677951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:13.677951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:31:13.677951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:13.677951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:31:13.677951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:13.677951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:31:13.677951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:31:13.677951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:13.677951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:31:13.677951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:13.677951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:31:13.677951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:31:13.677951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:13.678629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:31:13.678629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:13.678629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:31:18.492047 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:31:18.493171 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:31:18.493315 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:31:18.493935 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:31:18.530069 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:31:18.530506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:18.533956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:31:18.533956 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:18.533956 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:31:18.533956 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:31:18.533956 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:18.533956 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:31:18.533956 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:18.533956 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:31:18.533956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:18.533956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:31:18.533956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:31:18.533956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:31:18.533956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:31:18.533956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:31:18.533956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:31:18.534519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:18.534519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:31:18.534519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:18.534519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:31:18.534519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:18.534519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:31:18.534519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:18.534519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:31:18.534519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:18.534519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:31:18.534519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:31:18.534519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:18.534519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:31:18.534519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:18.535015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:31:18.535015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:31:18.535015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:18.535015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:31:18.535015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:18.535015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:31:18.535015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:18.535015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:31:18.535015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:31:18.535015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:31:18.535015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:31:18.535015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:31:18.535015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:31:18.535015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:31:18.535493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:31:18.535493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:31:18.535493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:31:18.535493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:31:18.535493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:31:18.535493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:31:18.535493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:31:18.535493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:31:18.535493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:31:18.535493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:31:18.535493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:18.535493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:31:18.535493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:18.535493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:31:18.535493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:18.535493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:31:18.535493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:18.535493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:31:18.535493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:18.535493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:31:18.535493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:31:18.536261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:18.536261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:31:18.536261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:18.536261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:31:18.536261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:31:18.536261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:18.536261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:31:18.536261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:18.536261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:31:18.646924 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:31:18.647795 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:31:18.648293 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:31:18.649908 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:31:18.650243 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:31:18.650496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:18.651873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:31:18.651873 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:18.651873 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:31:18.651873 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:31:18.651873 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:18.651873 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:31:18.651873 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:18.651873 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:31:18.653512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:18.653512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:31:18.653512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:31:18.653512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:31:18.653512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:31:18.653512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:31:18.653512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:31:18.653512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:18.653512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:31:18.653512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:18.653512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:31:18.653512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:18.654175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:31:18.654175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:18.654175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:31:18.654175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:18.654175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:31:18.654175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:31:18.654175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:18.654175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:31:18.654175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:18.654175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:31:18.654175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:31:18.654175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:18.654175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:31:18.654702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:18.654702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:31:18.655563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:18.655563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:31:18.655563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:31:18.655563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:31:18.655563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:31:18.655563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:31:18.655563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:31:18.655563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:31:18.655563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:31:18.655563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:31:18.655563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:31:18.655563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:31:18.655563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:31:18.655563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:31:18.655563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:31:18.655563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:31:18.656663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:31:18.656663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:31:18.656663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:18.656663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:31:18.656663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:18.656663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:31:18.656663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:18.656663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:31:18.656663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:18.656663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:31:18.656663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:18.656663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:31:18.656663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:31:18.656663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:18.656663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:31:18.656663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:18.656663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:31:18.656663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:31:18.656663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:18.657829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:31:18.657829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:18.657829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:31:18.657829 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:31:18.657829 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:31:18.657829 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:31:18.657829 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:31:18.657829 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:31:18.658422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:18.658765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:31:18.658765 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:18.658765 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:31:18.658765 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:31:18.658765 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:18.658765 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:31:18.658765 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:18.658765 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:31:18.659567 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:18.659567 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:31:18.659567 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:31:18.659567 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:31:18.659567 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:18.659567 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:31:18.659567 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:18.659567 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:31:18.659567 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:18.659567 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:31:18.659567 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:18.659567 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:31:18.659567 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:18.659567 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:31:18.659567 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:31:18.660491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:18.660491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:31:18.660491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:18.660491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:31:18.660491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:31:18.660491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:18.660491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:31:18.660491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:18.660491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:31:18.660491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:18.660491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:31:18.660491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:31:18.660491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:31:18.660491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:31:18.661317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:31:18.661317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:31:18.661317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:31:18.661317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:31:18.661317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:31:18.661317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:18.661317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:31:18.661317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:18.661317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:31:18.661317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:18.661317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:31:18.661317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:18.661317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:31:18.662218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:18.662218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:31:18.662218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:31:18.662218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:18.662218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:31:18.662218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:18.662218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:31:18.662218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:31:18.662218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:18.662218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:31:18.662218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:18.662218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:31:18.663134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:18.663134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:31:18.663134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:31:18.663134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:31:18.663134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:31:18.663134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:31:18.663134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:31:18.663134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:31:18.663134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:31:18.663134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:31:18.663134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:31:18.663134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:31:18.663134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:31:18.663134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:31:18.663134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:31:18.663134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:31:18.664039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:31:18.664039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:31:18.664039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:31:18.664039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:31:18.664039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:31:18.664039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:18.664039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:31:18.664039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:18.664039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:31:18.664039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:18.664039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:31:18.664039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:18.664039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:31:18.664039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:18.664039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:31:18.664039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:31:18.664039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:18.664039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:31:18.664698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:18.664698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:31:18.664698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:31:18.664698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:18.664698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:31:18.664698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:18.664698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:31:18.664698 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:31:18.664698 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:31:18.664698 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:31:18.665134 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:31:18.666051 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:31:18.666358 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:18.666856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:31:18.666856 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:18.666856 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:31:18.666856 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:31:18.666856 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:18.666856 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:31:18.666856 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:18.666856 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:31:18.667572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:18.667572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:31:18.667572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:31:18.667572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:31:18.667572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:18.667572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:31:18.667572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:18.667572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:31:18.667572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:18.667572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:31:18.667572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:18.667572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:31:18.667572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:18.667572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:31:18.667572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:31:18.667928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:18.667928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:31:18.667928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:18.667928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:31:18.667928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:31:18.667928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:18.667928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:31:18.667928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:18.667928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:31:18.668440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:18.668440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:31:18.668440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:31:18.668440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:31:18.668440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:31:18.668440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:31:18.668440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:31:18.668440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:31:18.668440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:31:18.668440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:18.668440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:31:18.668672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:18.668672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:31:18.668672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:18.668672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:31:18.668672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:18.668672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:31:18.668672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:18.668672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:31:18.668672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:31:18.668672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:18.668672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:31:18.668672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:18.668672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:31:18.668672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:31:18.669037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:18.669037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:31:18.669037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:18.669037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:31:18.669564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:18.669564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:31:18.669564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:31:18.669564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:31:18.669564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:31:18.669564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:31:18.669564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:31:18.669564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:31:18.669564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:31:18.669564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:31:18.669564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:31:18.669564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:31:18.669564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:31:18.669564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:31:18.669564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:31:18.669564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:31:18.669564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:31:18.670066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:31:18.670066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:31:18.670066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:31:18.670066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:18.670066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:31:18.670066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:18.670066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:31:18.670066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:18.670066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:31:18.670066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:18.670066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:31:18.670066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:18.670066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:31:18.670066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:31:18.670066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:18.670066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:31:18.670066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:18.670066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:31:18.670066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:31:18.670458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:18.670458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:31:18.670458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:18.670458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:31:18.671926 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:31:18.672214 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:31:18.672329 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:31:18.672482 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:31:18.672814 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:31:18.673026 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:31:18.673187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:18.673644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:31:18.673644 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:18.673644 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:31:18.673644 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:31:18.673644 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:18.673644 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:31:18.673644 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:18.673644 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:31:18.674400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:18.674400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:31:18.674400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:31:18.674400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:31:18.674400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:31:18.674400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:31:18.674400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:31:18.674400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:18.674400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:31:18.674400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:18.674400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:31:18.674400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:18.674813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:31:18.674813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:18.674813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:31:18.674813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:18.674813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:31:18.674813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:31:18.674813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:18.674813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:31:18.674813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:18.674813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:31:18.674813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:31:18.674813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:18.674813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:31:18.675200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:18.675200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:31:18.675421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:18.675421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:31:18.675421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:31:18.675421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:31:18.675421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:31:18.675421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:31:18.675421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:31:18.675421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:31:18.675421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:31:18.675421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:31:18.675421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:31:18.675421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:31:18.675421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:31:18.675421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:31:18.675421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:31:18.675421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:31:18.675892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:31:18.675892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:31:18.675892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:18.675892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:31:18.675892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:18.675892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:31:18.675892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:18.675892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:31:18.675892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:18.675892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:31:18.675892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:18.675892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:31:18.675892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:31:18.675892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:18.675892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:31:18.675892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:18.675892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:31:18.675892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:31:18.675892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:18.676501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:31:18.676501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:18.676501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:31:23.493206 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:31:23.494117 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:31:23.494361 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:31:23.495911 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:31:23.534313 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:31:23.534669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:23.535339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:31:23.535339 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:23.535339 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:31:23.535339 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:31:23.535339 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:23.535339 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:31:23.535339 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:23.535339 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:31:23.536551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:23.536551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:31:23.536551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:31:23.536551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:31:23.536551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:31:23.536551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:31:23.536551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:31:23.536551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:23.536551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:31:23.536551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:23.536551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:31:23.536551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:23.537018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:31:23.537018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:23.537018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:31:23.537018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:23.537018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:31:23.537018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:31:23.537018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:23.537018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:31:23.537018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:23.537018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:31:23.537018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:31:23.537018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:23.537018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:31:23.537410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:23.537410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:31:23.537729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:23.537729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:31:23.537729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:31:23.537729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:31:23.537729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:31:23.537729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:31:23.537729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:31:23.537729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:31:23.537729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:31:23.537729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:31:23.537729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:31:23.537729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:31:23.537729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:31:23.537729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:31:23.537729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:31:23.537729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:31:23.539499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:31:23.539499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:31:23.539499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:23.539499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:31:23.539499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:23.539499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:31:23.539499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:23.539499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:31:23.539499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:23.539499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:31:23.539499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:23.539499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:31:23.539499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:31:23.539499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:23.539499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:31:23.539499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:23.539499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:31:23.539499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:31:23.539499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:23.540899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:31:23.540899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:23.540899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:31:23.648001 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:31:23.648563 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:31:23.648728 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:31:23.649666 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:31:23.649939 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:31:23.650095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:23.650782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:31:23.650782 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:23.650782 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:31:23.650782 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:31:23.650782 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:23.650782 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:31:23.650782 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:23.650782 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:31:23.651686 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:23.651686 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:31:23.651686 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:31:23.651686 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:31:23.651686 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:31:23.651686 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:31:23.651686 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:31:23.651686 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:23.651686 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:31:23.651686 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:23.651686 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:31:23.651686 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:23.652337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:31:23.652337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:23.652337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:31:23.652337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:23.652337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:31:23.652337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:31:23.652337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:23.652337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:31:23.652337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:23.652337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:31:23.652337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:31:23.652337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:23.652337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:31:23.653021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:23.653021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:31:23.653021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:23.653021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:31:23.653021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:31:23.653021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:31:23.653021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:31:23.653021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:31:23.653021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:31:23.653021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:31:23.653021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:31:23.653021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:31:23.653021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:31:23.653021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:31:23.653021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:31:23.653838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:31:23.653838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:31:23.653838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:31:23.653838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:31:23.653838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:31:23.653838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:23.653838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:31:23.653838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:23.653838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:31:23.653838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:23.653838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:31:23.653838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:23.653838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:31:23.653838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:23.653838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:31:23.653838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:31:23.653838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:23.653838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:31:23.653838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:23.654726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:31:23.654726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:31:23.654726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:23.654726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:31:23.654726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:23.654726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:31:23.656570 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:31:23.656909 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:31:23.657071 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:31:23.657197 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:31:23.657400 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:31:23.657601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:23.658038 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:31:23.658038 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:23.658038 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:31:23.658038 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:31:23.658038 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:23.658038 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:31:23.658038 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:23.658038 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:31:23.658726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:23.658726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:31:23.658726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:31:23.658726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:31:23.658726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:23.658726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:31:23.658726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:23.658726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:31:23.658726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:23.658726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:31:23.658726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:23.658726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:31:23.658726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:23.658726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:31:23.658726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:31:23.659194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:23.659194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:31:23.659194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:23.659194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:31:23.659194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:31:23.659194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:23.659194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:31:23.659194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:23.659194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:31:23.659623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:23.659623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:31:23.659623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:31:23.659623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:31:23.659623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:31:23.659623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:31:23.659623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:31:23.659623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:31:23.659623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:31:23.659623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:31:23.659623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:23.659623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:31:23.660625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:23.660625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:31:23.660625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:23.660625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:31:23.660625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:23.660625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:31:23.660625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:23.660625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:31:23.660625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:31:23.660625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:23.660625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:31:23.660625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:23.660625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:31:23.660625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:31:23.661128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:23.661128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:31:23.661128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:23.661128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:31:23.661128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:23.661128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:31:23.661128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:31:23.661128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:31:23.661128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:31:23.661128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:31:23.661128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:31:23.661128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:31:23.661128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:31:23.661128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:31:23.661518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:31:23.661518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:31:23.661518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:31:23.661518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:31:23.661518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:31:23.661518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:31:23.661518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:31:23.661518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:31:23.661518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:31:23.661518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:31:23.661518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:31:23.661518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:23.661518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:31:23.661518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:23.661518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:31:23.661518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:23.661518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:31:23.661518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:23.661518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:31:23.662046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:23.662046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:31:23.662046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:31:23.662046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:23.662046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:31:23.662046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:23.662046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:31:23.662046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:31:23.662046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:23.662046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:31:23.662046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:23.662046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:31:23.663820 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:31:23.664132 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:31:23.664250 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:31:23.664378 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:31:23.664620 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:31:23.664865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:23.665823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:31:23.665823 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:23.665823 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:31:23.665823 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:31:23.665823 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:23.665823 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:31:23.665823 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:23.665823 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:31:23.666531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:23.666531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:31:23.666531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:31:23.666531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:31:23.666531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:23.666531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:31:23.666531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:23.666531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:31:23.666531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:23.666531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:31:23.666531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:23.666531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:31:23.666531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:23.666531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:31:23.666531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:31:23.666942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:23.666942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:31:23.666942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:23.666942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:31:23.666942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:31:23.666942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:23.666942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:31:23.666942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:23.666942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:31:23.667437 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:23.667437 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:31:23.667437 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:31:23.667437 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:31:23.667437 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:31:23.667437 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:31:23.667437 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:31:23.667437 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:31:23.667437 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:31:23.667437 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:23.667437 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:31:23.667680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:23.667680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:31:23.667680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:23.667680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:31:23.667680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:23.667680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:31:23.667680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:23.667680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:31:23.667680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:31:23.667680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:23.667680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:31:23.667680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:23.667680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:31:23.667680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:31:23.667971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:23.667971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:31:23.667971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:23.667971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:31:23.668704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:23.668704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:31:23.668704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:31:23.668704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:31:23.668704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:31:23.668704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:31:23.668704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:31:23.668704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:31:23.668704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:31:23.668704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:31:23.668704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:31:23.668704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:31:23.668704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:31:23.668704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:31:23.668704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:31:23.668704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:31:23.668704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:31:23.669068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:31:23.669068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:31:23.669068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:31:23.669068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:23.669068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:31:23.669068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:23.669068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:31:23.669068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:23.669068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:31:23.669068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:23.669068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:31:23.669068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:23.669068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:31:23.669068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:31:23.669068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:23.669068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:31:23.669068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:23.669068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:31:23.669068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:31:23.669450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:23.669450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:31:23.669450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:23.669450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:31:23.672403 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:31:23.672802 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:31:23.672997 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:31:23.673195 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:31:23.673936 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:31:23.674298 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:31:23.674512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:23.675274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:31:23.675274 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:23.675274 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:31:23.675274 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:31:23.675274 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:23.675274 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:31:23.675274 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:23.675274 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:31:23.676151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:23.676151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:31:23.676151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:31:23.676151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:31:23.676151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:31:23.676151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:31:23.676151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:31:23.676151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:23.676151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:31:23.676151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:23.676151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:31:23.676151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:23.676615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:31:23.676615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:23.676615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:31:23.676615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:23.676615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:31:23.676615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:31:23.676615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:23.676615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:31:23.676615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:23.676615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:31:23.676615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:31:23.676615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:23.676615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:31:23.677010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:23.677010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:31:23.677181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:23.677181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:31:23.677181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:31:23.677181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:31:23.677181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:31:23.677181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:31:23.677181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:31:23.677181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:31:23.677181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:31:23.677181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:31:23.677181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:31:23.677181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:31:23.677181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:31:23.677181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:31:23.677181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:31:23.677181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:31:23.677671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:31:23.677671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:31:23.677671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:23.677671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:31:23.677671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:23.677671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:31:23.677671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:23.677671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:31:23.677671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:23.677671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:31:23.677671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:23.677671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:31:23.677671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:31:23.677671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:23.677671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:31:23.677671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:23.677671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:31:23.677671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:31:23.677671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:23.678311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:31:23.678311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:23.678311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:31:28.493965 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:31:28.494601 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:31:28.494661 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:31:28.495998 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:31:28.524338 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:31:28.524730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:28.525598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:31:28.525598 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:28.525598 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:31:28.525598 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:31:28.525598 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:28.525598 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:31:28.525598 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:28.525598 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:31:28.526511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.031 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:28.526511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:31:28.526511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:31:28.526511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:31:28.526511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:31:28.526511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:31:28.526511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:31:28.526511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:28.526511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:31:28.526511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:28.526511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:31:28.526511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:28.527521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:31:28.527521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:28.527521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:31:28.527521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:28.527521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:31:28.527521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:31:28.527521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:28.527521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:31:28.527521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:28.527521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:31:28.527521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:31:28.527521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:28.527521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:31:28.528213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:28.528213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:31:28.528213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:28.528213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:31:28.528213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:31:28.528213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:31:28.528213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:31:28.528213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:31:28.528213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:31:28.528213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:31:28.528213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:31:28.528213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:31:28.528213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:31:28.528213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:31:28.528213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:31:28.528873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:31:28.528873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:31:28.528873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:31:28.528873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:31:28.528873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:31:28.528873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:28.528873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:31:28.528873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:28.528873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:31:28.528873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:28.528873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:31:28.528873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:28.528873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:31:28.528873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:28.528873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:31:28.528873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:31:28.528873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:28.528873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:31:28.528873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:28.529627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:31:28.529627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:31:28.529627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:28.529627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:31:28.529627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:28.529627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:31:28.648652 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:31:28.649398 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:31:28.649676 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:31:28.650974 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:31:28.651389 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:31:28.651701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:28.652667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:31:28.652667 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:28.652667 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:31:28.652667 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:31:28.652667 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:28.652667 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:31:28.652667 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:28.652667 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:31:28.654344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:28.654344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:31:28.654344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:31:28.654344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:31:28.654344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:31:28.654344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:31:28.654344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:31:28.654344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:28.654344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:31:28.654344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:28.654344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:31:28.654344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:28.655269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:31:28.655269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:28.655269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:31:28.655269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:28.655269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:31:28.655269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:31:28.655269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:28.655269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:31:28.655269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:28.655269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:31:28.655269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:31:28.655269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:28.655269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:31:28.656082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:28.656082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:31:28.656230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:28.656230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:31:28.656230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:31:28.656230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:31:28.656230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:31:28.656230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:31:28.656230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:31:28.656230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:31:28.656230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:31:28.656230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:31:28.656230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:31:28.656230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:31:28.656230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:31:28.656230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:31:28.656230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:31:28.656230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:31:28.657321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:31:28.657321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:31:28.657321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:28.657321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:31:28.657321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:28.657321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:31:28.657321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:28.657321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:31:28.657321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:28.657321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:31:28.657321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:28.657321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:31:28.657321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:31:28.657321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:28.657321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:31:28.657321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:28.657321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:31:28.657321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:31:28.657321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:28.658112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:31:28.658112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:28.658112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:31:28.658112 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:31:28.658112 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:31:28.658112 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:31:28.658112 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:31:28.658112 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:31:28.658504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:28.659242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:31:28.659242 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:28.659242 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:31:28.659242 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:31:28.659242 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:28.659242 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:31:28.659242 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:28.659242 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:31:28.660015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:28.660015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:31:28.660015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:31:28.660015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:31:28.660015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:28.660015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:31:28.660015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:28.660015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:31:28.660015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:28.660015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:31:28.660015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:28.660015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:31:28.660015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:28.660015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:31:28.660015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:31:28.660776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:28.660776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:31:28.660776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:28.660776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:31:28.660776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:31:28.660776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:28.660776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:31:28.660776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:28.660776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:31:28.661187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:28.661187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:31:28.661187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:31:28.661187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:31:28.661187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:31:28.661187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:31:28.661187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:31:28.661187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:31:28.661187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:31:28.661187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:31:28.661187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:28.661187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:31:28.661721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:28.661721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:31:28.661721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:28.661721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:31:28.661721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:28.661721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:31:28.661721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:28.661721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:31:28.661721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:31:28.661721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:28.661721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:31:28.661721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:28.661721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:31:28.661721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:31:28.662292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:28.662292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:31:28.662292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:28.662292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:31:28.662292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:28.662292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:31:28.662292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:31:28.662292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:31:28.662292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:31:28.662292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:31:28.662292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:31:28.662292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:31:28.662292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:31:28.662292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:31:28.662891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:31:28.662891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:31:28.662891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:31:28.662891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:31:28.662891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:31:28.662891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:31:28.662891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:31:28.662891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:31:28.662891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:31:28.662891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:31:28.662891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:31:28.662891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:28.662891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:31:28.662891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:28.662891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:31:28.662891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:28.662891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:31:28.662891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:28.662891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:31:28.663679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:28.663679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:31:28.663679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:31:28.663679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:28.663679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:31:28.663679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:28.663679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:31:28.663679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:31:28.663679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:28.663679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:31:28.663679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:28.663679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:31:28.664267 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:31:28.664562 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:31:28.664702 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:31:28.664877 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:31:28.665417 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:31:28.665883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:28.666587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:31:28.666587 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:28.666587 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:31:28.666587 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:31:28.666587 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:28.666587 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:31:28.666587 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:28.666587 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:31:28.667372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:28.667372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:31:28.667372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:31:28.667372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:31:28.667372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:28.667372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:31:28.667372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:28.667372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:31:28.667372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:28.667372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:31:28.667372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:28.667372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:31:28.667372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:28.667372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:31:28.667372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:31:28.667941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:28.667941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:31:28.667941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:28.667941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:31:28.667941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:31:28.667941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:28.667941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:31:28.667941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:28.667941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:31:28.668362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:28.668362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:31:28.668362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:31:28.668362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:31:28.668362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:31:28.668362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:31:28.668362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:31:28.668362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:31:28.668362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:31:28.668362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:28.668362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:31:28.668925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:28.668925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:31:28.668925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:28.668925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:31:28.668925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:28.668925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:31:28.668925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:28.668925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:31:28.668925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:31:28.668925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:28.668925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:31:28.668925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:28.668925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:31:28.668925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:31:28.669487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:28.669487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:31:28.669487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:28.669487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:31:28.669647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:28.669647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:31:28.669647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:31:28.669647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:31:28.669647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:31:28.669647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:31:28.669647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:31:28.669647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:31:28.669647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:31:28.669647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:31:28.669647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:31:28.669647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:31:28.669647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:31:28.669647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:31:28.669647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:31:28.669647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:31:28.669647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:31:28.670344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:31:28.670344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:31:28.670344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:31:28.670344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:28.670344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:31:28.670344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:28.670344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:31:28.670344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:28.670344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:31:28.670344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:28.670344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:31:28.670344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:28.670344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:31:28.670344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:31:28.670344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:28.670344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:31:28.670344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:28.670344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:31:28.670344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:31:28.671106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:28.671106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:31:28.671106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:28.671106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:31:28.672984 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:31:28.673618 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:31:28.673987 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:31:28.674117 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:31:28.674586 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:31:28.674963 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:31:28.675178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:28.675912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:31:28.675912 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:28.675912 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:31:28.675912 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:31:28.675912 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:28.675912 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:31:28.675912 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:28.675912 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:31:28.676533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:28.676533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:31:28.676533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:31:28.676533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:31:28.676533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:31:28.676533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:31:28.676533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:31:28.676533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:28.676533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:31:28.676533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:28.676533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:31:28.676533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:28.677115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:31:28.677115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:28.677115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:31:28.677115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:28.677115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:31:28.677115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:31:28.677115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:28.677115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:31:28.677115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:28.677115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:31:28.677115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:31:28.677115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:28.677115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:31:28.677519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:28.677519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:31:28.677519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:28.677519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:31:28.677519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:31:28.677519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:31:28.677519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:31:28.677519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:31:28.677519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:31:28.677519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:31:28.677519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:31:28.677519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:31:28.677519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:31:28.677519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:31:28.677519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:31:28.678044 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:31:28.678044 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:31:28.678044 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:31:28.678044 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:31:28.678044 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:31:28.678044 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:28.678044 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:31:28.678044 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:28.678044 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:31:28.678044 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:28.678044 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:31:28.678044 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:28.678044 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:31:28.678044 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:28.678044 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:31:28.678044 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:31:28.678044 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:28.678044 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:31:28.678044 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:28.678693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:31:28.678693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:31:28.678693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:28.678693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:31:28.678693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:28.678693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:31:33.494579 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:31:33.495250 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:31:33.495250 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:31:33.496105 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:31:33.533258 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:31:33.533640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:33.534550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:31:33.534550 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:33.534550 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:31:33.534550 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:31:33.534550 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:33.534550 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:31:33.534550 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:33.534550 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:31:33.535378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:33.535378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:31:33.535378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:31:33.535378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:31:33.535378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:31:33.535378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:31:33.535378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:31:33.535378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:33.535378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:31:33.535378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:33.535378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:31:33.535378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:33.535865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:31:33.535865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:33.535865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:31:33.535865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:33.535865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:31:33.535865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:31:33.535865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:33.535865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:31:33.535865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:33.535865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:31:33.535865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:31:33.535865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:33.535865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:31:33.536331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:33.536331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:31:33.536331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:33.536331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:31:33.536331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:31:33.536331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:31:33.536331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:31:33.536331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:31:33.536331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:31:33.536331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:31:33.536331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:31:33.536331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:31:33.536331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:31:33.536331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:31:33.536331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:31:33.536899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:31:33.536899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:31:33.536899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:31:33.536899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:31:33.536899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:31:33.536899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:33.536899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:31:33.536899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:33.536899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:31:33.536899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:33.536899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:31:33.536899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:33.536899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:31:33.536899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:33.536899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:31:33.536899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:31:33.536899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:33.536899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:31:33.536899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:33.537976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:31:33.537976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:31:33.537976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:33.537976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:31:33.537976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:33.537976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:31:33.649491 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:31:33.650279 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:31:33.650363 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:31:33.651443 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:31:33.651692 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:31:33.651889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:33.652738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:31:33.652738 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:33.652738 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:31:33.652738 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:31:33.652738 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:33.652738 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:31:33.652738 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:33.652738 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:31:33.653928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:33.653928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:31:33.653928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:31:33.653928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:31:33.653928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:31:33.653928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:31:33.653928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:31:33.653928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:33.653928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:31:33.653928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:33.653928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:31:33.653928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:33.654299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:31:33.654299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:33.654299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:31:33.654299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:33.654299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:31:33.654299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:31:33.654299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:33.654299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:31:33.654299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:33.654299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:31:33.654299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:31:33.654299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:33.654299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:31:33.654618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:33.654618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:31:33.655495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:33.655495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:31:33.655495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:31:33.655495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:31:33.655495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:31:33.655495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:31:33.655495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:31:33.655495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:31:33.655495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:31:33.655495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:31:33.655495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:31:33.655495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:31:33.655495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:31:33.655495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:31:33.655495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:31:33.655495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:31:33.655980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:31:33.655980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:31:33.655980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:33.655980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:31:33.655980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:33.655980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:31:33.655980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:33.655980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:31:33.655980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:33.655980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:31:33.655980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:33.655980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:31:33.655980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:31:33.655980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:33.655980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:31:33.655980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:33.655980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:31:33.655980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:31:33.655980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:33.656473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:31:33.656473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:33.656473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:31:33.660185 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:31:33.660441 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:31:33.660441 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:31:33.660591 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:31:33.661054 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:31:33.661261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:33.661904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:31:33.661904 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:33.661904 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:31:33.661904 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:31:33.661904 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:33.661904 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:31:33.661904 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:33.661904 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:31:33.662845 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:33.662845 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:31:33.662845 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:31:33.662845 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:31:33.662845 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:33.662845 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:31:33.662845 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:33.662845 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:31:33.662845 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:33.662845 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:31:33.662845 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:33.662845 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:31:33.662845 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:33.662845 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:31:33.662845 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:31:33.663496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:33.663496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:31:33.663496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:33.663496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:31:33.663496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:31:33.663496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:33.663496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:31:33.663496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:33.663496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:31:33.663944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:33.663944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:31:33.663944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:31:33.663944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:31:33.663944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:31:33.663944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:31:33.663944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:31:33.663944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:31:33.663944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:31:33.663944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:31:33.663944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:33.663944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:31:33.664483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:33.664483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:31:33.664483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:33.664483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:31:33.664483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:33.664483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:31:33.664483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:33.664483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:31:33.664483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:31:33.664483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:33.664483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:31:33.664483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:33.664483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:31:33.664483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:31:33.664942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:33.664942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:31:33.664942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:33.664942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:31:33.666126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:33.666126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:31:33.666126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:31:33.666126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:31:33.666126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:31:33.666126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:31:33.666126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:31:33.666126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:31:33.666126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:31:33.666126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:31:33.666126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:31:33.666126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:31:33.666126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:31:33.666126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:31:33.666126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:31:33.666126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:31:33.666910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:31:33.666910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:31:33.666910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:31:33.666910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:31:33.666910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:31:33.666910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:33.666910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:31:33.666910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:33.666910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:31:33.666910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:33.666910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:31:33.666910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:33.666910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:31:33.666910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:33.666910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:31:33.666910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:31:33.666910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:33.666910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:31:33.667263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:33.667263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:31:33.667263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:31:33.667263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:33.667263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:31:33.667263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:33.667263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:31:33.667263 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:31:33.667762 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:31:33.668379 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:31:33.668626 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:31:33.669323 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:31:33.669728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:33.670566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:31:33.670566 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:33.670566 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:31:33.670566 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:31:33.670566 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:33.670566 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:31:33.670566 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:33.670566 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:31:33.671692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:33.671692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:31:33.671692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:31:33.671692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:31:33.671692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:33.671692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:31:33.671692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:33.671692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:31:33.671692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:33.671692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:31:33.671692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:33.671692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:31:33.671692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:33.671692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:31:33.671692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:31:33.672078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:33.672078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:31:33.672078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:33.672078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:31:33.672078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:31:33.672078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:33.672078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:31:33.672078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:33.672078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:31:33.673197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:33.673197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:31:33.673197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:31:33.673197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:31:33.673197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:31:33.673197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:31:33.673197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:31:33.673197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:31:33.673197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:31:33.673197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:33.673197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:31:33.673438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:33.673438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:31:33.673438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:33.673438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:31:33.673438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:33.673438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:31:33.673438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:33.673438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:31:33.673438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:31:33.673438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:33.673438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:31:33.673438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:33.673438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:31:33.673438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:31:33.673714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:33.673714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:31:33.673714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:33.673714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:31:33.674072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:33.674072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:31:33.674072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:31:33.674072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:31:33.674072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:31:33.674072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:31:33.674072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:31:33.674072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:31:33.674072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:31:33.674072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:31:33.674072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:31:33.674072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:31:33.674072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:31:33.674072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:31:33.674072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:31:33.674072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:31:33.674072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:31:33.674414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:31:33.674414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:31:33.674414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:31:33.674414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:33.674414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:31:33.674414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:33.674414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:31:33.674414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:33.674414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:31:33.674414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:33.674414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:31:33.674414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:33.674414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:31:33.674414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:31:33.674414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:33.674414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:31:33.674414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:33.674414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:31:33.674414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:31:33.674774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:33.674774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:31:33.674774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:33.674774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:31:33.675887 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:31:33.675887 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:31:33.675887 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:31:33.676007 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:31:33.676375 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:31:33.676722 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:31:33.676994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:33.677707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:31:33.677707 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:33.677707 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:31:33.677707 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:31:33.677707 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:33.677707 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:31:33.677707 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:33.677707 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:31:33.678352 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:33.678352 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:31:33.678352 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:31:33.678352 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:31:33.678352 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:31:33.678352 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:31:33.678352 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:31:33.678352 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:33.678352 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:31:33.678352 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:33.678352 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:31:33.678352 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:33.678689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:31:33.678689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:33.678689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:31:33.678689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:33.678689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:31:33.678689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:31:33.678689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:33.678689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:31:33.678689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:33.678689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:31:33.678689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:31:33.678689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:33.678689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:31:33.678976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:33.678976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:31:33.679308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:33.679308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:31:33.679308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:31:33.679308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:31:33.679308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:31:33.679308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:31:33.679308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:31:33.679308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:31:33.679308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:31:33.679308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:31:33.679308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:31:33.679308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:31:33.679308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:31:33.679308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:31:33.679308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:31:33.679308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:31:33.679722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:31:33.679722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:31:33.679722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:33.679722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:31:33.679722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:33.679722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:31:33.679722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:33.679722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:31:33.679722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:33.679722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:31:33.679722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:33.679722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:31:33.679722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:31:33.679722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:33.679722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:31:33.679722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:33.679722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:31:33.679722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:31:33.679722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:33.680387 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:31:33.680387 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:33.680387 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:31:38.495347 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:31:38.496286 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:31:38.496286 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:31:38.496405 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:31:38.528355 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:31:38.528712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:38.529694 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:31:38.529694 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:38.529694 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:31:38.529694 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:31:38.529694 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:38.529694 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:31:38.529694 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:38.529694 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:31:38.531343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:38.531343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:31:38.531343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:31:38.531343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:31:38.531343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:31:38.531343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:31:38.531343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:31:38.531343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:38.531343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:31:38.531343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:38.531343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:31:38.531343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:38.531896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:31:38.531896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:38.531896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:31:38.531896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:38.531896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:31:38.531896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:31:38.531896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:38.531896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:31:38.531896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:38.531896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:31:38.531896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:31:38.531896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:38.531896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:31:38.532325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:38.532325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:31:38.533322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:38.533322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:31:38.533322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:31:38.533322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:31:38.533322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:31:38.533322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:31:38.533322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:31:38.533322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:31:38.533322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:31:38.533322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:31:38.533322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:31:38.533322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:31:38.533322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:31:38.533322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:31:38.533322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:31:38.533322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:31:38.533933 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:31:38.533933 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:31:38.533933 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:38.533933 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:31:38.533933 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:38.533933 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:31:38.533933 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:38.533933 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:31:38.533933 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:38.533933 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:31:38.533933 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:38.533933 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:31:38.533933 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:31:38.533933 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:38.533933 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:31:38.533933 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:38.533933 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:31:38.533933 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:31:38.533933 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:38.534596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:31:38.534596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:38.534596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:31:38.650507 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:31:38.650763 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:31:38.650892 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:31:38.651908 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:31:38.652260 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:31:38.652529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:38.653471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:31:38.653471 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:38.653471 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:31:38.653471 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:31:38.653471 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:38.653471 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:31:38.653471 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:38.653471 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:31:38.655074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:38.655074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:31:38.655074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:31:38.655074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:31:38.655074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:31:38.655074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:31:38.655074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:31:38.655074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:38.655074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:31:38.655074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:38.655074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:31:38.655074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:38.655467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:31:38.655467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:38.655467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:31:38.655467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:38.655467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:31:38.655467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:31:38.655467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:38.655467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:31:38.655467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:38.655467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:31:38.655467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:31:38.655467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:38.655467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:31:38.655821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:38.655821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:31:38.657230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:38.657230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:31:38.657230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:31:38.657230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:31:38.657230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:31:38.657230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:31:38.657230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:31:38.657230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:31:38.657230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:31:38.657230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:31:38.657230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:31:38.657230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:31:38.657230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:31:38.657230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:31:38.657230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:31:38.657230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:31:38.657964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:31:38.657964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:31:38.657964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:38.657964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:31:38.657964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:38.657964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:31:38.657964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:38.657964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:31:38.657964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:38.657964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:31:38.657964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:38.657964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:31:38.657964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:31:38.657964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:38.657964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:31:38.657964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:38.657964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:31:38.657964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:31:38.657964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:38.658511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:31:38.658511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:38.658511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:31:38.660562 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:31:38.660887 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:31:38.661059 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:31:38.661221 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:31:38.661654 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:31:38.661962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:38.662693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:31:38.662693 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:38.662693 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:31:38.662693 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:31:38.662693 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:38.662693 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:31:38.662693 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:38.662693 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:31:38.664074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:38.664074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:31:38.664074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:31:38.664074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:31:38.664074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:38.664074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:31:38.664074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:38.664074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:31:38.664074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:38.664074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:31:38.664074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:38.664074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:31:38.664074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:38.664074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:31:38.664074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:31:38.664568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:38.664568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:31:38.664568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:38.664568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:31:38.664568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:31:38.664568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:38.664568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:31:38.664568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:38.664568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:31:38.665808 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:38.665808 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:31:38.665808 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:31:38.665808 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:31:38.665808 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:31:38.665808 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:31:38.665808 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:31:38.665808 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:31:38.665808 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:31:38.665808 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:31:38.665808 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:38.665808 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:31:38.666209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:38.666209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:31:38.666209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:38.666209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:31:38.666209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:38.666209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:31:38.666209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:38.666209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:31:38.666209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:31:38.666209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:38.666209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:31:38.666209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:38.666209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:31:38.666209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:31:38.666564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:38.666564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:31:38.666564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:38.666564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:31:38.667376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:38.667376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:31:38.667376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:31:38.667376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:31:38.667376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:31:38.667376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:31:38.667376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:31:38.667376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:31:38.667376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:31:38.667376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:31:38.667376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:31:38.667376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:31:38.667376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:31:38.667376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:31:38.667376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:31:38.667376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:31:38.667900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:31:38.667900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:31:38.667900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:31:38.667900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:31:38.667900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:31:38.667900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:38.667900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:31:38.667900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:38.667900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:31:38.667900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:38.667900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:31:38.667900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:38.667900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:31:38.667900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:38.667900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:31:38.667900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:31:38.667900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:38.667900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:31:38.668407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:38.668407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:31:38.668407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:31:38.668407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:38.668407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:31:38.668407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:38.668407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:31:38.669452 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:31:38.669704 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:31:38.669858 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:31:38.670036 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:31:38.670545 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:31:38.670915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:38.671485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:31:38.671485 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:38.671485 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:31:38.671485 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:31:38.671485 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:38.671485 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:31:38.671485 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:38.671485 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:31:38.672329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:38.672329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:31:38.672329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:31:38.672329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:31:38.672329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:38.672329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:31:38.672329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:38.672329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:31:38.672329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:38.672329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:31:38.672329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:38.672329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:31:38.672329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:38.672329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:31:38.672329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:31:38.672774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:38.672774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:31:38.672774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:38.672774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:31:38.672774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:31:38.672774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:38.672774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:31:38.672774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:38.672774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:31:38.673410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:38.673410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:31:38.673410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:31:38.673410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:31:38.673410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:31:38.673410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:31:38.673410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:31:38.673410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:31:38.673410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:31:38.673410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:38.673410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:31:38.673820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:38.673820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:31:38.673820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:38.673820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:31:38.673820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:38.673820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:31:38.673820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:38.673820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:31:38.673820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:31:38.673820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:38.673820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:31:38.673820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:38.673820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:31:38.673820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:31:38.674263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:38.674263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:31:38.674263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:38.674263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:31:38.674891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:38.674891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:31:38.674891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:31:38.674891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:31:38.674891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:31:38.674891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:31:38.674891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:31:38.674891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:31:38.674891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:31:38.674891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:31:38.674891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:31:38.674891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:31:38.674891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:31:38.674891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:31:38.674891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:31:38.674891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:31:38.674891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:31:38.675539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:31:38.675539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:31:38.675539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:31:38.675539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:38.675539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:31:38.675539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:38.675539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:31:38.675539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:38.675539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:31:38.675539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:38.675539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:31:38.675539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:38.675539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:31:38.675539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:31:38.675539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:38.675539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:31:38.675539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:38.675539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:31:38.675539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:31:38.677497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:38.677497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:31:38.677497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:38.677497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:31:38.677497 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:31:38.677497 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:31:38.677497 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:31:38.677497 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:31:38.677769 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:31:38.677769 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:31:38.677769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:38.677769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:31:38.677769 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:38.677769 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:31:38.677769 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:31:38.677769 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:38.677769 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:31:38.677769 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:38.677769 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:31:38.678333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:38.678333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:31:38.678333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:31:38.678333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:31:38.678333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:31:38.678333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:31:38.678333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:31:38.678333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:38.678333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:31:38.678333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:38.678333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:31:38.678333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:38.678772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:31:38.678772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:38.678772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:31:38.678772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:38.678772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:31:38.678772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:31:38.678772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:38.678772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:31:38.678772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:38.678772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:31:38.678772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:31:38.678772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:38.678772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:31:38.679156 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:38.679156 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:31:38.679334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:38.679334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:31:38.679334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:31:38.679334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:31:38.679334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:31:38.679334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:31:38.679334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:31:38.679334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:31:38.679334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:31:38.679334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:31:38.679334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:31:38.679334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:31:38.679334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:31:38.679334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:31:38.679334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:31:38.679334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:31:38.679834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:31:38.679834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:31:38.679834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:38.679834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:31:38.679834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:38.679834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:31:38.679834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:38.679834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:31:38.679834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:38.679834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:31:38.679834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:38.679834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:31:38.679834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:31:38.679834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:38.679834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:31:38.679834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:38.679834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:31:38.679834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:31:38.679834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:38.681403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:31:38.681403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:38.681403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:31:43.496231 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:31:43.497176 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:31:43.497176 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:31:43.498388 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:31:43.531120 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:31:43.531307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:43.531609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:31:43.531609 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:43.531609 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:31:43.531609 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:31:43.531609 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:43.531609 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:31:43.531609 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:43.531609 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:31:43.532402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:43.532402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:31:43.532402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:31:43.532402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:31:43.532402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:31:43.532402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:31:43.532402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:31:43.532402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:43.532402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:31:43.532402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:43.532402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:31:43.532402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:43.533123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:31:43.533123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:43.533123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:31:43.533123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:43.533123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:31:43.533123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:31:43.533123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:43.533123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:31:43.533123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:43.533123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:31:43.533123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:31:43.533123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:43.533123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:31:43.533819 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:43.533819 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:31:43.533819 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:43.533819 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:31:43.533819 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:31:43.533819 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:31:43.533819 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:31:43.533819 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:31:43.533819 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:31:43.533819 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:31:43.533819 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:31:43.533819 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:31:43.533819 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:31:43.533819 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:31:43.533819 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:31:43.534581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:31:43.534581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:31:43.534581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:31:43.534581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:31:43.534581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:31:43.534581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:43.534581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:31:43.534581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:43.534581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:31:43.534581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:43.534581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:31:43.534581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:43.534581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:31:43.534581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:43.534581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:31:43.534581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:31:43.534581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:43.534581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:31:43.534581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:43.535550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:31:43.535550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:31:43.535550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:43.535550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:31:43.535550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:43.535550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:31:43.651176 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:31:43.651916 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:31:43.652203 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:31:43.653378 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:31:43.653624 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:31:43.653944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:43.654590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:31:43.654590 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:43.654590 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:31:43.654590 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:31:43.654590 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:43.654590 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:31:43.654590 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:43.654590 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:31:43.656166 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:43.656166 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:31:43.656166 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:31:43.656166 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:31:43.656166 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:31:43.656166 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:31:43.656166 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:31:43.656166 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:43.656166 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:31:43.656166 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:43.656166 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:31:43.656166 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:43.656913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:31:43.656913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:43.656913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:31:43.656913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:43.656913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:31:43.656913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:31:43.656913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:43.656913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:31:43.656913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:43.656913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:31:43.656913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:31:43.656913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:43.656913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:31:43.657500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:43.657500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:31:43.657500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:43.657500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:31:43.657500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:31:43.657500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:31:43.657500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:31:43.657500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:31:43.657500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:31:43.657500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:31:43.657500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:31:43.657500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:31:43.657500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:31:43.657500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:31:43.657500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:31:43.658048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:31:43.658048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:31:43.658048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:31:43.658048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:31:43.658048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:31:43.658048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:43.658048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:31:43.658048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:43.658048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:31:43.658048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:43.658048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:31:43.658048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:43.658048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:31:43.658048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:43.658048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:31:43.658048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:31:43.658048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:43.658048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:31:43.658048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:43.658564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:31:43.658564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:31:43.658564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:43.658564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:31:43.658564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:43.658564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:31:43.661005 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:31:43.661264 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:31:43.661378 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:31:43.661497 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:31:43.661917 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:31:43.662151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:43.662759 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:31:43.662759 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:43.662759 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:31:43.662759 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:31:43.662759 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:43.662759 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:31:43.662759 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:43.662759 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:31:43.663549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:43.663549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:31:43.663549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:31:43.663549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:31:43.663549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:43.663549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:31:43.663549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:43.663549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:31:43.663549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:43.663549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:31:43.663549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:43.663549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:31:43.663549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:43.663549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:31:43.663549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:31:43.664188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:43.664188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:31:43.664188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:43.664188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:31:43.664188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:31:43.664188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:43.664188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:31:43.664188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:43.664188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:31:43.664492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:43.664492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:31:43.664492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:31:43.664492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:31:43.664492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:31:43.664492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:31:43.664492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:31:43.664492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:31:43.664492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:31:43.664492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:31:43.664492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:43.664492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:31:43.664947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:43.664947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:31:43.664947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:43.664947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:31:43.664947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:43.664947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:31:43.664947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:43.664947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:31:43.664947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:31:43.664947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:43.664947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:31:43.664947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:43.664947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:31:43.664947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:31:43.665317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:43.665317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:31:43.665317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:43.665317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:31:43.665541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:43.665541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:31:43.665541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:31:43.665541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:31:43.665541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:31:43.665541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:31:43.665541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:31:43.665541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:31:43.665541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:31:43.665541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:31:43.665541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:31:43.665541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:31:43.665541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:31:43.665541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:31:43.665541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:31:43.665541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:31:43.665995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:31:43.665995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:31:43.665995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:31:43.665995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:31:43.665995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:31:43.665995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:43.665995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:31:43.665995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:43.665995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:31:43.665995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:43.665995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:31:43.665995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:43.665995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:31:43.665995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:43.665995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:31:43.665995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:31:43.665995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:43.665995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:31:43.666501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:43.666501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:31:43.666501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:31:43.666501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:43.666501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:31:43.666501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:43.666501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:31:43.669821 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:31:43.670085 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:31:43.670199 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:31:43.670325 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:31:43.670691 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:31:43.671013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:43.671546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:31:43.671546 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:43.671546 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:31:43.671546 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:31:43.671546 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:43.671546 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:31:43.671546 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:43.671546 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:31:43.672333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:43.672333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:31:43.672333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:31:43.672333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:31:43.672333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:43.672333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:31:43.672333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:43.672333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:31:43.672333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:43.672333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:31:43.672333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:43.672333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:31:43.672333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:43.672333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:31:43.672333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:31:43.672881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:43.672881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:31:43.672881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:43.672881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:31:43.672881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:31:43.672881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:43.672881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:31:43.672881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:43.672881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:31:43.673270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:43.673270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:31:43.673270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:31:43.673270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:31:43.673270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:31:43.673270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:31:43.673270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:31:43.673270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:31:43.673270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:31:43.673270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:43.673270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:31:43.673699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:43.673699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:31:43.673699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:43.673699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:31:43.673699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:43.673699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:31:43.673699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:43.673699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:31:43.673699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:31:43.673699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:43.673699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:31:43.673699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:43.673699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:31:43.673699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:31:43.674115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:43.674115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:31:43.674115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:43.674115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:31:43.674519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:43.674519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:31:43.674519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:31:43.674519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:31:43.674519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:31:43.674519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:31:43.674519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:31:43.674519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:31:43.674519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:31:43.674519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:31:43.674519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:31:43.674519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:31:43.674519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:31:43.674519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:31:43.674519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:31:43.674519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:31:43.674519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:31:43.675056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:31:43.675056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:31:43.675056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:31:43.675056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:43.675056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:31:43.675056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:43.675056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:31:43.675056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:43.675056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:31:43.675056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:43.675056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:31:43.675056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:43.675056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:31:43.675056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:31:43.675056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:43.675056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:31:43.675056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:43.675056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:31:43.675056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:31:43.675826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:43.675826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:31:43.675826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:43.675826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:31:43.676453 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:31:43.676726 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:31:43.676849 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:31:43.677069 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:31:43.677270 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:31:43.677429 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:31:43.677523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:43.677914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:31:43.677914 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:43.677914 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:31:43.677914 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:31:43.677914 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:43.677914 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:31:43.677914 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:43.677914 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:31:43.678860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:43.678860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:31:43.678860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:31:43.678860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:31:43.678860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:31:43.678860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:31:43.678860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:31:43.678860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:43.678860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:31:43.678860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:43.678860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:31:43.678860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:43.679234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:31:43.679234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:43.679234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:31:43.679234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:43.679234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:31:43.679234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:31:43.679234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:43.679234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:31:43.679234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:43.679234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:31:43.679234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:31:43.679234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:43.679234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:31:43.679589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:43.679589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:31:43.679942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:43.679942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:31:43.679942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:31:43.679942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:31:43.679942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:31:43.679942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:31:43.679942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:31:43.679942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:31:43.679942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:31:43.679942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:31:43.679942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:31:43.679942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:31:43.679942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:31:43.679942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:31:43.679942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:31:43.679942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:31:43.680394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:31:43.680394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:31:43.680394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:43.680394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:31:43.680394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:43.680394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:31:43.680394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:43.680394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:31:43.680394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:43.680394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:31:43.680394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:43.680394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:31:43.680394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:31:43.680394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:43.680394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:31:43.680394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:43.680394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:31:43.680394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:31:43.680394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:43.681035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:31:43.681035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:43.681035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:31:48.496915 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:31:48.497564 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:31:48.497564 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:31:48.498573 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:31:48.532372 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:31:48.532645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:48.533372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:31:48.533372 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:48.533372 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:31:48.533372 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:31:48.533372 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:48.533372 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:31:48.533372 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:48.533372 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:31:48.534173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:48.534173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:31:48.534173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:31:48.534173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:31:48.534173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:31:48.534173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:31:48.534173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:31:48.534173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:48.534173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:31:48.534173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:48.534173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:31:48.534173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:48.535068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:31:48.535068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:48.535068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:31:48.535068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:48.535068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:31:48.535068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:31:48.535068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:48.535068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:31:48.535068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:48.535068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:31:48.535068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:31:48.535068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:48.535068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:31:48.536007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:48.536007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:31:48.536007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:48.536007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:31:48.536007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:31:48.536007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:31:48.536007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:31:48.536007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:31:48.536007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:31:48.536007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:31:48.536007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:31:48.536007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:31:48.536007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:31:48.536007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:31:48.536007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:31:48.536853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:31:48.536853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:31:48.536853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:31:48.536853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:31:48.536853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:31:48.536853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:48.536853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:31:48.536853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:48.536853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:31:48.536853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:48.536853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:31:48.536853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:48.536853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:31:48.536853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:48.536853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:31:48.536853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:31:48.536853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:48.536853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:31:48.536853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:48.537854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:31:48.537854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:31:48.537854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:48.537854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:31:48.537854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:48.537854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:31:48.651992 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:31:48.652498 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:31:48.652625 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:31:48.653654 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:31:48.654213 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:31:48.654491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:48.655394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:31:48.655394 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:48.655394 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:31:48.655394 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:31:48.655394 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:48.655394 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:31:48.655394 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:48.655394 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:31:48.656569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:48.656569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:31:48.656569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:31:48.656569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:31:48.656569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:31:48.656569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:31:48.656569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:31:48.656569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:48.656569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:31:48.656569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:48.656569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:31:48.656569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:48.657200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:31:48.657200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:48.657200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:31:48.657200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:48.657200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:31:48.657200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:31:48.657200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:48.657200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:31:48.657200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:48.657200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:31:48.657200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:31:48.657200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:48.657200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:31:48.657708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:48.657708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:31:48.657708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:48.657708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:31:48.657708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:31:48.657708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:31:48.657708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:31:48.657708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:31:48.657708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:31:48.657708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:31:48.657708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:31:48.657708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:31:48.657708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:31:48.657708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:31:48.657708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:31:48.658296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:31:48.658296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:31:48.658296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:31:48.658296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:31:48.658296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:31:48.658296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:48.658296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:31:48.658296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:48.658296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:31:48.658296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:48.658296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:31:48.658296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:48.658296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:31:48.658296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:48.658296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:31:48.658296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:31:48.658296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:48.658296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:31:48.658296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:48.659005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:31:48.659005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:31:48.659005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:48.659005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:31:48.659005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:48.659005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:31:48.661390 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:31:48.661956 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:31:48.662084 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:31:48.662217 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:31:48.662695 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:31:48.662912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:48.663451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:31:48.663451 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:48.663451 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:31:48.663451 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:31:48.663451 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:48.663451 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:31:48.663451 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:48.663451 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:31:48.664303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:48.664303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:31:48.664303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:31:48.664303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:31:48.664303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:48.664303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:31:48.664303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:48.664303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:31:48.664303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:48.664303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:31:48.664303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:48.664303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:31:48.664303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:48.664303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:31:48.664303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:31:48.664929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:48.664929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:31:48.664929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:48.664929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:31:48.664929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:31:48.664929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:48.664929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:31:48.664929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:48.664929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:31:48.665276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:48.665276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:31:48.665276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:31:48.665276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:31:48.665276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:31:48.665276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:31:48.665276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:31:48.665276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:31:48.665276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:31:48.665276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:31:48.665276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:48.665276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:31:48.665697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:48.665697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:31:48.665697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:48.665697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:31:48.665697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:48.665697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:31:48.665697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:48.665697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:31:48.665697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:31:48.665697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:48.665697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:31:48.665697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:48.665697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:31:48.665697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:31:48.666120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:48.666120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:31:48.666120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:48.666120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:31:48.666312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:48.666312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:31:48.666312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:31:48.666312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:31:48.666312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:31:48.666312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:31:48.666312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:31:48.666312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:31:48.666312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:31:48.666312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:31:48.666312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:31:48.666312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:31:48.666312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:31:48.666312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:31:48.666312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:31:48.666312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:31:48.666702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:31:48.666702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:31:48.666702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:31:48.666702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:31:48.666702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:31:48.666702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:48.666702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:31:48.666702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:48.666702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:31:48.666702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:48.666702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:31:48.666702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:48.666702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:31:48.666702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:48.666702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:31:48.666702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:31:48.666702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:48.666702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:31:48.667183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:48.667183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:31:48.667183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:31:48.667183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:48.667183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:31:48.667183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:48.667183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:31:48.670231 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:31:48.670576 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:31:48.670706 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:31:48.670875 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:31:48.671356 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:31:48.671712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:48.672439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:31:48.672439 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:48.672439 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:31:48.672439 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:31:48.672439 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:48.672439 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:31:48.672439 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:48.672439 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:31:48.673430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:48.673430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:31:48.673430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:31:48.673430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:31:48.673430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:48.673430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:31:48.673430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:48.673430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:31:48.673430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:48.673430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:31:48.673430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:48.673430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:31:48.673430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:48.673430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:31:48.673430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:31:48.673790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:48.673790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:31:48.673790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:48.673790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:31:48.673790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:31:48.673790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:48.673790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:31:48.673790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:48.673790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:31:48.674380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:48.674380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:31:48.674380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:31:48.674380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:31:48.674380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:31:48.674380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:31:48.674380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:31:48.674380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:31:48.674380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:31:48.674380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:48.674380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:31:48.674684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:48.674684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:31:48.674684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:48.674684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:31:48.674684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:48.674684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:31:48.674684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:48.674684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:31:48.674684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:31:48.674684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:48.674684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:31:48.674684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:48.674684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:31:48.674684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:31:48.675044 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:48.675044 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:31:48.675044 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:48.675044 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:31:48.675657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:48.675657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:31:48.675657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:31:48.675657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:31:48.675657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:31:48.675657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:31:48.675657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:31:48.675657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:31:48.675657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:31:48.675657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:31:48.675657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:31:48.675657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:31:48.675657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:31:48.675657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:31:48.675657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:31:48.675657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:31:48.675657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:31:48.676242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:31:48.676242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:31:48.676242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:31:48.676242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:48.676242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:31:48.676242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:48.676242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:31:48.676242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:48.676242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:31:48.676242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:48.676242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:31:48.676242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:48.676242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:31:48.676242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:31:48.676242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:48.676242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:31:48.676242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:48.676242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:31:48.676242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:31:48.676888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:48.676888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:31:48.676888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:48.676888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:31:48.676888 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:31:48.677192 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:31:48.677362 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:31:48.677585 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:31:48.678017 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:31:48.678243 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:31:48.678342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:48.679066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:31:48.679066 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:48.679066 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:31:48.679066 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:31:48.679066 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:48.679066 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:31:48.679066 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:48.679066 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:31:48.680516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:48.680516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:31:48.680516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:31:48.680516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:31:48.680516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:31:48.680516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:31:48.680516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:31:48.680516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:48.680516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:31:48.680516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:48.680516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:31:48.680516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:48.681280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:31:48.681280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:48.681280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:31:48.681280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:48.681280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:31:48.681280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:31:48.681280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:48.681280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:31:48.681280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:48.681280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:31:48.681280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:31:48.681280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:48.681280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:31:48.682060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:48.682060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:31:48.682640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:48.682640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:31:48.682640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:31:48.682640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:31:48.682640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:31:48.682640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:31:48.682640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:31:48.682640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:31:48.682640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:31:48.682640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:31:48.682640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:31:48.682640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:31:48.682640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:31:48.682640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:31:48.682640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:31:48.682640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:31:48.683568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:31:48.683568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:31:48.683568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:48.683568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:31:48.683568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:48.683568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:31:48.683568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:48.683568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:31:48.683568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:48.683568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:31:48.683568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:48.683568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:31:48.683568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:31:48.683568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:48.683568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:31:48.683568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:48.683568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:31:48.683568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:31:48.683568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:48.684649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:31:48.684649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:48.684649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:31:53.497571 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:31:53.498654 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:31:53.498654 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:31:53.500265 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:31:53.537948 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:31:53.538357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:53.539380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:31:53.539380 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:53.539380 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:31:53.539380 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:31:53.539380 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:53.539380 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:31:53.539380 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:53.539380 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:31:53.540280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:53.540280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:31:53.540280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:31:53.540280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:31:53.540280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:31:53.540280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:31:53.540280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:31:53.540280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:53.540280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:31:53.540280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:53.540280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:31:53.540280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:53.540958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:31:53.540958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:53.540958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:31:53.540958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:53.540958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:31:53.540958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:31:53.540958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:53.540958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:31:53.540958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:53.540958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:31:53.540958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:31:53.540958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:53.540958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:31:53.541587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:53.541587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:31:53.541771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:53.541771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:31:53.541771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:31:53.541771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:31:53.541771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:31:53.541771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:31:53.541771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:31:53.541771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:31:53.541771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:31:53.541771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:31:53.541771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:31:53.541771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:31:53.541771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:31:53.541771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:31:53.541771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:31:53.541771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:31:53.542596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:31:53.542596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:31:53.542596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:53.542596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:31:53.542596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:53.542596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:31:53.542596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:53.542596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:31:53.542596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:53.542596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:31:53.542596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:53.542596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:31:53.542596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:31:53.542596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:53.542596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:31:53.542596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:53.542596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:31:53.542596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:31:53.542596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:53.543607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:31:53.543607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:53.543607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:31:53.652829 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:31:53.653165 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:31:53.653165 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:31:53.655867 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:31:53.655867 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:31:53.655867 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:53.656204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:31:53.656204 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:53.656204 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:31:53.656204 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:31:53.656204 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:53.656204 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:31:53.656204 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:53.656204 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:31:53.658262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:53.658262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:31:53.658262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:31:53.658262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:31:53.658262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:31:53.658262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:31:53.658262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:31:53.658262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:53.658262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:31:53.658262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:53.658262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:31:53.658262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:53.659188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:31:53.659188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:53.659188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:31:53.659188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:53.659188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:31:53.659188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:31:53.659188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:53.659188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:31:53.659188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:53.659188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:31:53.659188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:31:53.659188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:53.659188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:31:53.659867 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:53.659867 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:31:53.664316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:53.664316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:31:53.664316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:31:53.664316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:31:53.664316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:31:53.664316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:31:53.664316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:31:53.664316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:31:53.664316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:31:53.664316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:31:53.664316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:31:53.664316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:31:53.664316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:31:53.664316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:31:53.664316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:31:53.664316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:31:53.665243 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:31:53.665243 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:31:53.665243 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:53.665243 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:31:53.665243 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:53.665243 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:31:53.665243 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:53.665243 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:31:53.665243 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:53.665243 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:31:53.665243 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:53.665243 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:31:53.665243 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:31:53.665243 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:53.665243 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:31:53.665243 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:53.665243 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:31:53.665243 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:31:53.665243 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:53.666221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:31:53.666221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:53.666221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:31:53.667001 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:31:53.667169 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:31:53.667277 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:31:53.667430 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:31:53.668880 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:31:53.668880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:53.669483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:31:53.669483 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:53.669483 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:31:53.669483 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:31:53.669483 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:53.669483 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:31:53.669483 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:53.669483 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:31:53.671741 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:53.671741 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:31:53.671741 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:31:53.671741 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:31:53.671741 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:53.671741 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:31:53.671741 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:53.671741 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:31:53.671741 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:53.671741 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:31:53.671741 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:53.671741 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:31:53.671741 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:53.671741 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:31:53.671741 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:31:53.672323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:53.672323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:31:53.672323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:53.672323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:31:53.672323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:31:53.672323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:53.672323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:31:53.672323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:53.672323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:31:53.672323 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:31:53.673227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:53.673227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:31:53.673227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:31:53.673227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:31:53.673227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:31:53.673227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:31:53.673227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:31:53.673227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:31:53.673227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:31:53.673227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:31:53.673227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:53.673227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:31:53.673639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:53.673639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:31:53.673639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:53.673639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:31:53.673639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:53.673639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:31:53.673639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:53.673639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:31:53.673639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:31:53.673639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:53.673639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:31:53.673639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:53.673639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:31:53.673639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:31:53.674148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:53.674148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:31:53.674148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:53.674148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:31:53.674148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:53.674148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:31:53.674148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:31:53.674148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:31:53.674148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:31:53.674148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:31:53.674148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:31:53.674148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:31:53.674148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:31:53.674148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:31:53.674728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:31:53.674728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:31:53.674728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:31:53.674728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:31:53.674728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:31:53.674728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:31:53.674728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:31:53.674728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:31:53.674728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:31:53.674728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:31:53.674728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:31:53.674728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:53.674728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:31:53.674728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:53.674728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:31:53.674728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:53.674728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:31:53.674728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:53.674728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:31:53.675428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:53.675428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:31:53.675428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:31:53.675428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:53.675428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:31:53.675428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:53.675428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:31:53.675428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:31:53.675428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:53.675428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:31:53.675428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:53.675428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:31:53.675428 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:31:53.675428 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:31:53.676224 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:31:53.676224 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:31:53.676224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:53.676224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:31:53.676224 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:53.676224 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:31:53.676224 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:31:53.676224 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:53.676224 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:31:53.676224 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:53.676224 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:31:53.676224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:53.677117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:31:53.677117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:31:53.677117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:31:53.677117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:53.677117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:31:53.677117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:53.677117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:31:53.677117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:53.677117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:31:53.677117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:53.677117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:31:53.677117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:53.677117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:31:53.677117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:31:53.677117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:53.677117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:31:53.677117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:53.678256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:31:53.678256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:31:53.678256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:53.678256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:31:53.678256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:53.678256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:31:53.678256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:53.678256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:31:53.678256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:31:53.678256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:31:53.678256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:31:53.678256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:31:53.678256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:31:53.679810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:31:53.679810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:31:53.679810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:53.679810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:31:53.679810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:53.679810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:31:53.679810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:53.679810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:31:53.679810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:53.679810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:31:53.679810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:53.679810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:31:53.679810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:31:53.680242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:53.680242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:31:53.680242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:53.680242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:31:53.680242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:31:53.680242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:53.680242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:31:53.680242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:53.680242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:31:53.680242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:53.680242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:31:53.680242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:31:53.680561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:31:53.680561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:31:53.680561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:31:53.680561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:31:53.680561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:31:53.680561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:31:53.680561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:31:53.680561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:31:53.680561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:31:53.680561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:31:53.680561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:31:53.680561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:31:53.680561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:31:53.680561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:31:53.680561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:31:53.680561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:31:53.680561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:31:53.680561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:53.680561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:31:53.681074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:53.681074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:31:53.681074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:53.681074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:31:53.681074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:53.681074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:31:53.681074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:53.681074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:31:53.681074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:31:53.681074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:53.681074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:31:53.681074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:53.681074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:31:53.681074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:31:53.681074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:53.681074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:31:53.681074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:53.681074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:31:53.681448 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:31:53.681448 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:31:53.681448 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:31:53.681448 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:31:53.681448 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:31:53.681448 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:31:53.681602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:53.681602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:31:53.681602 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:53.681602 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:31:53.681602 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:31:53.681602 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:53.681602 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:31:53.681602 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:53.681602 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:31:53.681602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:53.681602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:31:53.681602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:31:53.681602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:31:53.681602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:31:53.681602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:31:53.681928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:31:53.681928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:53.681928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:31:53.681928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:53.681928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:31:53.681928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:53.681928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:31:53.681928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:53.681928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:31:53.681928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:53.681928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:31:53.681928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:31:53.681928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:53.681928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:31:53.682229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:53.682229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:31:53.682229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:31:53.682229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:53.682229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:31:53.682229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:53.682229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:31:53.682229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:53.682229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:31:53.682229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:31:53.682229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:31:53.682229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:31:53.682229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:31:53.682515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:31:53.682515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:31:53.682515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:31:53.682515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:31:53.682515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:31:53.682515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:31:53.682515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:31:53.682515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:31:53.682515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:31:53.682515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:31:53.682515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:31:53.682515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:31:53.682515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:53.682515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:31:53.682515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:53.682515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:31:53.682515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:53.682515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:31:53.682515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:53.682515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:31:53.683287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:53.683287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:31:53.683287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:31:53.683287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:53.683287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:31:53.683287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:53.683287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:31:53.683287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:31:53.683287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:53.683287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:31:53.683287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:53.683287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:31:58.498181 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:31:58.498764 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:31:58.498809 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:31:58.499705 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:31:58.504933 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG dbcounter [-] [106314] Writing DB stats neutron:SELECT=241 {{(pid=106314) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 09 09:31:58.535085 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:31:58.535678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:58.536594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:31:58.536594 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:58.536594 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:31:58.536594 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:31:58.536594 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:58.536594 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:31:58.536594 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:58.536594 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:31:58.537451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:58.537451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:31:58.537451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:31:58.537451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:31:58.537451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:31:58.537451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:31:58.537451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:31:58.537451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:58.537451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:31:58.537451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:58.537451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:31:58.537451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:58.538492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:31:58.538492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:58.538492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:31:58.538492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:58.538492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:31:58.538492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:31:58.538492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:58.538492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:31:58.538492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:58.538492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:31:58.538492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:31:58.538492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:58.538492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:31:58.539560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:58.539560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:31:58.539560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:58.539560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:31:58.539560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:31:58.539560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:31:58.539560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:31:58.539560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:31:58.539560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:31:58.539560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:31:58.539560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:31:58.539560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:31:58.539560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:31:58.539560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:31:58.539560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:31:58.540429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:31:58.540429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:31:58.540429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:31:58.540429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:31:58.540429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:31:58.540429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:58.540429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:31:58.540429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:58.540429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:31:58.540429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:58.540429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:31:58.540429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:58.540429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:31:58.540429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:58.540429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:31:58.540429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:31:58.540429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:58.540429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:31:58.540429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:58.541263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:31:58.541263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:31:58.541263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:58.541263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:31:58.541263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:58.541263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:31:58.653409 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:31:58.653905 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:31:58.653994 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:31:58.654953 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:31:58.655306 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:31:58.655556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:58.656386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:31:58.656386 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:58.656386 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:31:58.656386 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:31:58.656386 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:58.656386 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:31:58.656386 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:58.656386 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:31:58.657583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:58.657583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:31:58.657583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:31:58.657583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:31:58.657583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:31:58.657583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:31:58.657583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:31:58.657583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:58.657583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:31:58.657583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:58.657583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:31:58.657583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:58.658189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:31:58.658189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:58.658189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:31:58.658189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:58.658189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:31:58.658189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:31:58.658189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:58.658189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:31:58.658189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:58.658189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:31:58.658189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:31:58.658189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:58.658189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:31:58.658834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:58.658834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:31:58.658834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:58.658834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:31:58.658834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:31:58.658834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:31:58.658834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:31:58.658834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:31:58.658834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:31:58.658834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:31:58.658834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:31:58.658834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:31:58.658834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:31:58.658834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:31:58.658834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:31:58.659599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:31:58.659599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:31:58.659599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:31:58.659599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:31:58.659599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:31:58.659599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:58.659599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:31:58.659599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:58.659599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:31:58.659599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:58.659599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:31:58.659599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:58.659599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:31:58.659599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:58.659599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:31:58.659599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:31:58.659599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:58.659599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:31:58.659599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:58.660461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:31:58.660461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:31:58.660461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:58.660461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:31:58.660461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:58.660461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:31:58.667141 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:31:58.668354 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:31:58.668495 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:31:58.668661 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:31:58.669521 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:31:58.669909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:58.670813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:31:58.670813 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:58.670813 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:31:58.670813 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:31:58.670813 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:58.670813 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:31:58.670813 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:58.670813 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:31:58.671667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:58.671667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:31:58.671667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:31:58.671667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:31:58.671667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:58.671667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:31:58.671667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:58.671667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:31:58.671667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:58.671667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:31:58.671667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:58.671667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:31:58.671667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:58.671667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:31:58.671667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:31:58.673274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:58.673274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:31:58.673274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:58.673274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:31:58.673274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:31:58.673274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:58.673274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:31:58.673274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:58.673274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:31:58.673274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:58.673274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:31:58.673274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:31:58.673274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:31:58.673274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:31:58.674096 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:31:58.674096 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:31:58.674096 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:31:58.674096 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:31:58.674096 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:31:58.674096 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:58.674096 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:31:58.674096 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:58.674096 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:31:58.674096 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:58.674096 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:31:58.674096 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:58.674096 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:31:58.674601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:58.674601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:31:58.674601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:31:58.674601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:58.674601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:31:58.674601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:58.674601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:31:58.674601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:31:58.674601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:58.674601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:31:58.674601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:58.674601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:31:58.675102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:58.675102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:31:58.675102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:31:58.675102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:31:58.675102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:31:58.675102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:31:58.675102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:31:58.675102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:31:58.675102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:31:58.675102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:31:58.675102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:31:58.675102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:31:58.675102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:31:58.675102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:31:58.675102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:31:58.675102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:31:58.676702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:31:58.676702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:31:58.676702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:31:58.676702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:31:58.676702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:31:58.676702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:58.676702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:31:58.676702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:58.676702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:31:58.676702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:58.676702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:31:58.676702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:58.676702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:31:58.676702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:58.676702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:31:58.676702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:31:58.676702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:58.676702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:31:58.678603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:58.678603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:31:58.678603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:31:58.678603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:58.678603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:31:58.678603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:58.678603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:31:58.678603 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:31:58.678603 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:31:58.678603 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:31:58.679796 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:31:58.679796 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:31:58.679796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:58.679796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:31:58.679796 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:58.679796 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:31:58.679796 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:31:58.679796 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:58.679796 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:31:58.679796 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:58.679796 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:31:58.679796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:58.680270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:31:58.680270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:31:58.680270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:31:58.680270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:58.680270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:31:58.680270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:58.680270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:31:58.680270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:58.680270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:31:58.680270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:58.680270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:31:58.680270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:58.680270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:31:58.680270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:31:58.680270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:58.680270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:31:58.680270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:58.681536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:31:58.681536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:31:58.681536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:58.681536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:31:58.681536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:58.681536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:31:58.681536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:58.681536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:31:58.681536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:31:58.681536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:31:58.681536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:31:58.681536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:31:58.681536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:31:58.682085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:31:58.682085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:31:58.682085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:58.682085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:31:58.682085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:58.682085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:31:58.682085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:58.682085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:31:58.682085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:58.682085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:31:58.682085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:58.682085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:31:58.682085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:31:58.682567 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:58.682567 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:31:58.682567 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:58.682567 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:31:58.682567 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:31:58.682567 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:58.682567 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:31:58.682567 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:58.682567 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:31:58.682567 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:31:58.683013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:58.683013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:31:58.683013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:31:58.683013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:31:58.683013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:31:58.683013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:31:58.683013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:31:58.683013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:31:58.683013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:31:58.683013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:31:58.683013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:31:58.683013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:31:58.683013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:31:58.683013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:31:58.683013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:31:58.683013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:31:58.683013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:31:58.683312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:31:58.683312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:31:58.683312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:31:58.683312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:58.683312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:31:58.683312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:58.683312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:31:58.683312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:58.683312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:31:58.683312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:58.683312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:31:58.683312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:58.683312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:31:58.683312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:31:58.683312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:58.683312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:31:58.683312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:58.683312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:31:58.683312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:31:58.683626 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:58.683626 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:31:58.683626 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:58.683626 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:31:58.683626 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:31:58.683626 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:31:58.683626 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:31:58.683626 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:31:58.683805 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:31:58.683805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:58.683805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:31:58.683805 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:58.683805 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:31:58.683805 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:31:58.683805 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:58.683805 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:31:58.683805 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:58.683805 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:31:58.683805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:58.683805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:31:58.683805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:31:58.683805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:31:58.684052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:31:58.684052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:31:58.684052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:31:58.684052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:58.684052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:31:58.684052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:58.684052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:31:58.684052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:58.684052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:31:58.684052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:58.684052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:31:58.684052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:58.684052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:31:58.684052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:31:58.684323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:58.684323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:31:58.684323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:58.684323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:31:58.684323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:31:58.684323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:58.684323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:31:58.684323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:58.684323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:31:58.684483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:58.684483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:31:58.684483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:31:58.684483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:31:58.684483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:31:58.684483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:31:58.684483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:31:58.684483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:31:58.684483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:31:58.684483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:31:58.684483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:31:58.684483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:31:58.684483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:31:58.684483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:31:58.684483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:31:58.684483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:31:58.684879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:31:58.684879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:31:58.684879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:58.684879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:31:58.684879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:31:58.684879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:31:58.684879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:31:58.684879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:31:58.684879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:31:58.684879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:31:58.684879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:31:58.684879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:31:58.684879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:31:58.684879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:31:58.684879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:31:58.684879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:31:58.684879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:31:58.684879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:31:58.684879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:31:58.685199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:31:58.685199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:31:58.685199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:32:03.498930 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:32:03.499938 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:32:03.499938 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:32:03.500326 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:32:03.537417 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:32:03.537845 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:03.538863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:32:03.538863 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:03.538863 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:32:03.538863 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:32:03.538863 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:03.538863 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:32:03.538863 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:03.538863 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:32:03.539888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:03.539888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:32:03.539888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:32:03.539888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:32:03.539888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:32:03.539888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:32:03.539888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:32:03.539888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:03.539888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:32:03.539888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:03.539888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:32:03.539888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:03.540419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:32:03.540419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:03.540419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:32:03.540419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:03.540419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:32:03.540419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:32:03.540419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:03.540419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:32:03.540419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:03.540419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:32:03.540419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:32:03.540419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:03.540419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:32:03.540991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:03.540991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:32:03.540991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:03.540991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:32:03.540991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:32:03.540991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:32:03.540991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:32:03.540991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:32:03.540991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:32:03.540991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:32:03.540991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:32:03.540991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:32:03.540991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:32:03.540991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:32:03.540991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:32:03.541524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:32:03.541524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:32:03.541524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:32:03.541524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:32:03.541524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:32:03.541524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:03.541524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:32:03.541524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:03.541524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:32:03.541524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:03.541524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:32:03.541524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:03.541524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:32:03.541524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:03.541524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:32:03.541524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:32:03.541524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:03.541524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:32:03.541524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:03.542127 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:32:03.542127 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:32:03.542127 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:03.542127 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:32:03.542127 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:03.542127 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:32:03.653872 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:32:03.654784 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:32:03.655108 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:32:03.656274 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:32:03.656626 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:32:03.656895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:03.657666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:32:03.657666 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:03.657666 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:32:03.657666 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:32:03.657666 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:03.657666 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:32:03.657666 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:03.657666 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:32:03.659884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:03.659884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:32:03.659884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:32:03.659884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:32:03.659884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:32:03.659884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:32:03.659884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:32:03.659884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:03.659884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:32:03.659884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:03.659884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:32:03.659884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:03.660578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:32:03.660578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:03.660578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:32:03.660578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:03.660578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:32:03.660578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:32:03.660578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:03.660578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:32:03.660578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:03.660578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:32:03.660578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:32:03.660578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:03.660578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:32:03.661209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:03.661209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:32:03.662190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:03.662190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:32:03.662190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:32:03.662190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:32:03.662190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:32:03.662190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:32:03.662190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:32:03.662190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:32:03.662190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:32:03.662190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:32:03.662190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:32:03.662190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:32:03.662190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:32:03.662190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:32:03.662190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:32:03.662190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:32:03.662902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:32:03.662902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:32:03.662902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:03.662902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:32:03.662902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:03.662902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:32:03.662902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:03.662902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:32:03.662902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:03.662902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:32:03.662902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:03.662902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:32:03.662902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:32:03.662902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:03.662902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:32:03.662902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:03.662902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:32:03.662902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:32:03.662902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:03.663423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:32:03.663423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:03.663423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:32:03.668241 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:32:03.668520 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:32:03.668652 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:32:03.668919 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:32:03.669228 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:32:03.669377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:03.670003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:32:03.670003 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:03.670003 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:32:03.670003 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:32:03.670003 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:03.670003 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:32:03.670003 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:03.670003 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:32:03.670907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:03.670907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:32:03.670907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:32:03.670907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:32:03.670907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:03.670907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:32:03.670907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:03.670907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:32:03.670907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:03.670907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:32:03.670907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:03.670907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:32:03.670907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:03.670907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:32:03.670907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:32:03.671426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:03.671426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:32:03.671426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:03.671426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:32:03.671426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:32:03.671426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:03.671426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:32:03.671426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:03.671426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:32:03.674441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:03.674441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:32:03.674441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:32:03.674441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:32:03.674441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:32:03.674441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:32:03.674441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:32:03.674441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:32:03.674441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:32:03.674441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:32:03.674441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:03.674441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:32:03.675046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:03.675046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:32:03.675046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:03.675046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:32:03.675046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:03.675046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:32:03.675046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:03.675046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:32:03.675046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:32:03.675046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:03.675046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:32:03.675046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:03.675046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:32:03.675046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:32:03.675580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:03.675580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:32:03.675580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:03.675580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:32:03.675580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:03.675580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:32:03.675580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:32:03.675580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:32:03.675580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:32:03.675580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:32:03.675580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:32:03.675580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:32:03.675580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:32:03.675580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:32:03.676006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:32:03.676006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:32:03.676006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:32:03.676006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:32:03.676006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:32:03.676006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:32:03.676006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:32:03.676006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:32:03.676006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:32:03.676006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:32:03.676006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:32:03.676006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:03.676006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:32:03.676006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:03.676006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:32:03.676006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:03.676006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:32:03.676006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:03.676006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:32:03.676501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:03.676501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:32:03.676501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:32:03.676501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:03.676501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:32:03.676501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:03.676501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:32:03.676501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:32:03.676501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:03.676501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:32:03.676501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:03.676501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:32:03.676501 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:32:03.676501 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:32:03.676999 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:32:03.676999 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:32:03.676999 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:32:03.676999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:03.676999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:32:03.676999 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:03.676999 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:32:03.676999 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:32:03.676999 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:03.676999 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:32:03.676999 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:03.676999 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:32:03.677487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:03.677487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:32:03.677487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:32:03.677487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:32:03.677487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:03.677487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:32:03.677487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:03.677487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:32:03.677487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:03.677487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:32:03.677487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:03.677487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:32:03.677487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:03.677487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:32:03.677487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:32:03.677899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:03.677899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:32:03.677899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:03.677899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:32:03.677899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:32:03.677899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:03.677899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:32:03.677899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:03.677899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:32:03.678649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:03.678649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:32:03.678649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:32:03.678649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:32:03.678649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:32:03.678649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:32:03.678649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:32:03.678649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:32:03.678649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:32:03.678649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:03.678649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:32:03.678915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:03.678915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:32:03.678915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:03.678915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:32:03.678915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:03.678915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:32:03.678915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:03.678915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:32:03.678915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:32:03.678915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:03.678915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:32:03.678915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:03.678915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:32:03.678915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:32:03.679174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:03.679174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:32:03.679174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:03.679174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:32:03.679729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:03.679729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:32:03.679729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:32:03.679729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:32:03.679729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:32:03.679729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:32:03.679729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:32:03.679729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:32:03.679729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:32:03.679729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:32:03.679729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:32:03.679729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:32:03.679729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:32:03.679729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:32:03.679729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:32:03.679729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:32:03.679729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:32:03.680090 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:32:03.680090 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:32:03.680090 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:32:03.680090 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:03.680090 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:32:03.680090 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:03.680090 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:32:03.680090 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:03.680090 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:32:03.680090 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:03.680090 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:32:03.680090 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:03.680090 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:32:03.680090 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:32:03.680090 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:03.680090 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:32:03.680090 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:03.680090 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:32:03.680090 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:32:03.680482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:03.680482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:32:03.680482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:03.680482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:32:03.680567 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:32:03.680887 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:32:03.681038 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:32:03.681197 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:32:03.681534 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:32:03.681753 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:32:03.681942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:03.682470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:32:03.682470 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:03.682470 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:32:03.682470 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:32:03.682470 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:03.682470 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:32:03.682470 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:03.682470 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:32:03.683483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:03.683483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:32:03.683483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:32:03.683483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:32:03.683483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:32:03.683483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:32:03.683483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:32:03.683483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:03.683483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:32:03.683483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:03.683483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:32:03.683483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:03.684142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:32:03.684142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:03.684142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:32:03.684142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:03.684142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:32:03.684142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:32:03.684142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:03.684142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:32:03.684142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:03.684142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:32:03.684142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:32:03.684142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:03.684142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:32:03.684830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:03.684830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:32:03.684830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:03.684830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:32:03.684830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:32:03.684830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:32:03.684830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:32:03.684830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:32:03.684830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:32:03.684830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:32:03.684830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:32:03.684830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:32:03.684830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:32:03.684830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:32:03.684830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:32:03.685710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:32:03.685710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:32:03.685710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:32:03.685710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:32:03.685710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:32:03.685710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:03.685710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:32:03.685710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:03.685710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:32:03.685710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:03.685710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:32:03.685710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:03.685710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:32:03.685710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:03.685710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:32:03.685710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:32:03.685710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:03.685710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:32:03.685710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:03.686721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:32:03.686721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:32:03.686721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:03.686721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:32:03.686721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:03.686721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:32:08.499415 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:32:08.500143 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:32:08.500143 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:32:08.500816 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:32:08.536832 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:32:08.537042 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:08.537924 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:32:08.537924 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:08.537924 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:32:08.537924 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:32:08.537924 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:08.537924 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:32:08.537924 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:08.537924 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:32:08.538797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:08.538797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:32:08.538797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:32:08.538797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:32:08.538797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:32:08.538797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:32:08.538797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:32:08.538797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:08.538797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:32:08.538797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:08.538797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:32:08.538797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:08.539252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:32:08.539252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:08.539252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:32:08.539252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:08.539252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:32:08.539252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:32:08.539252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:08.539252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:32:08.539252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:08.539252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:32:08.539252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:32:08.539252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:08.539252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:32:08.539663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:08.539663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:32:08.539848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:08.539848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:32:08.539848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:32:08.539848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:32:08.539848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:32:08.539848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:32:08.539848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:32:08.539848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:32:08.539848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:32:08.539848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:32:08.539848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:32:08.539848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:32:08.539848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:32:08.539848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:32:08.539848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:32:08.539848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:32:08.541500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:32:08.541500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:32:08.541500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:08.541500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:32:08.541500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:08.541500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:32:08.541500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:08.541500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:32:08.541500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:08.541500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:32:08.541500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:08.541500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:32:08.541500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:32:08.541500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:08.541500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:32:08.541500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:08.541500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:32:08.541500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:32:08.541500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:08.542740 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:32:08.542740 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:08.542740 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:32:08.654606 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:32:08.654981 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:32:08.655254 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:32:08.656593 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:32:08.657010 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:32:08.657937 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:08.658225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:32:08.658225 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:08.658225 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:32:08.658225 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:32:08.658225 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:08.658225 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:32:08.658225 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:08.658225 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:32:08.660089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:08.660089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:32:08.660089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:32:08.660089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:32:08.660089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:32:08.660089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:32:08.660089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:32:08.660089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:08.660089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:32:08.660089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:08.660089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:32:08.660089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:08.660673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:32:08.660673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:08.660673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:32:08.660673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:08.660673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:32:08.660673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:32:08.660673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:08.660673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:32:08.660673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:08.660673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:32:08.660673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:32:08.660673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:08.660673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:32:08.661200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:08.661200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:32:08.661813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:08.661813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:32:08.661813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:32:08.661813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:32:08.661813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:32:08.661813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:32:08.661813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:32:08.661813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:32:08.661813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:32:08.661813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:32:08.661813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:32:08.661813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:32:08.661813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:32:08.661813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:32:08.661813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:32:08.661813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:32:08.663330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:32:08.663330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:32:08.663330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:08.663330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:32:08.663330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:08.663330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:32:08.663330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:08.663330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:32:08.663330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:08.663330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:32:08.663330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:08.663330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:32:08.663330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:32:08.663330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:08.663330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:32:08.663330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:08.663330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:32:08.663330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:32:08.663330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:08.664561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:32:08.664561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:08.664561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:32:08.668584 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:32:08.668818 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:32:08.668987 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:32:08.669149 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:32:08.669491 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:32:08.669665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:08.670224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:32:08.670224 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:08.670224 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:32:08.670224 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:32:08.670224 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:08.670224 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:32:08.670224 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:08.670224 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:32:08.671001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:08.671001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:32:08.671001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:32:08.671001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:32:08.671001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:08.671001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:32:08.671001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:08.671001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:32:08.671001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:08.671001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:32:08.671001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:08.671001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:32:08.671001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:08.671001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:32:08.671001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:32:08.671441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:08.671441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:32:08.671441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:08.671441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:32:08.671441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:32:08.671441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:08.671441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:32:08.671441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:08.671441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:32:08.671991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:08.671991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:32:08.671991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:32:08.671991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:32:08.671991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:32:08.671991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:32:08.671991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:32:08.671991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:32:08.671991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:32:08.671991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:32:08.671991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:08.671991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:32:08.672709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:08.672709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:32:08.672709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:08.672709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:32:08.672709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:08.672709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:32:08.672709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:08.672709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:32:08.672709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:32:08.672709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:08.672709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:32:08.672709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:08.672709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:32:08.672709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:32:08.673637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:08.673637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:32:08.673637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:08.673637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:32:08.673637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:08.673637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:32:08.673637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:32:08.673637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:32:08.673637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:32:08.673637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:32:08.673637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:32:08.673637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:32:08.673637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:32:08.673637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:32:08.674356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:32:08.674356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:32:08.674356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:32:08.674356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:32:08.674356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:32:08.674356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:32:08.674356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:32:08.674356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:32:08.674356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:32:08.674356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:32:08.674356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:32:08.674356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:08.674356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:32:08.674356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:08.674356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:32:08.674356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:08.674356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:32:08.674356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:08.674356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:32:08.674875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:08.674875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:32:08.674875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:32:08.674875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:08.674875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:32:08.674875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:08.674875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:32:08.674875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:32:08.674875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:08.674875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:32:08.674875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:08.674875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:32:08.675212 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:32:08.675889 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:32:08.675984 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:32:08.676246 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:32:08.676663 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:32:08.676880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:08.677482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:32:08.677482 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:08.677482 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:32:08.677482 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:32:08.677482 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:08.677482 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:32:08.677482 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:08.677482 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:32:08.678998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:08.678998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:32:08.678998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:32:08.678998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:32:08.678998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:08.678998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:32:08.678998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:08.678998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:32:08.678998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:08.678998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:32:08.678998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:08.678998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:32:08.678998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:08.678998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:32:08.678998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:32:08.679907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:08.679907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:32:08.679907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:08.679907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:32:08.679907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:32:08.679907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:08.679907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:32:08.679907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:08.679907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:32:08.680690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:08.680690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:32:08.680690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:32:08.680690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:32:08.680690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:32:08.680690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:32:08.680690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:32:08.680690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:32:08.680690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:32:08.680690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:08.680690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:32:08.681321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:08.681321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:32:08.681321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:08.681321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:32:08.681321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:08.681321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:32:08.681321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:08.681321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:32:08.681321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:32:08.681321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:08.681321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:32:08.681321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:08.681321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:32:08.681321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:32:08.681653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:08.681653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:32:08.681653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:08.681653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:32:08.683267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:08.683267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:32:08.683267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:32:08.683267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:32:08.683267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:32:08.683267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:32:08.683267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:32:08.683267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:32:08.683267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:32:08.683267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:32:08.683267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:32:08.683267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:32:08.683267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:32:08.683267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:32:08.683267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:32:08.683267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:32:08.683267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:32:08.683727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:32:08.683727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:32:08.683727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:32:08.683727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:08.683727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:32:08.683727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:08.683727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:32:08.683727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:08.683727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:32:08.683727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:08.683727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:32:08.683727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:08.683727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:32:08.683727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:32:08.683727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:08.683727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:32:08.683727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:08.683727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:32:08.683727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:32:08.684263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:08.684263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:32:08.684263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:08.684263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:32:08.684263 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:32:08.684263 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:32:08.684263 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:32:08.684263 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:32:08.685143 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:32:08.685143 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:32:08.685143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:08.685774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:32:08.685774 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:08.685774 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:32:08.685774 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:32:08.685774 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:08.685774 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:32:08.685774 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:08.685774 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:32:08.686627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:08.686627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:32:08.686627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:32:08.686627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:32:08.686627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:32:08.686627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:32:08.686627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:32:08.686627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:08.686627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:32:08.686627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:08.686627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:32:08.686627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:08.687544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:32:08.687544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:08.687544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:32:08.687544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:08.687544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:32:08.687544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:32:08.687544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:08.687544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:32:08.687544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:08.687544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:32:08.687544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:32:08.687544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:08.687544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:32:08.688244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:08.688244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:32:08.688244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:08.688244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:32:08.688244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:32:08.688244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:32:08.688244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:32:08.688244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:32:08.688244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:32:08.688244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:32:08.688244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:32:08.688244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:32:08.688244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:32:08.688244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:32:08.688244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:32:08.689153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:32:08.689153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:32:08.689153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:32:08.689153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:32:08.689153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:32:08.689153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:08.689153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:32:08.689153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:08.689153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:32:08.689153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:08.689153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:32:08.689153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:08.689153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:32:08.689153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:08.689153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:32:08.689153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:32:08.689153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:08.689153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:32:08.689153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:08.689675 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:32:08.689675 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:32:08.689675 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:08.689675 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:32:08.689675 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:08.689675 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:32:13.500275 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:32:13.501930 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:32:13.501930 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:32:13.502174 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:32:13.539434 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:32:13.539892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:13.540876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:32:13.540876 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:13.540876 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:32:13.540876 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:32:13.540876 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:13.540876 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:32:13.540876 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:13.540876 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:32:13.542185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:13.542185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:32:13.542185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:32:13.542185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:32:13.542185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:32:13.542185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:32:13.542185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:32:13.542185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:13.542185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:32:13.542185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:13.542185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:32:13.542185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:13.542619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:32:13.542619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:13.542619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:32:13.542619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:13.542619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:32:13.542619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:32:13.542619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:13.542619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:32:13.542619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:13.542619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:32:13.542619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:32:13.542619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:13.542619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:32:13.543068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:13.543068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:32:13.543343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:13.543343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:32:13.543343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:32:13.543343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:32:13.543343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:32:13.543343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:32:13.543343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:32:13.543343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:32:13.543343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:32:13.543343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:32:13.543343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:32:13.543343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:32:13.543343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:32:13.543343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:32:13.543343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:32:13.543343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:32:13.545041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:32:13.545041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:32:13.545041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:13.545041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:32:13.545041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:13.545041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:32:13.545041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:13.545041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:32:13.545041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:13.545041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:32:13.545041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:13.545041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:32:13.545041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:32:13.545041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:13.545041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:32:13.545041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:13.545041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:32:13.545041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:32:13.545041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:13.546644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:32:13.546644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:13.546644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:32:13.655364 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:32:13.656078 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:32:13.656206 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:32:13.658188 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:32:13.658665 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:32:13.659003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:13.660517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:32:13.660517 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:13.660517 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:32:13.660517 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:32:13.660517 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:13.660517 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:32:13.660517 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:13.660517 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:32:13.661564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:13.661564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:32:13.661564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:32:13.661564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:32:13.661564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:32:13.661564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:32:13.661564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:32:13.661564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:13.661564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:32:13.661564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:13.661564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:32:13.661564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:13.662117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:32:13.662117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:13.662117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:32:13.662117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:13.662117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:32:13.662117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:32:13.662117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:13.662117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:32:13.662117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:13.662117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:32:13.662117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:32:13.662117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:13.662117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:32:13.662625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:13.662625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:32:13.662625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:13.662625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:32:13.662625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:32:13.662625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:32:13.662625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:32:13.662625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:32:13.662625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:32:13.662625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:32:13.662625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:32:13.662625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:32:13.662625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:32:13.662625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:32:13.662625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:32:13.663277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:32:13.663277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:32:13.663277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:32:13.663277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:32:13.663277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:32:13.663277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:13.663277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:32:13.663277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:13.663277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:32:13.663277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:13.663277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:32:13.663277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:13.663277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:32:13.663277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:13.663277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:32:13.663277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:32:13.663277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:13.663277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:32:13.663277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:13.664081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:32:13.664081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:32:13.664081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:13.664081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:32:13.664081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:13.664081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:32:13.669116 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:32:13.669502 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:32:13.669605 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:32:13.669718 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:32:13.670334 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:32:13.670644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:13.671513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:32:13.671513 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:13.671513 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:32:13.671513 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:32:13.671513 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:13.671513 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:32:13.671513 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:13.671513 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:32:13.672347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:13.672347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:32:13.672347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:32:13.672347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:32:13.672347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:13.672347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:32:13.672347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:13.672347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:32:13.672347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:13.672347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:32:13.672347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:13.672347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:32:13.672347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:13.672347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:32:13.672347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:32:13.672966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:13.672966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:32:13.672966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:13.672966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:32:13.672966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:32:13.672966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:13.672966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:32:13.672966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:13.672966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:32:13.673274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:13.673274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:32:13.673274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:32:13.673274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:32:13.673274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:32:13.673274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:32:13.673274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:32:13.673274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:32:13.673274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:32:13.673274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:32:13.673274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:13.673274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:32:13.673696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:13.673696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:32:13.673696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:13.673696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:32:13.673696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:13.673696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:32:13.673696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:13.673696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:32:13.673696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:32:13.673696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:13.673696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:32:13.673696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:13.673696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:32:13.673696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:32:13.674106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:13.674106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:32:13.674106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:13.674106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:32:13.674235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:13.674235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:32:13.674235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:32:13.674235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:32:13.674235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:32:13.674235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:32:13.674235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:32:13.674235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:32:13.674235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:32:13.674235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:32:13.674235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:32:13.674235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:32:13.674235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:32:13.674235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:32:13.674235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:32:13.674235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:32:13.674687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:32:13.674687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:32:13.674687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:32:13.674687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:32:13.674687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:32:13.674687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:13.674687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:32:13.674687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:13.674687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:32:13.674687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:13.674687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:32:13.674687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:13.674687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:32:13.674687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:13.674687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:32:13.674687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:32:13.674687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:13.674687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:32:13.675223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:13.675223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:32:13.675223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:32:13.675223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:13.675223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:32:13.675223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:13.675223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:32:13.675648 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:32:13.675969 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:32:13.676077 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:32:13.676222 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:32:13.676754 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:32:13.677150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:13.678101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:32:13.678101 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:13.678101 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:32:13.678101 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:32:13.678101 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:13.678101 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:32:13.678101 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:13.678101 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:32:13.678918 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:13.678918 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:32:13.678918 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:32:13.678918 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:32:13.678918 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:13.678918 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:32:13.678918 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:13.678918 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:32:13.678918 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:13.678918 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:32:13.678918 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:13.678918 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:32:13.678918 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:13.678918 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:32:13.678918 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:32:13.679326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:13.679326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:32:13.679326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:13.679326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:32:13.679326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:32:13.679326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:13.679326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:32:13.679326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:13.679326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:32:13.679889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:13.679889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:32:13.679889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:32:13.679889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:32:13.679889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:32:13.679889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:32:13.679889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:32:13.679889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:32:13.679889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:32:13.679889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:13.679889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:32:13.680225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:13.680225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:32:13.680225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:13.680225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:32:13.680225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:13.680225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:32:13.680225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:13.680225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:32:13.680225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:32:13.680225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:13.680225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:32:13.680225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:13.680225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:32:13.680225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:32:13.680583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:13.680583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:32:13.680583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:13.680583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:32:13.680998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:13.680998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:32:13.680998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:32:13.680998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:32:13.680998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:32:13.680998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:32:13.680998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:32:13.680998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:32:13.680998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:32:13.680998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:32:13.680998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:32:13.680998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:32:13.680998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:32:13.680998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:32:13.680998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:32:13.680998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:32:13.680998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:32:13.681467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:32:13.681467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:32:13.681467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:32:13.681467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:13.681467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:32:13.681467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:13.681467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:32:13.681467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:13.681467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:32:13.681467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:13.681467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:32:13.681467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:13.681467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:32:13.681467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:32:13.681467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:13.681467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:32:13.681467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:13.681467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:32:13.681467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:32:13.681997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:13.681997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:32:13.681997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:13.681997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:32:13.684026 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:32:13.684312 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:32:13.684430 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:32:13.684595 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:32:13.684979 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:32:13.685207 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:32:13.685356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:13.686005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:32:13.686005 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:13.686005 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:32:13.686005 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:32:13.686005 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:13.686005 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:32:13.686005 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:13.686005 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:32:13.686955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:13.686955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:32:13.686955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:32:13.686955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:32:13.686955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:32:13.686955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:32:13.686955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:32:13.686955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:13.686955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:32:13.686955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:13.686955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:32:13.686955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:13.687336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:32:13.687336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:13.687336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:32:13.687336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:13.687336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:32:13.687336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:32:13.687336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:13.687336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:32:13.687336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:13.687336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:32:13.687336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:32:13.687336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:13.687336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:32:13.687726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:13.687726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:32:13.687949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:13.687949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:32:13.687949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:32:13.687949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:32:13.687949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:32:13.687949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:32:13.687949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:32:13.687949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:32:13.687949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:32:13.687949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:32:13.687949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:32:13.687949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:32:13.687949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:32:13.687949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:32:13.687949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:32:13.687949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:32:13.688410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:32:13.688410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:32:13.688410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:13.688410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:32:13.688410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:13.688410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:32:13.688410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:13.688410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:32:13.688410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:13.688410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:32:13.688410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:13.688410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:32:13.688410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:32:13.688410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:13.688410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:32:13.688410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:13.688410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:32:13.688410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:32:13.688410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:13.688977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:32:13.688977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:13.688977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:32:18.501094 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:32:18.502295 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:32:18.502393 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:32:18.503333 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:32:18.535268 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:32:18.535544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:18.536272 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:32:18.536272 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:18.536272 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:32:18.536272 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:32:18.536272 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:18.536272 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:32:18.536272 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:18.536272 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:32:18.537481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:18.537481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:32:18.537481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:32:18.537481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:32:18.537481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:32:18.537481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:32:18.537481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:32:18.537481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:18.537481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:32:18.537481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:18.537481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:32:18.537481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:18.538273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:32:18.538273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:18.538273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:32:18.538273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:18.538273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:32:18.538273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:32:18.538273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:18.538273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:32:18.538273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:18.538273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:32:18.538273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:32:18.538273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:18.538273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:32:18.538993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:18.538993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:32:18.538993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:18.538993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:32:18.538993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:32:18.538993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:32:18.538993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:32:18.538993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:32:18.538993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:32:18.538993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:32:18.538993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:32:18.538993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:32:18.538993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:32:18.538993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:32:18.538993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:32:18.539861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:32:18.539861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:32:18.539861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:32:18.539861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:32:18.539861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:32:18.539861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:18.539861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:32:18.539861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:18.539861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:32:18.539861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:18.539861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:32:18.539861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:18.539861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:32:18.539861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:18.539861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:32:18.539861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:32:18.539861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:18.539861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:32:18.539861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:18.540962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:32:18.540962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:32:18.540962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:18.540962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:32:18.540962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:18.540962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:32:18.656433 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:32:18.657048 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:32:18.657399 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:32:18.658760 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:32:18.659083 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:32:18.659269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:18.660102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:32:18.660102 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:18.660102 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:32:18.660102 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:32:18.660102 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:18.660102 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:32:18.660102 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:18.660102 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:32:18.661832 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:18.661832 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:32:18.661832 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:32:18.661832 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:32:18.661832 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:32:18.661832 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:32:18.661832 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:32:18.661832 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:18.661832 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:32:18.661832 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:18.661832 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:32:18.661832 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:18.662191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:32:18.662191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:18.662191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:32:18.662191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:18.662191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:32:18.662191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:32:18.662191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:18.662191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:32:18.662191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:18.662191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:32:18.662191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:32:18.662191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:18.662191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:32:18.662427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:18.662427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:32:18.663862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:18.663862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:32:18.663862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:32:18.663862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:32:18.663862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:32:18.663862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:32:18.663862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:32:18.663862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:32:18.663862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:32:18.663862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:32:18.663862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:32:18.663862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:32:18.663862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:32:18.663862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:32:18.663862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:32:18.663862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:32:18.664339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:32:18.664339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:32:18.664339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:18.664339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:32:18.664339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:18.664339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:32:18.664339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:18.664339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:32:18.664339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:18.664339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:32:18.664339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:18.664339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:32:18.664339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:32:18.664339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:18.664339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:32:18.664339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:18.664339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:32:18.664339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:32:18.664339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:18.664880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:32:18.664880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:18.664880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:32:18.670185 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:32:18.670481 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:32:18.670583 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:32:18.670771 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:32:18.671265 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:32:18.671522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:18.672216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:32:18.672216 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:18.672216 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:32:18.672216 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:32:18.672216 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:18.672216 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:32:18.672216 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:18.672216 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:32:18.673300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:18.673300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:32:18.673300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:32:18.673300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:32:18.673300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:18.673300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:32:18.673300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:18.673300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:32:18.673300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:18.673300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:32:18.673300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:18.673300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:32:18.673300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:18.673300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:32:18.673300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:32:18.674227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:18.674227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:32:18.674227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:18.674227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:32:18.674227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:32:18.674227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:18.674227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:32:18.674227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:18.674227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:32:18.674227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:18.674227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:32:18.674227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:32:18.674227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:32:18.674227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:32:18.674979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:32:18.674979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:32:18.674979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:32:18.674979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:32:18.674979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:32:18.674979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:18.674979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:32:18.674979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:18.674979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:32:18.674979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:18.674979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:32:18.674979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:18.674979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:32:18.675597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:18.675597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:32:18.675597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:32:18.675597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:18.675597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:32:18.675597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:18.675597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:32:18.675597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:32:18.675597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:18.675597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:32:18.675597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:18.675597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:32:18.676155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:18.676155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:32:18.676155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:32:18.676155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:32:18.676155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:32:18.676155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:32:18.676155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:32:18.676155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:32:18.676155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:32:18.676155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:32:18.676155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:32:18.676155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:32:18.676155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:32:18.676155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:32:18.676155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:32:18.676155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:32:18.677193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:32:18.677193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:32:18.677193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:32:18.677193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:32:18.677193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:32:18.677193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:18.677193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:32:18.677193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:18.677193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:32:18.677193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:18.677193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:32:18.677193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:18.677193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:32:18.677193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:18.677193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:32:18.677193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:32:18.677193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:18.677193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:32:18.678105 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:18.678105 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:32:18.678105 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:32:18.678105 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:18.678105 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:32:18.678105 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:18.678105 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:32:18.678105 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:32:18.678105 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:32:18.678105 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:32:18.678569 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:32:18.678569 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:32:18.678569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:18.678569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:32:18.678569 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:18.678569 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:32:18.678569 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:32:18.678569 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:18.678569 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:32:18.678569 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:18.678569 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:32:18.679258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:18.679258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:32:18.679258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:32:18.679258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:32:18.679258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:18.679258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:32:18.679258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:18.679258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:32:18.679258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:18.679258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:32:18.679258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:18.679258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:32:18.679258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:18.679258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:32:18.679258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:32:18.679889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:18.679889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:32:18.679889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:18.679889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:32:18.679889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:32:18.679889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:18.679889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:32:18.679889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:18.679889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:32:18.680136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:18.680136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:32:18.680136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:32:18.680136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:32:18.680136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:32:18.680136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:32:18.680136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:32:18.680136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:32:18.680136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:32:18.680136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:18.680136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:32:18.680453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:18.680453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:32:18.680453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:18.680453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:32:18.680453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:18.680453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:32:18.680453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:18.680453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:32:18.680453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:32:18.680453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:18.680453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:32:18.680453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:18.680453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:32:18.680453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:32:18.680810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:18.680810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:32:18.680810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:18.680810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:32:18.681268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:18.681268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:32:18.681268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:32:18.681268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:32:18.681268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:32:18.681268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:32:18.681268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:32:18.681268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:32:18.681268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:32:18.681268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:32:18.681268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:32:18.681268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:32:18.681268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:32:18.681268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:32:18.681268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:32:18.681268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:32:18.681268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:32:18.681792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:32:18.681792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:32:18.681792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:32:18.681792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:18.681792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:32:18.681792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:18.681792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:32:18.681792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:18.681792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:32:18.681792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:18.681792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:32:18.681792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:18.681792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:32:18.681792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:32:18.681792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:18.681792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:32:18.681792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:18.681792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:32:18.681792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:32:18.682282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:18.682282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:32:18.682282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:18.682282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:32:18.684354 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:32:18.684630 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:32:18.684768 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:32:18.684965 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:32:18.685446 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:32:18.685976 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:32:18.686257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:18.687080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:32:18.687080 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:18.687080 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:32:18.687080 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:32:18.687080 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:18.687080 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:32:18.687080 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:18.687080 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:32:18.687984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:18.687984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:32:18.687984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:32:18.687984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:32:18.687984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:32:18.687984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:32:18.687984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:32:18.687984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:18.687984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:32:18.687984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:18.687984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:32:18.687984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:18.688597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:32:18.688597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:18.688597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:32:18.688597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:18.688597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:32:18.688597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:32:18.688597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:18.688597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:32:18.688597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:18.688597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:32:18.688597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:32:18.688597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:18.688597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:32:18.689188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:18.689188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:32:18.689188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:18.689188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:32:18.689188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:32:18.689188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:32:18.689188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:32:18.689188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:32:18.689188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:32:18.689188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:32:18.689188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:32:18.689188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:32:18.689188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:32:18.689188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:32:18.689188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:32:18.690023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:32:18.690023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:32:18.690023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:32:18.690023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:32:18.690023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:32:18.690023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:18.690023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:32:18.690023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:18.690023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:32:18.690023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:18.690023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:32:18.690023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:18.690023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:32:18.690023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:18.690023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:32:18.690023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:32:18.690023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:18.690023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:32:18.690023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:18.690907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:32:18.690907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:32:18.690907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:18.690907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:32:18.690907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:18.690907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:32:23.502584 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:32:23.503497 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:32:23.503497 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:32:23.503616 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:32:23.536604 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:32:23.536799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:23.537248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:32:23.537248 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:23.537248 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:32:23.537248 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:32:23.537248 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:23.537248 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:32:23.537248 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:23.537248 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:32:23.537972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:23.537972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:32:23.537972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:32:23.537972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:32:23.537972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:32:23.537972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:32:23.537972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:32:23.537972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:23.537972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:32:23.537972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:23.537972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:32:23.537972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:23.538424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:32:23.538424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:23.538424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:32:23.538424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:23.538424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:32:23.538424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:32:23.538424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:23.538424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:32:23.538424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:23.538424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:32:23.538424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:32:23.538424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:23.538424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:32:23.538850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:23.538850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:32:23.538850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:23.538850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:32:23.538850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:32:23.538850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:32:23.538850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:32:23.538850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:32:23.538850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:32:23.538850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:32:23.538850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:32:23.538850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:32:23.538850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:32:23.538850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:32:23.538850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:32:23.539359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:32:23.539359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:32:23.539359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:32:23.539359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:32:23.539359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:32:23.539359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:23.539359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:32:23.539359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:23.539359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:32:23.539359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:23.539359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:32:23.539359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:23.539359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:32:23.539359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:23.539359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:32:23.539359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:32:23.539359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:23.539359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:32:23.539359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:23.539947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:32:23.539947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:32:23.539947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:23.539947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:32:23.539947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:23.539947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:32:23.657034 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:32:23.657837 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:32:23.657924 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:32:23.659008 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:32:23.659292 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:32:23.659463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:23.660341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:32:23.660341 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:23.660341 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:32:23.660341 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:32:23.660341 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:23.660341 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:32:23.660341 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:23.660341 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:32:23.661385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:23.661385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:32:23.661385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:32:23.661385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:32:23.661385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:32:23.661385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:32:23.661385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:32:23.661385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:23.661385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:32:23.661385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:23.661385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:32:23.661385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:23.661957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:32:23.661957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:23.661957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:32:23.661957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:23.661957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:32:23.661957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:32:23.661957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:23.661957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:32:23.661957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:23.661957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:32:23.661957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:32:23.661957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:23.661957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:32:23.662332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:23.662332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:32:23.662393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:23.662393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:32:23.662393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:32:23.662393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:32:23.662393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:32:23.662393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:32:23.662393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:32:23.662393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:32:23.662393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:32:23.662393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:32:23.662393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:32:23.662393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:32:23.662393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:32:23.662393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:32:23.662393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:32:23.662393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:32:23.662823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:32:23.662823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:32:23.662823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:23.662823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:32:23.662823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:23.662823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:32:23.662823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:23.662823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:32:23.662823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:23.662823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:32:23.662823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:23.662823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:32:23.662823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:32:23.662823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:23.662823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:32:23.662823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:23.662823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:32:23.662823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:32:23.662823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:23.663296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:32:23.663296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:23.663296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:32:23.670759 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:32:23.671332 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:32:23.671460 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:32:23.671626 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:32:23.672503 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:32:23.672984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:23.673923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:32:23.673923 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:23.673923 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:32:23.673923 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:32:23.673923 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:23.673923 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:32:23.673923 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:23.673923 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:32:23.675630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:23.675630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:32:23.675630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:32:23.675630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:32:23.675630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:23.675630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:32:23.675630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:23.675630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:32:23.675630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:23.675630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:32:23.675630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:23.675630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:32:23.675630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:23.675630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:32:23.675630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:32:23.676656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:23.676656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:32:23.676656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:23.676656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:32:23.676656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:32:23.676656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:23.676656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:32:23.676656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:23.676656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:32:23.677267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:23.677267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:32:23.677267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:32:23.677267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:32:23.677267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:32:23.677267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:32:23.677267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:32:23.677267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:32:23.677267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:32:23.677267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:32:23.677267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:23.677267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:32:23.677919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:23.677919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:32:23.677919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:23.677919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:32:23.677919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:23.677919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:32:23.677919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:23.677919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:32:23.677919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:32:23.677919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:23.677919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:32:23.677919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:23.677919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:32:23.677919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:32:23.678563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:23.678563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:32:23.678563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:23.678563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:32:23.678563 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:32:23.680622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:23.680622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:32:23.680622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:32:23.680622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:32:23.680622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:32:23.680622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:32:23.680622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:32:23.680622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:32:23.680622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:32:23.680622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:32:23.680622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:32:23.680622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:32:23.680622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:32:23.680622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:32:23.680622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:32:23.680622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:32:23.681293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:32:23.681293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:32:23.681293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:32:23.681293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:32:23.681293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:32:23.681293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:23.681293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:32:23.681293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:23.681293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:32:23.681293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:23.681293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:32:23.681293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:23.681293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:32:23.681293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:23.681293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:32:23.681293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:32:23.681293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:23.681293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:32:23.681983 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:23.681983 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:32:23.681983 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:32:23.681983 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:23.681983 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:32:23.681983 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:23.681983 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:32:23.681983 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:32:23.681983 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:32:23.681983 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:32:23.682356 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:32:23.682356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:23.682945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:32:23.682945 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:23.682945 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:32:23.682945 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:32:23.682945 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:23.682945 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:32:23.682945 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:23.682945 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:32:23.684565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:23.684565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:32:23.684565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:32:23.684565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:32:23.684565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:23.684565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:32:23.684565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:23.684565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:32:23.684565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:23.684565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:32:23.684565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:23.684565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:32:23.684565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:23.684565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:32:23.684565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:32:23.685150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:23.685150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:32:23.685150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:23.685150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:32:23.685150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:32:23.685150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:23.685150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:32:23.685150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:23.685150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:32:23.685506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:23.685506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:32:23.685506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:32:23.685506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:32:23.685506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:32:23.685506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:32:23.685506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:32:23.685506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:32:23.685506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:32:23.685506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:23.685506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:32:23.686115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:23.686115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:32:23.686115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:23.686115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:32:23.686115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:23.686115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:32:23.686115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:23.686115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:32:23.686115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:32:23.686115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:23.686115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:32:23.686115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:23.686115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:32:23.686115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:32:23.686583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:23.686583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:32:23.686583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:23.686583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:32:23.686583 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:32:23.687359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:23.687359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:32:23.687359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:32:23.687359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:32:23.687359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:32:23.687359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:32:23.687359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:32:23.687359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:32:23.687359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:32:23.687359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:32:23.687359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:32:23.687359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:32:23.687359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:32:23.687359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:32:23.687359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:32:23.687359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:32:23.687359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:32:23.688158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:32:23.688158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:32:23.688158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:32:23.688158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:23.688158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:32:23.688158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:23.688158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:32:23.688158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:23.688158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:32:23.688158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:23.688158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:32:23.688158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:23.688158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:32:23.688158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:32:23.688158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:23.688158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:32:23.688158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:23.688158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:32:23.688158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:32:23.688857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:23.688857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:32:23.688857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:23.688857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:32:23.688857 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:32:23.688857 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:32:23.688857 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:32:23.689151 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:32:23.689151 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:32:23.689364 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:23.689978 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:32:23.689978 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:23.689978 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:32:23.689978 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:32:23.689978 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:23.689978 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:32:23.689978 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:23.689978 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:32:23.690603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:23.690603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:32:23.690603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:32:23.690603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:32:23.690603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:32:23.690603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:32:23.690603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:32:23.690603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:23.690603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:32:23.690603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:23.690603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:32:23.690603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:23.691218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:32:23.691218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:23.691218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:32:23.691218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:23.691218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:32:23.691218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:32:23.691218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:23.691218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:32:23.691218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:23.691218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:32:23.691218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:32:23.691218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:23.691218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:32:23.691665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:23.691665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:32:23.691665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:23.691665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:32:23.691665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:32:23.691665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:32:23.691665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:32:23.691665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:32:23.691665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:32:23.691665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:32:23.691665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:32:23.691665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:32:23.691665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:32:23.691665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:32:23.691665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:32:23.692253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:32:23.692253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:32:23.692253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:32:23.692253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:32:23.692253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:32:23.692253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:23.692253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:32:23.692253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:23.692253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:32:23.692253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:23.692253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:32:23.692253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:23.692253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:32:23.692253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:23.692253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:32:23.692253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:32:23.692253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:23.692253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:32:23.692253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:23.693017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:32:23.693017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:32:23.693017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:23.693017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:32:23.693017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:23.693017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:32:28.503402 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:32:28.504039 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:32:28.504039 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:32:28.504885 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:32:28.544349 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:32:28.544835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:28.545904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:32:28.545904 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:28.545904 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:32:28.545904 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:32:28.545904 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:28.545904 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:32:28.545904 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:28.545904 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:32:28.546965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:28.546965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:32:28.546965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:32:28.546965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:32:28.546965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:32:28.546965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:32:28.546965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:32:28.546965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:28.546965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:32:28.546965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:28.546965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:32:28.546965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:28.547718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:32:28.547718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:28.547718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:32:28.547718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:28.547718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:32:28.547718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:32:28.547718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:28.547718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:32:28.547718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:28.547718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:32:28.547718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:32:28.547718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:28.547718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:32:28.548594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:28.548594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:32:28.548594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:28.548594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:32:28.548594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:32:28.548594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:32:28.548594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:32:28.548594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:32:28.548594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:32:28.548594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:32:28.548594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:32:28.548594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:32:28.548594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:32:28.548594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:32:28.548594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:32:28.549506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:32:28.549506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:32:28.549506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:32:28.549506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:32:28.549506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:32:28.549506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:28.549506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:32:28.549506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:28.549506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:32:28.549506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:28.549506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:32:28.549506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:28.549506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:32:28.549506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:28.549506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:32:28.549506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:32:28.549506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:28.549506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:32:28.549506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:28.550608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:32:28.550608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:32:28.550608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:28.550608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:32:28.550608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:28.550608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:32:28.658302 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:32:28.659523 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:32:28.659523 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:32:28.661243 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:32:28.661243 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:32:28.661243 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:28.661436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:32:28.661436 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:28.661436 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:32:28.661436 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:32:28.661436 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:28.661436 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:32:28.661436 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:28.661436 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:32:28.662431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:28.662431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:32:28.662431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:32:28.662431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:32:28.662431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:32:28.662431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:32:28.662431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:32:28.662431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:28.662431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:32:28.662431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:28.662431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:32:28.662431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:28.662999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:32:28.662999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:28.662999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:32:28.662999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:28.662999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:32:28.662999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:32:28.662999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:28.662999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:32:28.662999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:28.662999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:32:28.662999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:32:28.662999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:28.662999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:32:28.663623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:28.663623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:32:28.663623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:28.663623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:32:28.663623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:32:28.663623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:32:28.663623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:32:28.663623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:32:28.663623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:32:28.663623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:32:28.663623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:32:28.663623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:32:28.663623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:32:28.663623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:32:28.663623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:32:28.664477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:32:28.664477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:32:28.664477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:32:28.664477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:32:28.664477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:32:28.664477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:28.664477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:32:28.664477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:28.664477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:32:28.664477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:28.664477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:32:28.664477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:28.664477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:32:28.664477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:28.664477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:32:28.664477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:32:28.664477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:28.664477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:32:28.664477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:28.665271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:32:28.665271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:32:28.665271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:28.665271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:32:28.665271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:28.665271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:32:28.671286 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:32:28.671584 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:32:28.671700 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:32:28.671840 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:32:28.672707 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:32:28.672707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:28.672971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:32:28.672971 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:28.672971 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:32:28.672971 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:32:28.672971 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:28.672971 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:32:28.672971 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:28.672971 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:32:28.674008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:28.674008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:32:28.674008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:32:28.674008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:32:28.674008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:28.674008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:32:28.674008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:28.674008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:32:28.674008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:28.674008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:32:28.674008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:28.674008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:32:28.674008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:28.674008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:32:28.674008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:32:28.674519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:28.674519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:32:28.674519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:28.674519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:32:28.674519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:32:28.674519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:28.674519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:32:28.674519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:28.674519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:32:28.675150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:28.675150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:32:28.675150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:32:28.675150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:32:28.675150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:32:28.675150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:32:28.675150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:32:28.675150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:32:28.675150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:32:28.675150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:32:28.675150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:28.675150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:32:28.675598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:28.675598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:32:28.675598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:28.675598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:32:28.675598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:28.675598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:32:28.675598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:28.675598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:32:28.675598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:32:28.675598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:28.675598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:32:28.675598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:28.675598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:32:28.675598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:32:28.676095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:28.676095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:32:28.676095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:28.676095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:32:28.676382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:28.676382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:32:28.676382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:32:28.676382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:32:28.676382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:32:28.676382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:32:28.676382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:32:28.676382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:32:28.676382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:32:28.676382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:32:28.676382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:32:28.676382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:32:28.676382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:32:28.676382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:32:28.676382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:32:28.676382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:32:28.676949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:32:28.676949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:32:28.676949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:32:28.676949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:32:28.676949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:32:28.676949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:28.676949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:32:28.676949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:28.676949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:32:28.676949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:28.676949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:32:28.676949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:28.676949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:32:28.676949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:28.676949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:32:28.676949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:32:28.676949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:28.676949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:32:28.677561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:28.677561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:32:28.677561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:32:28.677561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:28.677561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:32:28.677561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:28.677561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:32:28.682928 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:32:28.683284 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:32:28.683467 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:32:28.683652 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:32:28.684278 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:32:28.684462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:28.685072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:32:28.685072 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:28.685072 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:32:28.685072 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:32:28.685072 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:28.685072 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:32:28.685072 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:28.685072 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:32:28.685693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:28.685693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:32:28.685693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:32:28.685693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:32:28.685693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:28.685693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:32:28.685693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:28.685693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:32:28.685693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:28.685693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:32:28.685693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:28.685693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:32:28.685693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:28.685693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:32:28.685693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:32:28.686224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:28.686224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:32:28.686224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:28.686224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:32:28.686224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:32:28.686224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:28.686224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:32:28.686224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:28.686224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:32:28.686496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:28.686496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:32:28.686496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:32:28.686496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:32:28.686496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:32:28.686496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:32:28.686496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:32:28.686496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:32:28.686496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:32:28.686496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:28.686496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:32:28.686852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:28.686852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:32:28.686852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:28.686852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:32:28.686852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:28.686852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:32:28.686852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:28.686852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:32:28.686852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:32:28.686852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:28.686852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:32:28.686852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:28.686852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:32:28.686852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:32:28.687406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:28.687406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:32:28.687406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:28.687406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:32:28.687406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:28.687406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:32:28.687406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:32:28.687406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:32:28.687406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:32:28.687406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:32:28.687406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:32:28.687406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:32:28.687406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:32:28.687406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:32:28.687888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:32:28.687888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:32:28.687888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:32:28.687888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:32:28.687888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:32:28.687888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:32:28.687888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:32:28.687888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:32:28.687888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:32:28.687888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:32:28.687888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:28.687888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:32:28.687888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:28.687888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:32:28.687888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:28.687888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:32:28.687888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:28.687888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:32:28.687888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:28.688654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:32:28.688654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:32:28.688654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:28.688654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:32:28.688654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:28.688654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:32:28.688654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:32:28.688654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:28.688654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:32:28.688654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:28.688654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:32:28.688654 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:32:28.688654 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:32:28.689375 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:32:28.689375 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:32:28.689375 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:32:28.689375 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:32:28.689375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:28.689375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:32:28.689375 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:28.689375 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:32:28.689375 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:32:28.689773 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:28.689773 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:32:28.689773 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:28.689773 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:32:28.689954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:28.689954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:32:28.689954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:32:28.689954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:32:28.689954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:32:28.689954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:32:28.689954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:32:28.689954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:28.689954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:32:28.689954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:28.689954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:32:28.689954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:28.690589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:32:28.690589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:28.690589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:32:28.690589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:28.690589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:32:28.690589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:32:28.690589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:28.690589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:32:28.690589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:28.690589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:32:28.690589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:32:28.690589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:28.690589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:32:28.691102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:28.691102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:32:28.691102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:28.691102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:32:28.691102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:32:28.691102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:32:28.691102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:32:28.691102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:32:28.691102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:32:28.691102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:32:28.691102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:32:28.691102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:32:28.691102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:32:28.691102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:32:28.691102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:32:28.691651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:32:28.691651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:32:28.691651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:32:28.691651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:32:28.691651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:32:28.691651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:28.691651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:32:28.691651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:28.691651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:32:28.691651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:28.691651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:32:28.691651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:28.691651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:32:28.691651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:28.691651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:32:28.691651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:32:28.691651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:28.691651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:32:28.691651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:28.692404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:32:28.692404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:32:28.692404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:28.692404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:32:28.692404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:28.692404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:32:33.504129 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:32:33.505055 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:32:33.505186 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:32:33.506572 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:32:33.545002 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:32:33.545299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:33.546124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:32:33.546124 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:33.546124 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:32:33.546124 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:32:33.546124 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:33.546124 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:32:33.546124 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:33.546124 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:32:33.547259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:33.547259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:32:33.547259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:32:33.547259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:32:33.547259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:32:33.547259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:32:33.547259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:32:33.547259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:33.547259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:32:33.547259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:33.547259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:32:33.547259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:33.547993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:32:33.547993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:33.547993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:32:33.547993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:33.547993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:32:33.547993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:32:33.547993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:33.547993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:32:33.547993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:33.547993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:32:33.547993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:32:33.547993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:33.547993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:32:33.548671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:33.548671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:32:33.548671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:33.548671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:32:33.548671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:32:33.548671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:32:33.548671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:32:33.548671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:32:33.548671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:32:33.548671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:32:33.548671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:32:33.548671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:32:33.548671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:32:33.548671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:32:33.548671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:32:33.549441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:32:33.549441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:32:33.549441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:32:33.549441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:32:33.549441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:32:33.549441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:33.549441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:32:33.549441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:33.549441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:32:33.549441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:33.549441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:32:33.549441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:33.549441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:32:33.549441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:33.549441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:32:33.549441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:32:33.549441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:33.549441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:32:33.549441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:33.550388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:32:33.550388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:32:33.550388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:33.550388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:32:33.550388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:33.550388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:32:33.658831 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:32:33.659954 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:32:33.660237 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:32:33.661595 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:32:33.661893 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:32:33.662098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:33.662968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:32:33.662968 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:33.662968 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:32:33.662968 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:32:33.662968 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:33.662968 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:32:33.662968 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:33.662968 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:32:33.664776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:33.664776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:32:33.664776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:32:33.664776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:32:33.664776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:32:33.664776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:32:33.664776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:32:33.664776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:33.664776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:32:33.664776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:33.664776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:32:33.664776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:33.665234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:32:33.665234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:33.665234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:32:33.665234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:33.665234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:32:33.665234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:32:33.665234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:33.665234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:32:33.665234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:33.665234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:32:33.665234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:32:33.665234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:33.665234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:32:33.665610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:33.665610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:32:33.666409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:33.666409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:32:33.666409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:32:33.666409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:32:33.666409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:32:33.666409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:32:33.666409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:32:33.666409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:32:33.666409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:32:33.666409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:32:33.666409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:32:33.666409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:32:33.666409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:32:33.666409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:32:33.666409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:32:33.666409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:32:33.666931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:32:33.666931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:32:33.666931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:33.666931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:32:33.666931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:33.666931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:32:33.666931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:33.666931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:32:33.666931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:33.666931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:32:33.666931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:33.666931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:32:33.666931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:32:33.666931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:33.666931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:32:33.666931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:33.666931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:32:33.666931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:32:33.666931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:33.667495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:32:33.667495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:33.667495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:32:33.671759 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:32:33.672059 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:32:33.672139 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:32:33.672245 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:32:33.672695 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:32:33.673133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:33.673996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:32:33.673996 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:33.673996 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:32:33.673996 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:32:33.673996 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:33.673996 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:32:33.673996 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:33.673996 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:32:33.675173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:33.675173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:32:33.675173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:32:33.675173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:32:33.675173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:33.675173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:32:33.675173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:33.675173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:32:33.675173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:33.675173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:32:33.675173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:33.675173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:32:33.675173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:33.675173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:32:33.675173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:32:33.675766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:33.675766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:32:33.675766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:33.675766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:32:33.675766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:32:33.675766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:33.675766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:32:33.675766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:33.675766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:32:33.676312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:33.676312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:32:33.676312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:32:33.676312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:32:33.676312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:32:33.676312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:32:33.676312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:32:33.676312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:32:33.676312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:32:33.676312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:32:33.676312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:33.676312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:32:33.676729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:33.676729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:32:33.676729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:33.676729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:32:33.676729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:33.676729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:32:33.676729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:33.676729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:32:33.676729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:32:33.676729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:33.676729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:32:33.676729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:33.676729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:32:33.676729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:32:33.677381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:33.677381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:32:33.677381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:33.677381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:32:33.677381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:33.677381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:32:33.677381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:32:33.677381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:32:33.677381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:32:33.677381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:32:33.677381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:32:33.677381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:32:33.677381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:32:33.677381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:32:33.678456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:32:33.678456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:32:33.678456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:32:33.678456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:32:33.678456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:32:33.678456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:32:33.678456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:32:33.678456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:32:33.678456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:32:33.678456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:32:33.678456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:32:33.678456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:33.678456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:32:33.678456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:33.678456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:32:33.678456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:33.678456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:32:33.678456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:33.678456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:32:33.679918 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:33.679918 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:32:33.679918 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:32:33.679918 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:33.679918 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:32:33.679918 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:33.679918 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:32:33.679918 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:32:33.679918 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:33.679918 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:32:33.679918 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:33.679918 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:32:33.683451 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:32:33.683892 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:32:33.684053 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:32:33.684386 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:32:33.684929 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:32:33.685343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:33.685738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:32:33.685738 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:33.685738 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:32:33.685738 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:32:33.685738 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:33.685738 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:32:33.685738 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:33.685738 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:32:33.686329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:33.686329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:32:33.686329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:32:33.686329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:32:33.686329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:33.686329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:32:33.686329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:33.686329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:32:33.686329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:33.686329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:32:33.686329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:33.686329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:32:33.686329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:33.686329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:32:33.686329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:32:33.687106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:33.687106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:32:33.687106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:33.687106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:32:33.687106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:32:33.687106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:33.687106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:32:33.687106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:33.687106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:32:33.687106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:33.687106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:32:33.687106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:32:33.687106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:32:33.687106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:32:33.687820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:32:33.687820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:32:33.687820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:32:33.687820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:32:33.687820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:33.687820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:32:33.687820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:33.687820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:32:33.687820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:33.687820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:32:33.687820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:33.687820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:32:33.687820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:33.688503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:32:33.688503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:32:33.688503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:33.688503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:32:33.688503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:33.688503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:32:33.688503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:32:33.688503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:33.688503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:32:33.688503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:33.688503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:32:33.688503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:33.689177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:32:33.689177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:32:33.689177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:32:33.689177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:32:33.689177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:32:33.689177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:32:33.689177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:32:33.689177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:32:33.689177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:32:33.689177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:32:33.689177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:32:33.689177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:32:33.689177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:32:33.689177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:32:33.689177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:32:33.689177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:32:33.689177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:32:33.689177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:32:33.689177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:32:33.690812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:33.690812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:32:33.690812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:33.690812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:32:33.690812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:33.690812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:32:33.690812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:33.690812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:32:33.690812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:33.690812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:32:33.690812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:32:33.690812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:33.690812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:32:33.690812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:33.690812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:32:33.690812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:32:33.690812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:33.690812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:32:33.690812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:33.690812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:32:33.691964 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:32:33.691964 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:32:33.691964 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:32:33.691964 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:32:33.691964 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:32:33.691964 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:32:33.692376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:33.692376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:32:33.692376 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:33.692376 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:32:33.692376 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:32:33.692376 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:33.692376 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:32:33.692376 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:33.692376 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:32:33.692376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:33.692376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:32:33.692376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:32:33.692376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:32:33.692376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:32:33.692376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:32:33.692896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:32:33.692896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:33.692896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:32:33.692896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:33.692896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:32:33.692896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:33.692896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:32:33.692896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:33.692896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:32:33.692896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:33.692896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:32:33.692896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:32:33.692896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:33.692896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:32:33.693318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:33.693318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:32:33.693318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:32:33.693318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:33.693318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:32:33.693318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:33.693318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:32:33.693318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:33.693318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:32:33.693318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:32:33.693318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:32:33.693318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:32:33.693318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:32:33.693775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:32:33.693775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:32:33.693775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:32:33.693775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:32:33.693775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:32:33.693775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:32:33.693775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:32:33.693775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:32:33.693775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:32:33.693775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:32:33.693775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:32:33.693775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:32:33.693775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:33.693775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:32:33.693775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:33.693775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:32:33.693775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:33.693775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:32:33.693775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:33.693775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:32:33.694356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:33.694356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:32:33.694356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:32:33.694356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:33.694356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:32:33.694356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:33.694356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:32:33.694356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:32:33.694356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:33.694356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:32:33.694356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:33.694356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:32:38.504953 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:32:38.505778 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:32:38.506200 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:32:38.507783 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:32:38.556739 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:32:38.557082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:38.558054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:32:38.558054 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:38.558054 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:32:38.558054 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:32:38.558054 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:38.558054 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:32:38.558054 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:38.558054 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:32:38.559213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.052 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:38.559213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:32:38.559213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:32:38.559213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:32:38.559213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:32:38.559213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:32:38.559213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:32:38.559213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:38.559213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:32:38.559213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:38.559213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:32:38.559213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:38.559575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:32:38.559575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:38.559575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:32:38.559575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:38.559575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:32:38.559575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:32:38.559575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:38.559575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:32:38.559575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:38.559575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:32:38.559575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:32:38.559575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:38.559575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:32:38.560029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:38.560029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:32:38.560474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:38.560474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:32:38.560474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:32:38.560474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:32:38.560474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:32:38.560474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:32:38.560474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:32:38.560474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:32:38.560474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:32:38.560474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:32:38.560474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:32:38.560474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:32:38.560474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:32:38.560474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:32:38.560474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:32:38.560474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:32:38.560957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:32:38.560957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:32:38.560957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:38.560957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:32:38.560957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:38.560957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:32:38.560957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:38.560957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:32:38.560957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:38.560957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:32:38.560957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:38.560957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:32:38.560957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:32:38.560957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:38.560957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:32:38.560957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:38.560957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:32:38.560957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:32:38.560957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:38.561528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:32:38.561528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:38.561528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:32:38.660371 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:32:38.661130 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:32:38.661325 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:32:38.664236 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:32:38.664717 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:32:38.665079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:38.666142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:32:38.666142 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:38.666142 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:32:38.666142 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:32:38.666142 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:38.666142 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:32:38.666142 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:38.666142 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:32:38.668043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:38.668043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:32:38.668043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:32:38.668043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:32:38.668043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:32:38.668043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:32:38.668043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:32:38.668043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:38.668043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:32:38.668043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:38.668043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:32:38.668043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:38.668719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:32:38.668719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:38.668719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:32:38.668719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:38.668719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:32:38.668719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:32:38.668719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:38.668719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:32:38.668719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:38.668719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:32:38.668719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:32:38.668719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:38.668719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:32:38.669432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:38.669432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:32:38.669432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:38.669432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:32:38.669432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:32:38.669432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:32:38.669432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:32:38.669432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:32:38.669432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:32:38.669432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:32:38.669432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:32:38.669432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:32:38.669432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:32:38.669432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:32:38.669432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:32:38.670200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:32:38.670200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:32:38.670200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:32:38.670200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:32:38.670200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:32:38.670200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:38.670200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:32:38.670200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:38.670200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:32:38.670200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:38.670200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:32:38.670200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:38.670200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:32:38.670200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:38.670200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:32:38.670200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:32:38.670200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:38.670200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:32:38.670200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:38.671192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:32:38.671192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:32:38.671192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:38.671192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:32:38.671192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:38.671192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:32:38.672590 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:32:38.672943 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:32:38.673193 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:32:38.673193 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:32:38.674282 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:32:38.674510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:38.675119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:32:38.675119 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:38.675119 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:32:38.675119 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:32:38.675119 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:38.675119 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:32:38.675119 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:38.675119 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:32:38.676252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:38.676252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:32:38.676252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:32:38.676252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:32:38.676252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:38.676252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:32:38.676252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:38.676252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:32:38.676252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:38.676252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:32:38.676252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:38.676252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:32:38.676252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:38.676252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:32:38.676252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:32:38.677031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:38.677031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:32:38.677031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:38.677031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:32:38.677031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:32:38.677031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:38.677031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:32:38.677031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:38.677031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:32:38.677031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:38.677031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:32:38.677031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:32:38.677031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:32:38.677031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:32:38.677611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:32:38.677611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:32:38.677611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:32:38.677611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:32:38.677611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:32:38.677611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:38.677611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:32:38.677611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:38.677611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:32:38.677611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:38.677611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:32:38.677611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:38.677611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:32:38.678293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:38.678293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:32:38.678293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:32:38.678293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:38.678293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:32:38.678293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:38.678293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:32:38.678293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:32:38.678293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:38.678293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:32:38.678293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:38.678293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:32:38.678808 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:38.678808 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:32:38.678808 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:32:38.678808 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:32:38.678808 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:32:38.678808 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:32:38.678808 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:32:38.678808 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:32:38.678808 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:32:38.678808 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:32:38.678808 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:32:38.678808 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:32:38.678808 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:32:38.678808 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:32:38.678808 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:32:38.678808 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:32:38.679411 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:32:38.679411 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:32:38.679411 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:32:38.679411 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:32:38.679411 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:32:38.679411 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:38.679411 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:32:38.679411 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:38.679411 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:32:38.679411 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:38.679411 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:32:38.679411 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:38.679411 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:32:38.679411 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:38.679411 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:32:38.679411 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:32:38.679411 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:38.679411 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:32:38.680165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:38.680165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:32:38.680165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:32:38.680165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:38.680165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:32:38.680165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:38.680165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:32:38.684312 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:32:38.684659 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:32:38.684812 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:32:38.684956 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:32:38.685396 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:32:38.685639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:38.686549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:32:38.686549 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:38.686549 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:32:38.686549 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:32:38.686549 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:38.686549 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:32:38.686549 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:38.686549 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:32:38.687823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:38.687823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:32:38.687823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:32:38.687823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:32:38.687823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:38.687823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:32:38.687823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:38.687823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:32:38.687823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:38.687823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:32:38.687823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:38.687823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:32:38.687823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:38.687823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:32:38.687823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:32:38.688275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:38.688275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:32:38.688275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:38.688275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:32:38.688275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:32:38.688275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:38.688275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:32:38.688275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:38.688275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:32:38.689437 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:38.689437 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:32:38.689437 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:32:38.689437 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:32:38.689437 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:32:38.689437 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:32:38.689437 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:32:38.689437 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:32:38.689437 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:32:38.689437 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:38.689437 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:32:38.689878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:38.689878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:32:38.689878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:38.689878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:32:38.689878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:38.689878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:32:38.689878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:38.689878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:32:38.689878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:32:38.689878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:38.689878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:32:38.689878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:38.689878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:32:38.689878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:32:38.690329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:38.690329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:32:38.690329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:38.690329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:32:38.691242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:38.691242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:32:38.691242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:32:38.691242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:32:38.691242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:32:38.691242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:32:38.691242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:32:38.691242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:32:38.691242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:32:38.691242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:32:38.691242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:32:38.691242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:32:38.691242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:32:38.691242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:32:38.691242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:32:38.691242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:32:38.691242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:32:38.691866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:32:38.691866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:32:38.691866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:32:38.691866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:38.691866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:32:38.691866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:38.691866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:32:38.691866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:38.691866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:32:38.691866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:38.691866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:32:38.691866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:38.691866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:32:38.691866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:32:38.691866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:38.691866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:32:38.691866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:38.691866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:32:38.691866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:32:38.692595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:38.692595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:32:38.692595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:38.692595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:32:38.692780 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:32:38.693217 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:32:38.693761 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:32:38.693761 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:32:38.694309 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:32:38.694719 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:32:38.694993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:38.695824 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:32:38.695824 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:38.695824 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:32:38.695824 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:32:38.695824 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:38.695824 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:32:38.695824 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:38.695824 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:32:38.696704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:38.696704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:32:38.696704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:32:38.696704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:32:38.696704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:32:38.696704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:32:38.696704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:32:38.696704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:38.696704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:32:38.696704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:38.696704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:32:38.696704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:38.697256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:32:38.697256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:38.697256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:32:38.697256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:38.697256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:32:38.697256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:32:38.697256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:38.697256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:32:38.697256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:38.697256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:32:38.697256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:32:38.697256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:38.697256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:32:38.697798 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:38.697798 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:32:38.697798 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:38.697798 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:32:38.697798 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:32:38.697798 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:32:38.697798 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:32:38.697798 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:32:38.697798 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:32:38.697798 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:32:38.697798 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:32:38.697798 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:32:38.697798 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:32:38.697798 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:32:38.697798 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:32:38.698383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:32:38.698383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:32:38.698383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:32:38.698383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:32:38.698383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:32:38.698383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:38.698383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:32:38.698383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:38.698383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:32:38.698383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:38.698383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:32:38.698383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:38.698383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:32:38.698383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:38.698383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:32:38.698383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:32:38.698383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:38.698383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:32:38.698383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:38.699136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:32:38.699136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:32:38.699136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:38.699136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:32:38.699136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:38.699136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:32:43.505825 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:32:43.506534 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:32:43.506835 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:32:43.508189 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:32:43.546562 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:32:43.546984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:43.547911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:32:43.547911 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:43.547911 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:32:43.547911 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:32:43.547911 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:43.547911 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:32:43.547911 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:43.547911 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:32:43.548791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:43.548791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:32:43.548791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:32:43.548791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:32:43.548791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:32:43.548791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:32:43.548791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:32:43.548791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:43.548791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:32:43.548791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:43.548791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:32:43.548791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:43.549418 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:32:43.549418 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:43.549418 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:32:43.549418 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:43.549418 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:32:43.549418 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:32:43.549418 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:43.549418 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:32:43.549418 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:43.549418 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:32:43.549418 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:32:43.549418 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:43.549418 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:32:43.550199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:43.550199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:32:43.550199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:43.550199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:32:43.550199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:32:43.550199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:32:43.550199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:32:43.550199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:32:43.550199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:32:43.550199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:32:43.550199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:32:43.550199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:32:43.550199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:32:43.550199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:32:43.550199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:32:43.550890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:32:43.550890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:32:43.550890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:32:43.550890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:32:43.550890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:32:43.550890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:43.550890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:32:43.550890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:43.550890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:32:43.550890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:43.550890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:32:43.550890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:43.550890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:32:43.550890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:43.550890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:32:43.550890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:32:43.550890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:43.550890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:32:43.550890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:43.551809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:32:43.551809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:32:43.551809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:43.551809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:32:43.551809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:43.551809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:32:43.660999 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:32:43.661547 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:32:43.661716 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:32:43.662571 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:32:43.662809 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:32:43.662963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:43.664233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:32:43.664233 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:43.664233 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:32:43.664233 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:32:43.664233 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:43.664233 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:32:43.664233 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:43.664233 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:32:43.665260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:43.665260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:32:43.665260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:32:43.665260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:32:43.665260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:32:43.665260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:32:43.665260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:32:43.665260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:43.665260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:32:43.665260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:43.665260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:32:43.665260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:43.665533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:32:43.665533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:43.665533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:32:43.665533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:43.665533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:32:43.665533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:32:43.665533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:43.665533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:32:43.665533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:43.665533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:32:43.665533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:32:43.665533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:43.665533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:32:43.665792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:43.665792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:32:43.666290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:43.666290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:32:43.666290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:32:43.666290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:32:43.666290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:32:43.666290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:32:43.666290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:32:43.666290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:32:43.666290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:32:43.666290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:32:43.666290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:32:43.666290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:32:43.666290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:32:43.666290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:32:43.666290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:32:43.666290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:32:43.666593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:32:43.666593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:32:43.666593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:43.666593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:32:43.666593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:43.666593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:32:43.666593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:43.666593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:32:43.666593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:43.666593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:32:43.666593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:43.666593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:32:43.666593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:32:43.666593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:43.666593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:32:43.666593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:43.666593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:32:43.666593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:32:43.666593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:43.666957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:32:43.666957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:43.666957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:32:43.673668 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:32:43.674122 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:32:43.674298 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:32:43.674440 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:32:43.674917 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:32:43.675195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:43.675886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:32:43.675886 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:43.675886 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:32:43.675886 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:32:43.675886 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:43.675886 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:32:43.675886 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:43.675886 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:32:43.676773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:43.676773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:32:43.676773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:32:43.676773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:32:43.676773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:43.676773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:32:43.676773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:43.676773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:32:43.676773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:43.676773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:32:43.676773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:43.676773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:32:43.676773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:43.676773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:32:43.676773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:32:43.677677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:43.677677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:32:43.677677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:43.677677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:32:43.677677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:32:43.677677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:43.677677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:32:43.677677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:43.677677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:32:43.677677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:43.677677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:32:43.677677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:32:43.677677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:32:43.677677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:32:43.678593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:32:43.678593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:32:43.678593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:32:43.678593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:32:43.678593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:32:43.678593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:43.678593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:32:43.678593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:43.678593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:32:43.678593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:43.678593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:32:43.678593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:43.678593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:32:43.679380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:43.679380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:32:43.679380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:32:43.679380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:43.679380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:32:43.679380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:43.679380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:32:43.679380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:32:43.679380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:43.679380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:32:43.679380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:43.679380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:32:43.680236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:43.680236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:32:43.680236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:32:43.680236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:32:43.680236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:32:43.680236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:32:43.680236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:32:43.680236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:32:43.680236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:32:43.680236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:32:43.680236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:32:43.680236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:32:43.680236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:32:43.680236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:32:43.680236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:32:43.680236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:32:43.681563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:32:43.681563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:32:43.681563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:32:43.681563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:32:43.681563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:32:43.681563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:43.681563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:32:43.681563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:43.681563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:32:43.681563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:43.681563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:32:43.681563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:43.681563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:32:43.681563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:43.681563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:32:43.681563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:32:43.681563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:43.681563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:32:43.682663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:43.682663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:32:43.682663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:32:43.682663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:43.682663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:32:43.682663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:43.682663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:32:43.684827 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:32:43.685180 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:32:43.685334 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:32:43.685543 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:32:43.686040 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:32:43.686308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:43.687101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:32:43.687101 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:43.687101 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:32:43.687101 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:32:43.687101 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:43.687101 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:32:43.687101 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:43.687101 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:32:43.688562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:43.688562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:32:43.688562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:32:43.688562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:32:43.688562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:43.688562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:32:43.688562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:43.688562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:32:43.688562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:43.688562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:32:43.688562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:43.688562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:32:43.688562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:43.688562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:32:43.688562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:32:43.689249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:43.689249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:32:43.689249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:43.689249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:32:43.689249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:32:43.689249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:43.689249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:32:43.689249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:43.689249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:32:43.689249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:43.689249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:32:43.689249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:32:43.689249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:32:43.689249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:32:43.689821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:32:43.689821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:32:43.689821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:32:43.689821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:32:43.689821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:43.689821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:32:43.689821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:43.689821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:32:43.689821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:43.689821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:32:43.689821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:43.689821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:32:43.689821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:43.690281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:32:43.690281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:32:43.690281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:43.690281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:32:43.690281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:43.690281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:32:43.690281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:32:43.690281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:43.690281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:32:43.690281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:43.690281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:32:43.690665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:43.690665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:32:43.690665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:32:43.690665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:32:43.690665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:32:43.690665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:32:43.690665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:32:43.690665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:32:43.690665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:32:43.690665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:32:43.690665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:32:43.690665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:32:43.690665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:32:43.690665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:32:43.690665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:32:43.690665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:32:43.690665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:32:43.691383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:32:43.691383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:32:43.691383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:32:43.691383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:43.691383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:32:43.691383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:43.691383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:32:43.691383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:43.691383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:32:43.691383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:43.691383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:32:43.691383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:43.691383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:32:43.691383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:32:43.691383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:43.691383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:32:43.691383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:43.691383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:32:43.691383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:32:43.692271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:43.692271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:32:43.692271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:43.692271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:32:43.693270 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:32:43.693586 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:32:43.693693 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:32:43.693903 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:32:43.694283 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:32:43.694534 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:32:43.694687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:43.695294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:32:43.695294 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:43.695294 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:32:43.695294 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:32:43.695294 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:43.695294 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:32:43.695294 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:43.695294 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:32:43.696259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:43.696259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:32:43.696259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:32:43.696259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:32:43.696259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:32:43.696259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:32:43.696259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:32:43.696259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:43.696259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:32:43.696259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:43.696259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:32:43.696259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:43.696606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:32:43.696606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:43.696606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:32:43.696606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:43.696606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:32:43.696606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:32:43.696606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:43.696606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:32:43.696606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:43.696606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:32:43.696606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:32:43.696606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:43.696606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:32:43.696981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:43.696981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:32:43.697425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:43.697425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:32:43.697425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:32:43.697425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:32:43.697425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:32:43.697425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:32:43.697425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:32:43.697425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:32:43.697425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:32:43.697425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:32:43.697425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:32:43.697425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:32:43.697425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:32:43.697425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:32:43.697425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:32:43.697425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:32:43.697888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:32:43.697888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:32:43.697888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:43.697888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:32:43.697888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:43.697888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:32:43.697888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:43.697888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:32:43.697888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:43.697888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:32:43.697888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:43.697888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:32:43.697888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:32:43.697888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:43.697888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:32:43.697888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:43.697888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:32:43.697888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:32:43.697888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:43.698332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:32:43.698332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:43.698332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:32:48.506722 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:32:48.507923 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:32:48.508010 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:32:48.509592 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:32:48.544958 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:32:48.545245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:48.545992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:32:48.545992 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:48.545992 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:32:48.545992 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:32:48.545992 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:48.545992 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:32:48.545992 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:48.545992 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:32:48.547153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:48.547153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:32:48.547153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:32:48.547153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:32:48.547153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:32:48.547153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:32:48.547153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:32:48.547153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:48.547153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:32:48.547153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:48.547153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:32:48.547153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:48.548157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:32:48.548157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:48.548157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:32:48.548157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:48.548157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:32:48.548157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:32:48.548157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:48.548157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:32:48.548157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:48.548157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:32:48.548157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:32:48.548157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:48.548157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:32:48.548894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:48.548894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:32:48.548894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:48.548894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:32:48.548894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:32:48.548894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:32:48.548894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:32:48.548894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:32:48.548894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:32:48.548894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:32:48.548894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:32:48.548894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:32:48.548894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:32:48.548894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:32:48.548894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:32:48.549682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:32:48.549682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:32:48.549682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:32:48.549682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:32:48.549682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:32:48.549682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:48.549682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:32:48.549682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:48.549682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:32:48.549682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:48.549682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:32:48.549682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:48.549682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:32:48.549682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:48.549682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:32:48.549682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:32:48.549682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:48.549682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:32:48.549682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:48.550709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:32:48.550709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:32:48.550709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:48.550709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:32:48.550709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:48.550709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:32:48.661980 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:32:48.662486 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:32:48.662590 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:32:48.663619 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:32:48.664053 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:32:48.664307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:48.665284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:32:48.665284 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:48.665284 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:32:48.665284 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:32:48.665284 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:48.665284 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:32:48.665284 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:48.665284 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:32:48.666426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:48.666426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:32:48.666426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:32:48.666426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:32:48.666426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:32:48.666426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:32:48.666426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:32:48.666426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:48.666426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:32:48.666426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:48.666426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:32:48.666426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:48.667059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:32:48.667059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:48.667059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:32:48.667059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:48.667059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:32:48.667059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:32:48.667059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:48.667059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:32:48.667059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:48.667059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:32:48.667059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:32:48.667059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:48.667059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:32:48.667600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:48.667600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:32:48.667600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:48.667600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:32:48.667600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:32:48.667600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:32:48.667600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:32:48.667600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:32:48.667600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:32:48.667600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:32:48.667600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:32:48.667600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:32:48.667600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:32:48.667600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:32:48.667600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:32:48.668366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:32:48.668366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:32:48.668366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:32:48.668366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:32:48.668366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:32:48.668366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:48.668366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:32:48.668366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:48.668366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:32:48.668366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:48.668366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:32:48.668366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:48.668366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:32:48.668366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:48.668366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:32:48.668366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:32:48.668366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:48.668366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:32:48.668366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:48.669145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:32:48.669145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:32:48.669145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:48.669145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:32:48.669145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:48.669145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:32:48.674008 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:32:48.674335 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:32:48.674444 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:32:48.674579 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:32:48.675057 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:32:48.675283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:48.675945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:32:48.675945 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:48.675945 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:32:48.675945 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:32:48.675945 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:48.675945 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:32:48.675945 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:48.675945 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:32:48.676770 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:48.676770 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:32:48.676770 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:32:48.676770 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:32:48.676770 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:48.676770 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:32:48.676770 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:48.676770 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:32:48.676770 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:48.676770 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:32:48.676770 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:48.676770 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:32:48.676770 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:48.676770 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:32:48.676770 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:32:48.677302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:48.677302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:32:48.677302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:48.677302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:32:48.677302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:32:48.677302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:48.677302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:32:48.677302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:48.677302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:32:48.677594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:48.677594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:32:48.677594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:32:48.677594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:32:48.677594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:32:48.677594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:32:48.677594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:32:48.677594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:32:48.677594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:32:48.677594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:32:48.677594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:48.677594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:32:48.678020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:48.678020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:32:48.678020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:48.678020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:32:48.678020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:48.678020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:32:48.678020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:48.678020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:32:48.678020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:32:48.678020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:48.678020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:32:48.678020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:48.678020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:32:48.678020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:32:48.678405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:48.678405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:32:48.678405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:48.678405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:32:48.678655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:48.678655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:32:48.678655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:32:48.678655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:32:48.678655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:32:48.678655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:32:48.678655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:32:48.678655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:32:48.678655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:32:48.678655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:32:48.678655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:32:48.678655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:32:48.678655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:32:48.678655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:32:48.678655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:32:48.678655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:32:48.679213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:32:48.679213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:32:48.679213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:32:48.679213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:32:48.679213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:32:48.679213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:48.679213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:32:48.679213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:48.679213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:32:48.679213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:48.679213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:32:48.679213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:48.679213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:32:48.679213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:48.679213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:32:48.679213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:32:48.679213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:48.679213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:32:48.679761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:48.679761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:32:48.679761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:32:48.679761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:48.679761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:32:48.679761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:48.679761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:32:48.685495 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:32:48.685891 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:32:48.686068 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:32:48.686143 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:32:48.686729 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:32:48.687192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:48.688024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:32:48.688024 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:48.688024 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:32:48.688024 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:32:48.688024 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:48.688024 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:32:48.688024 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:48.688024 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:32:48.688839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:48.688839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:32:48.688839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:32:48.688839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:32:48.688839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:48.688839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:32:48.688839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:48.688839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:32:48.688839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:48.688839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:32:48.688839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:48.688839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:32:48.688839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:48.688839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:32:48.688839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:32:48.689318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:48.689318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:32:48.689318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:48.689318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:32:48.689318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:32:48.689318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:48.689318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:32:48.689318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:48.689318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:32:48.689730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:48.689730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:32:48.689730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:32:48.689730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:32:48.689730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:32:48.689730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:32:48.689730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:32:48.689730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:32:48.689730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:32:48.689730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:48.689730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:32:48.690319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:48.690319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:32:48.690319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:48.690319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:32:48.690319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:48.690319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:32:48.690319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:48.690319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:32:48.690319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:32:48.690319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:48.690319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:32:48.690319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:48.690319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:32:48.690319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:32:48.691072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:48.691072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:32:48.691072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:48.691072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:32:48.691072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:48.691072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:32:48.691072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:32:48.691072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:32:48.691072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:32:48.691072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:32:48.691072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:32:48.691072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:32:48.691072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:32:48.691072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:32:48.691514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:32:48.691514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:32:48.691514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:32:48.691514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:32:48.691514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:32:48.691514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:32:48.691514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:32:48.691514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:32:48.691514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:32:48.691514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:32:48.691514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:48.691514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:32:48.691514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:48.691514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:32:48.691514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:48.691514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:32:48.691514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:48.691514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:32:48.691514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:48.692123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:32:48.692123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:32:48.692123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:48.692123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:32:48.692123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:48.692123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:32:48.692123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:32:48.692123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:48.692123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:32:48.692123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:48.692123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:32:48.693769 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:32:48.694349 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:32:48.694503 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:32:48.694815 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:32:48.695328 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:32:48.695614 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:32:48.695783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:48.696615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:32:48.696615 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:48.696615 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:32:48.696615 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:32:48.696615 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:48.696615 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:32:48.696615 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:48.696615 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:32:48.697837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:48.697837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:32:48.697837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:32:48.697837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:32:48.697837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:32:48.697837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:32:48.697837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:32:48.697837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:48.697837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:32:48.697837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:48.697837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:32:48.697837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:48.698311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:32:48.698311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:48.698311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:32:48.698311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:48.698311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:32:48.698311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:32:48.698311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:48.698311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:32:48.698311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:48.698311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:32:48.698311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:32:48.698311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:48.698311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:32:48.698771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:48.698771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:32:48.699303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:48.699303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:32:48.699303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:32:48.699303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:32:48.699303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:32:48.699303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:32:48.699303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:32:48.699303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:32:48.699303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:32:48.699303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:32:48.699303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:32:48.699303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:32:48.699303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:32:48.699303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:32:48.699303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:32:48.699303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:32:48.699942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:32:48.699942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:32:48.699942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:48.699942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:32:48.699942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:48.699942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:32:48.699942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:48.699942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:32:48.699942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:48.699942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:32:48.699942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:48.699942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:32:48.699942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:32:48.699942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:48.699942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:32:48.699942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:48.699942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:32:48.699942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:32:48.699942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:48.700568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:32:48.700568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:48.700568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:32:53.507413 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:32:53.508515 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:32:53.508515 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:32:53.508867 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:32:53.547057 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:32:53.547480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:53.548383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:32:53.548383 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:53.548383 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:32:53.548383 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:32:53.548383 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:53.548383 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:32:53.548383 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:53.548383 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:32:53.549283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:53.549283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:32:53.549283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:32:53.549283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:32:53.549283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:32:53.549283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:32:53.549283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:32:53.549283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:53.549283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:32:53.549283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:53.549283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:32:53.549283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:53.549999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:32:53.549999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:53.549999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:32:53.549999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:53.549999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:32:53.549999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:32:53.549999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:53.549999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:32:53.549999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:53.549999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:32:53.549999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:32:53.549999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:53.549999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:32:53.550450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:53.550450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:32:53.550450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:53.550450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:32:53.550450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:32:53.550450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:32:53.550450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:32:53.550450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:32:53.550450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:32:53.550450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:32:53.550450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:32:53.550450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:32:53.550450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:32:53.550450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:32:53.550450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:32:53.550966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:32:53.550966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:32:53.550966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:32:53.550966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:32:53.550966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:32:53.550966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:53.550966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:32:53.550966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:53.550966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:32:53.550966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:53.550966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:32:53.550966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:53.550966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:32:53.550966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:53.550966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:32:53.550966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:32:53.550966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:53.550966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:32:53.550966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:53.551466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:32:53.551466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:32:53.551466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:53.551466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:32:53.551466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:53.551466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:32:53.662356 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:32:53.662835 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:32:53.663004 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:32:53.663803 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:32:53.664062 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:32:53.664414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:53.665015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:32:53.665015 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:53.665015 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:32:53.665015 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:32:53.665015 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:53.665015 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:32:53.665015 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:53.665015 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:32:53.666625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:53.666625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:32:53.666625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:32:53.666625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:32:53.666625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:32:53.666625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:32:53.666625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:32:53.666625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:53.666625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:32:53.666625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:53.666625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:32:53.666625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:53.667118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:32:53.667118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:53.667118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:32:53.667118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:53.667118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:32:53.667118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:32:53.667118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:53.667118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:32:53.667118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:53.667118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:32:53.667118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:32:53.667118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:53.667118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:32:53.667624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:53.667624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:32:53.668766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:53.668766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:32:53.668766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:32:53.668766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:32:53.668766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:32:53.668766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:32:53.668766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:32:53.668766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:32:53.668766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:32:53.668766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:32:53.668766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:32:53.668766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:32:53.668766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:32:53.668766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:32:53.668766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:32:53.668766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:32:53.669366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:32:53.669366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:32:53.669366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:53.669366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:32:53.669366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:53.669366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:32:53.669366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:53.669366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:32:53.669366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:53.669366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:32:53.669366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:53.669366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:32:53.669366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:32:53.669366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:53.669366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:32:53.669366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:53.669366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:32:53.669366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:32:53.669366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:53.670060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:32:53.670060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:53.670060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:32:53.674758 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:32:53.675243 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:32:53.675360 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:32:53.675487 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:32:53.676191 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:32:53.676528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:53.677441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:32:53.677441 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:53.677441 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:32:53.677441 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:32:53.677441 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:53.677441 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:32:53.677441 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:53.677441 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:32:53.678258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:53.678258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:32:53.678258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:32:53.678258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:32:53.678258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:53.678258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:32:53.678258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:53.678258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:32:53.678258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:53.678258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:32:53.678258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:53.678258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:32:53.678258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:53.678258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:32:53.678258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:32:53.678899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:53.678899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:32:53.678899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:53.678899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:32:53.678899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:32:53.678899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:53.678899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:32:53.678899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:53.678899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:32:53.679447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:53.679447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:32:53.679447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:32:53.679447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:32:53.679447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:32:53.679447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:32:53.679447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:32:53.679447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:32:53.679447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:32:53.679447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:32:53.679447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:53.679447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:32:53.679974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:53.679974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:32:53.679974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:53.679974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:32:53.679974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:53.679974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:32:53.679974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:53.679974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:32:53.679974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:32:53.679974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:53.679974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:32:53.679974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:53.679974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:32:53.679974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:32:53.680827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:53.680827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:32:53.680827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:53.680827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:32:53.680827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:53.680827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:32:53.680827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:32:53.680827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:32:53.680827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:32:53.680827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:32:53.680827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:32:53.680827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:32:53.680827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:32:53.680827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:32:53.681800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:32:53.681800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:32:53.681800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:32:53.681800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:32:53.681800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:32:53.681800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:32:53.681800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:32:53.681800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:32:53.681800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:32:53.681800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:32:53.681800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:32:53.681800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:53.681800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:32:53.681800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:53.681800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:32:53.681800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:53.681800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:32:53.681800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:53.681800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:32:53.683059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:53.683059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:32:53.683059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:32:53.683059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:53.683059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:32:53.683059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:53.683059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:32:53.683059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:32:53.683059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:53.683059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:32:53.683059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:53.683059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:32:53.686088 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:32:53.686453 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:32:53.686550 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:32:53.686687 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:32:53.687398 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:32:53.688139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:53.688654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:32:53.688654 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:53.688654 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:32:53.688654 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:32:53.688654 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:53.688654 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:32:53.688654 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:53.688654 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:32:53.689492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:53.689492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:32:53.689492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:32:53.689492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:32:53.689492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:53.689492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:32:53.689492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:53.689492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:32:53.689492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:53.689492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:32:53.689492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:53.689492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:32:53.689492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:53.689492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:32:53.689492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:32:53.689908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:53.689908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:32:53.689908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:53.689908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:32:53.689908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:32:53.689908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:53.689908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:32:53.689908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:53.689908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:32:53.690919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:53.690919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:32:53.690919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:32:53.690919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:32:53.690919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:32:53.690919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:32:53.690919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:32:53.690919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:32:53.690919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:32:53.690919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:53.690919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:32:53.691252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:53.691252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:32:53.691252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:53.691252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:32:53.691252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:53.691252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:32:53.691252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:53.691252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:32:53.691252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:32:53.691252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:53.691252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:32:53.691252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:53.691252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:32:53.691252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:32:53.691642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:53.691642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:32:53.691642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:53.691642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:32:53.692840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:53.692840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:32:53.692840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:32:53.692840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:32:53.692840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:32:53.692840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:32:53.692840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:32:53.692840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:32:53.692840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:32:53.692840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:32:53.692840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:32:53.692840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:32:53.692840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:32:53.692840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:32:53.692840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:32:53.692840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:32:53.692840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:32:53.693298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:32:53.693298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:32:53.693298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:32:53.693298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:53.693298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:32:53.693298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:53.693298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:32:53.693298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:53.693298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:32:53.693298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:53.693298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:32:53.693298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:53.693298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:32:53.693298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:32:53.693298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:53.693298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:32:53.693298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:53.693298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:32:53.693298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:32:53.693889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:53.693889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:32:53.693889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:53.693889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:32:53.694340 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:32:53.694633 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:32:53.694937 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:32:53.695173 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:32:53.695602 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:32:53.695984 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:32:53.696213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:53.697153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:32:53.697153 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:53.697153 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:32:53.697153 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:32:53.697153 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:53.697153 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:32:53.697153 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:53.697153 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:32:53.698579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:53.698579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:32:53.698579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:32:53.698579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:32:53.698579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:32:53.698579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:32:53.698579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:32:53.698579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:53.698579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:32:53.698579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:53.698579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:32:53.698579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:53.698922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:32:53.698922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:53.698922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:32:53.698922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:53.698922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:32:53.698922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:32:53.698922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:53.698922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:32:53.698922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:53.698922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:32:53.698922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:32:53.698922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:53.698922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:32:53.699351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:53.699351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:32:53.700343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:53.700343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:32:53.700343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:32:53.700343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:32:53.700343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:32:53.700343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:32:53.700343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:32:53.700343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:32:53.700343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:32:53.700343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:32:53.700343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:32:53.700343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:32:53.700343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:32:53.700343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:32:53.700343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:32:53.700343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:32:53.700781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:32:53.700781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:32:53.700781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:53.700781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:32:53.700781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:53.700781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:32:53.700781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:53.700781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:32:53.700781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:53.700781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:32:53.700781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:53.700781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:32:53.700781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:32:53.700781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:53.700781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:32:53.700781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:53.700781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:32:53.700781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:32:53.700781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:53.701262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:32:53.701262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:53.701262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:32:58.507955 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:32:58.507955 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:32:58.507955 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:32:58.507955 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:32:58.514778 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG dbcounter [-] [106314] Writing DB stats neutron:SELECT=239 {{(pid=106314) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 09 09:32:58.551995 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:32:58.552374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:58.554862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:32:58.554862 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:58.554862 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:32:58.554862 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:32:58.554862 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:58.554862 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:32:58.554862 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:58.554862 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:32:58.554862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.045 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:58.554862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:32:58.554862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:32:58.554862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:32:58.554862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:32:58.554862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:32:58.554862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:32:58.555266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:58.555266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:32:58.555266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:58.555266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:32:58.555266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:58.555266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:32:58.555266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:58.555266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:32:58.555266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:58.555266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:32:58.555266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:32:58.555266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:58.555266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:32:58.555266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:58.555572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:32:58.555572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:32:58.555572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:58.555572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:32:58.555572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:58.555572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:32:58.555572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:58.555572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:32:58.555572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:32:58.555572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:32:58.555572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:32:58.555572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:32:58.555572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:32:58.555572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:32:58.555909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:32:58.555909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:32:58.555909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:32:58.555909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:32:58.555909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:32:58.555909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:32:58.555909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:32:58.555909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:32:58.555909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:32:58.555909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:32:58.555909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:58.555909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:32:58.555909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:58.555909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:32:58.555909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:58.555909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:32:58.555909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:58.555909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:32:58.555909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:58.555909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:32:58.555909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:32:58.556396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:58.556396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:32:58.556396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:58.556396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:32:58.556396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:32:58.556396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:58.556396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:32:58.556396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:58.556396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:32:58.663082 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:32:58.664657 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:32:58.664657 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:32:58.665345 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:32:58.665527 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:32:58.665623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:58.668782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:32:58.668782 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:58.668782 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:32:58.668782 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:32:58.668782 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:58.668782 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:32:58.668782 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:58.668782 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:32:58.668782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:58.668782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:32:58.668782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:32:58.668782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:32:58.668782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:32:58.668782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:32:58.668782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:32:58.669625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:58.669625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:32:58.669625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:58.669625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:32:58.669625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:58.669625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:32:58.669625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:58.669625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:32:58.669625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:58.669625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:32:58.669625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:32:58.669625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:58.669625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:32:58.669625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:58.670318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:32:58.670318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:32:58.670318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:58.670318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:32:58.670318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:58.670318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:32:58.670869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:58.670869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:32:58.670869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:32:58.670869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:32:58.670869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:32:58.670869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:32:58.670869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:32:58.670869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:32:58.670869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:32:58.670869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:32:58.670869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:32:58.670869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:32:58.670869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:32:58.670869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:32:58.670869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:32:58.670869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:32:58.671581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:32:58.671581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:32:58.671581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:58.671581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:32:58.671581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:58.671581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:32:58.671581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:58.671581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:32:58.671581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:58.671581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:32:58.671581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:58.671581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:32:58.671581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:32:58.671581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:58.671581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:32:58.671581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:58.671581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:32:58.671581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:32:58.671581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:58.672555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:32:58.672555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:58.672555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:32:58.675296 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:32:58.675667 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:32:58.676002 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:32:58.676269 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:32:58.677035 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:32:58.677268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:58.677900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:32:58.677900 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:58.677900 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:32:58.677900 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:32:58.677900 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:58.677900 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:32:58.677900 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:58.677900 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:32:58.678937 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:58.678937 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:32:58.678937 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:32:58.678937 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:32:58.678937 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:58.678937 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:32:58.678937 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:58.678937 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:32:58.678937 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:58.678937 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:32:58.678937 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:58.678937 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:32:58.678937 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:58.678937 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:32:58.678937 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:32:58.679263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:58.679263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:32:58.679263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:58.679263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:32:58.679263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:32:58.679263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:58.679263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:32:58.679263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:58.679263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:32:58.680678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:58.680678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:32:58.680678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:32:58.680678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:32:58.680678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:32:58.680678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:32:58.680678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:32:58.680678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:32:58.680678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:32:58.680678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:32:58.680678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:58.680678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:32:58.680987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:58.680987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:32:58.680987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:58.680987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:32:58.680987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:58.680987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:32:58.680987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:58.680987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:32:58.680987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:32:58.680987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:58.680987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:32:58.680987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:58.680987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:32:58.680987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:32:58.681250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:58.681250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:32:58.681250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:58.681250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:32:58.682460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:58.682460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:32:58.682460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:32:58.682460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:32:58.682460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:32:58.682460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:32:58.682460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:32:58.682460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:32:58.682460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:32:58.682460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:32:58.682460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:32:58.682460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:32:58.682460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:32:58.682460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:32:58.682460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:32:58.682460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:32:58.683179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:32:58.683179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:32:58.683179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:32:58.683179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:32:58.683179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:32:58.683179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:58.683179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:32:58.683179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:58.683179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:32:58.683179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:58.683179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:32:58.683179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:58.683179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:32:58.683179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:58.683179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:32:58.683179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:32:58.683179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:58.683179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:32:58.684024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:58.684024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:32:58.684024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:32:58.684024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:58.684024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:32:58.684024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:58.684024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:32:58.686682 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:32:58.687011 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:32:58.687248 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:32:58.687823 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:32:58.688906 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:32:58.689169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:58.689840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:32:58.689840 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:58.689840 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:32:58.689840 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:32:58.689840 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:58.689840 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:32:58.689840 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:58.689840 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:32:58.690727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:58.690727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:32:58.690727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:32:58.690727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:32:58.690727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:58.690727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:32:58.690727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:58.690727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:32:58.690727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:58.690727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:32:58.690727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:58.690727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:32:58.690727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:58.690727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:32:58.690727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:32:58.691138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:58.691138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:32:58.691138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:58.691138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:32:58.691138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:32:58.691138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:58.691138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:32:58.691138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:58.691138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:32:58.691622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:58.691622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:32:58.691622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:32:58.691622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:32:58.691622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:32:58.691622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:32:58.691622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:32:58.691622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:32:58.691622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:32:58.691622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:58.691622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:32:58.692039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:58.692039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:32:58.692039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:58.692039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:32:58.692039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:58.692039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:32:58.692039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:58.692039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:32:58.692039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:32:58.692039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:58.692039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:32:58.692039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:58.692039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:32:58.692039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:32:58.692374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:58.692374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:32:58.692374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:58.692374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:32:58.692963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:58.692963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:32:58.692963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:32:58.692963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:32:58.692963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:32:58.692963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:32:58.692963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:32:58.692963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:32:58.692963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:32:58.692963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:32:58.692963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:32:58.692963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:32:58.692963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:32:58.692963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:32:58.692963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:32:58.692963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:32:58.692963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:32:58.693412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:32:58.693412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:32:58.693412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:32:58.693412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:58.693412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:32:58.693412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:58.693412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:32:58.693412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:58.693412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:32:58.693412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:58.693412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:32:58.693412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:58.693412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:32:58.693412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:32:58.693412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:58.693412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:32:58.693412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:58.693412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:32:58.693412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:32:58.693892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:58.693892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:32:58.693892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:58.693892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:32:58.694651 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:32:58.694997 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:32:58.695126 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:32:58.695314 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:32:58.695878 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:32:58.696175 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:32:58.696340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:58.697042 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:32:58.697042 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:58.697042 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:32:58.697042 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:32:58.697042 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:58.697042 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:32:58.697042 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:58.697042 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:32:58.698169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:58.698169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:32:58.698169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:32:58.698169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:32:58.698169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:32:58.698169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:32:58.698169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:32:58.698169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:58.698169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:32:58.698169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:58.698169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:32:58.698169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:58.698536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:32:58.698536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:58.698536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:32:58.698536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:58.698536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:32:58.698536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:32:58.698536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:58.698536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:32:58.698536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:58.698536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:32:58.698536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:32:58.698536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:58.698536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:32:58.698879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:58.698879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:32:58.699284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:58.699284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:32:58.699284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:32:58.699284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:32:58.699284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:32:58.699284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:32:58.699284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:32:58.699284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:32:58.699284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:32:58.699284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:32:58.699284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:32:58.699284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:32:58.699284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:32:58.699284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:32:58.699284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:32:58.699284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:32:58.699766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:32:58.699766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:32:58.699766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:58.699766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:32:58.699766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:32:58.699766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:32:58.699766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:32:58.699766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:32:58.699766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:32:58.699766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:32:58.699766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:32:58.699766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:32:58.699766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:32:58.699766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:32:58.699766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:32:58.699766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:32:58.699766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:32:58.699766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:32:58.699766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:32:58.700258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:32:58.700258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:32:58.700258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:33:03.508502 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:33:03.509249 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:33:03.509341 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:33:03.510013 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:33:03.545170 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:33:03.545514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:03.546397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:33:03.546397 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:03.546397 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:33:03.546397 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:33:03.546397 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:03.546397 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:33:03.546397 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:03.546397 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:33:03.547513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:03.547513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:33:03.547513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:33:03.547513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:33:03.547513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:33:03.547513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:33:03.547513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:33:03.547513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:03.547513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:33:03.547513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:03.547513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:33:03.547513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:03.548211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:33:03.548211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:03.548211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:33:03.548211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:03.548211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:33:03.548211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:33:03.548211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:03.548211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:33:03.548211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:03.548211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:33:03.548211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:33:03.548211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:03.548211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:33:03.548822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:03.548822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:33:03.548822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:03.548822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:33:03.548822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:33:03.548822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:33:03.548822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:33:03.548822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:33:03.548822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:33:03.548822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:33:03.548822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:33:03.548822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:33:03.548822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:33:03.548822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:33:03.548822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:33:03.549519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:33:03.549519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:33:03.549519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:33:03.549519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:33:03.549519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:33:03.549519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:03.549519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:33:03.549519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:03.549519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:33:03.549519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:03.549519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:33:03.549519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:03.549519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:33:03.549519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:03.549519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:33:03.549519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:33:03.549519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:03.549519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:33:03.549519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:03.550425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:33:03.550425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:33:03.550425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:03.550425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:33:03.550425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:03.550425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:33:03.664003 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:33:03.664583 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:33:03.665176 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:33:03.666077 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:33:03.666388 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:33:03.666559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:03.667477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:33:03.667477 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:03.667477 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:33:03.667477 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:33:03.667477 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:03.667477 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:33:03.667477 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:03.667477 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:33:03.669201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:03.669201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:33:03.669201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:33:03.669201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:33:03.669201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:33:03.669201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:33:03.669201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:33:03.669201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:03.669201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:33:03.669201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:03.669201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:33:03.669201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:03.669698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:33:03.669698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:03.669698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:33:03.669698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:03.669698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:33:03.669698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:33:03.669698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:03.669698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:33:03.669698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:03.669698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:33:03.669698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:33:03.669698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:03.669698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:33:03.670363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:03.670363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:33:03.670936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:03.670936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:33:03.670936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:33:03.670936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:33:03.670936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:33:03.670936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:33:03.670936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:33:03.670936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:33:03.670936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:33:03.670936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:33:03.670936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:33:03.670936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:33:03.670936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:33:03.670936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:33:03.670936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:33:03.670936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:33:03.671576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:33:03.671576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:33:03.671576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:03.671576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:33:03.671576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:03.671576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:33:03.671576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:03.671576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:33:03.671576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:03.671576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:33:03.671576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:03.671576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:33:03.671576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:33:03.671576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:03.671576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:33:03.671576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:03.671576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:33:03.671576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:33:03.671576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:03.672114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:33:03.672114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:03.672114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:33:03.675793 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:33:03.676144 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:33:03.676316 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:33:03.676541 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:33:03.677239 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:33:03.677609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:03.678451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:33:03.678451 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:03.678451 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:33:03.678451 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:33:03.678451 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:03.678451 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:33:03.678451 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:03.678451 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:33:03.679347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:03.679347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:33:03.679347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:33:03.679347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:33:03.679347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:03.679347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:33:03.679347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:03.679347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:33:03.679347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:03.679347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:33:03.679347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:03.679347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:33:03.679347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:03.679347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:33:03.679347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:33:03.679664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:03.679664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:33:03.679664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:03.679664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:33:03.679664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:33:03.679664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:03.679664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:33:03.679664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:03.679664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:33:03.680193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:03.680193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:33:03.680193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:33:03.680193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:33:03.680193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:33:03.680193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:33:03.680193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:33:03.680193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:33:03.680193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:33:03.680193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:33:03.680193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:03.680193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:33:03.680480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:03.680480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:33:03.680480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:03.680480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:33:03.680480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:03.680480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:33:03.680480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:03.680480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:33:03.680480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:33:03.680480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:03.680480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:33:03.680480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:03.680480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:33:03.680480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:33:03.680723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:03.680723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:33:03.680723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:03.680723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:33:03.680998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:03.680998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:33:03.680998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:33:03.680998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:33:03.680998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:33:03.680998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:33:03.680998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:33:03.680998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:33:03.680998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:33:03.680998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:33:03.680998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:33:03.680998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:33:03.680998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:33:03.680998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:33:03.680998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:33:03.680998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:33:03.681288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:33:03.681288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:33:03.681288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:33:03.681288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:33:03.681288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:33:03.681288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:03.681288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:33:03.681288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:03.681288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:33:03.681288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:03.681288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:33:03.681288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:03.681288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:33:03.681288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:03.681288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:33:03.681288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:33:03.681288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:03.681288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:33:03.681613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:03.681613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:33:03.681613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:33:03.681613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:03.681613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:33:03.681613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:03.681613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:33:03.687080 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:33:03.687424 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:33:03.687548 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:33:03.687686 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:33:03.688262 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:33:03.688558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:03.689405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:33:03.689405 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:03.689405 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:33:03.689405 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:33:03.689405 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:03.689405 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:33:03.689405 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:03.689405 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:33:03.690449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:03.690449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:33:03.690449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:33:03.690449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:33:03.690449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:03.690449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:33:03.690449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:03.690449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:33:03.690449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:03.690449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:33:03.690449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:03.690449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:33:03.690449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:03.690449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:33:03.690449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:33:03.691064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:03.691064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:33:03.691064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:03.691064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:33:03.691064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:33:03.691064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:03.691064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:33:03.691064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:03.691064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:33:03.691387 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:03.691387 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:33:03.691387 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:33:03.691387 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:33:03.691387 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:33:03.691387 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:33:03.691387 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:33:03.691387 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:33:03.691387 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:33:03.691387 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:03.691387 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:33:03.691691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:03.691691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:33:03.691691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:03.691691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:33:03.691691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:03.691691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:33:03.691691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:03.691691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:33:03.691691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:33:03.691691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:03.691691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:33:03.691691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:03.691691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:33:03.691691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:33:03.692071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:03.692071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:33:03.692071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:03.692071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:33:03.692698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:03.692698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:33:03.692698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:33:03.692698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:33:03.692698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:33:03.692698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:33:03.692698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:33:03.692698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:33:03.692698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:33:03.692698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:33:03.692698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:33:03.692698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:33:03.692698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:33:03.692698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:33:03.692698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:33:03.692698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:33:03.692698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:33:03.693186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:33:03.693186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:33:03.693186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:33:03.693186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:03.693186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:33:03.693186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:03.693186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:33:03.693186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:03.693186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:33:03.693186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:03.693186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:33:03.693186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:03.693186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:33:03.693186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:33:03.693186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:03.693186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:33:03.693186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:03.693186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:33:03.693186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:33:03.693866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:03.693866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:33:03.693866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:03.693866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:33:03.695799 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:33:03.695799 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:33:03.695799 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:33:03.695799 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:33:03.697374 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:33:03.697374 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:33:03.697374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:03.697664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:33:03.697664 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:03.697664 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:33:03.697664 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:33:03.697664 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:03.697664 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:33:03.697664 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:03.697664 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:33:03.699057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:03.699057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:33:03.699057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:33:03.699057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:33:03.699057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:33:03.699057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:33:03.699057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:33:03.699057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:03.699057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:33:03.699057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:03.699057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:33:03.699057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:03.699523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:33:03.699523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:03.699523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:33:03.699523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:03.699523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:33:03.699523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:33:03.699523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:03.699523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:33:03.699523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:03.699523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:33:03.699523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:33:03.699523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:03.699523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:33:03.699938 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:03.699938 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:33:03.700496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:03.700496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:33:03.700496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:33:03.700496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:33:03.700496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:33:03.700496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:33:03.700496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:33:03.700496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:33:03.700496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:33:03.700496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:33:03.700496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:33:03.700496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:33:03.700496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:33:03.700496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:33:03.700496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:33:03.700496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:33:03.701095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:33:03.701095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:33:03.701095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:03.701095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:33:03.701095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:03.701095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:33:03.701095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:03.701095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:33:03.701095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:03.701095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:33:03.701095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:03.701095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:33:03.701095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:33:03.701095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:03.701095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:33:03.701095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:03.701095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:33:03.701095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:33:03.701095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:03.701619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:33:03.701619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:03.701619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:33:08.529854 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:33:08.530904 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:33:08.531667 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:33:08.533038 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:33:08.572152 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:33:08.572625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:08.573907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:33:08.573907 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:08.573907 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:33:08.573907 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:33:08.573907 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:08.573907 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:33:08.573907 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:08.573907 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:33:08.575141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.043 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:08.575141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:33:08.575141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:33:08.575141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:33:08.575141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:33:08.575141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:33:08.575141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:33:08.575141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:08.575141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:33:08.575141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:08.575141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:33:08.575141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:08.575568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:33:08.575568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:08.575568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:33:08.575568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:08.575568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:33:08.575568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:33:08.575568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:08.575568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:33:08.575568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:08.575568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:33:08.575568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:33:08.575568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:08.575568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:33:08.576089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:08.576089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:33:08.576302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:08.576302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:33:08.576302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:33:08.576302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:33:08.576302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:33:08.576302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:33:08.576302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:33:08.576302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:33:08.576302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:33:08.576302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:33:08.576302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:33:08.576302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:33:08.576302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:33:08.576302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:33:08.576302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:33:08.576302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:33:08.576936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:33:08.576936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:33:08.576936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:08.576936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:33:08.576936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:08.576936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:33:08.576936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:08.576936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:33:08.576936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:08.576936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:33:08.576936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:08.576936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:33:08.576936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:33:08.576936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:08.576936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:33:08.576936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:08.576936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:33:08.576936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:33:08.576936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:08.578055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:33:08.578055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:08.578055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:33:08.665007 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:33:08.665841 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:33:08.666446 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:33:08.668152 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:33:08.668837 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:33:08.669084 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:08.670094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:33:08.670094 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:08.670094 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:33:08.670094 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:33:08.670094 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:08.670094 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:33:08.670094 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:08.670094 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:33:08.672538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:08.672538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:33:08.672538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:33:08.672538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:33:08.672538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:33:08.672538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:33:08.672538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:33:08.672538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:08.672538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:33:08.672538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:08.672538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:33:08.672538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:08.673450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:33:08.673450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:08.673450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:33:08.673450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:08.673450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:33:08.673450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:33:08.673450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:08.673450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:33:08.673450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:08.673450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:33:08.673450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:33:08.673450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:08.673450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:33:08.674047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:08.674047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:33:08.674333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:08.674333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:33:08.674333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:33:08.674333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:33:08.674333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:33:08.674333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:33:08.674333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:33:08.674333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:33:08.674333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:33:08.674333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:33:08.674333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:33:08.674333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:33:08.674333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:33:08.674333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:33:08.674333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:33:08.674333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:33:08.675002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:33:08.675002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:33:08.675002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:08.675002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:33:08.675002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:08.675002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:33:08.675002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:08.675002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:33:08.675002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:08.675002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:33:08.675002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:08.675002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:33:08.675002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:33:08.675002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:08.675002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:33:08.675002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:08.675002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:33:08.675002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:33:08.675002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:08.675600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:33:08.675600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:08.675600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:33:08.676351 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:33:08.676833 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:33:08.677062 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:33:08.677169 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:33:08.677737 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:33:08.677985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:08.678608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:33:08.678608 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:08.678608 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:33:08.678608 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:33:08.678608 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:08.678608 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:33:08.678608 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:08.678608 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:33:08.679638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:08.679638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:33:08.679638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:33:08.679638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:33:08.679638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:08.679638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:33:08.679638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:08.679638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:33:08.679638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:08.679638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:33:08.679638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:08.679638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:33:08.679638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:08.679638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:33:08.679638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:33:08.680359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:08.680359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:33:08.680359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:08.680359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:33:08.680359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:33:08.680359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:08.680359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:33:08.680359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:08.680359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:33:08.680868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:08.680868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:33:08.680868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:33:08.680868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:33:08.680868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:33:08.680868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:33:08.680868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:33:08.680868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:33:08.680868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:33:08.680868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:33:08.680868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:08.680868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:33:08.681542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:08.681542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:33:08.681542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:08.681542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:33:08.681542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:08.681542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:33:08.681542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:08.681542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:33:08.681542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:33:08.681542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:08.681542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:33:08.681542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:08.681542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:33:08.681542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:33:08.682038 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:08.682038 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:33:08.682038 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:08.682038 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:33:08.682308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:08.682308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:33:08.682308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:33:08.682308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:33:08.682308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:33:08.682308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:33:08.682308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:33:08.682308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:33:08.682308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:33:08.682308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:33:08.682308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:33:08.682308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:33:08.682308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:33:08.682308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:33:08.682308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:33:08.682308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:33:08.682823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:33:08.682823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:33:08.682823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:33:08.682823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:33:08.682823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:33:08.682823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:08.682823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:33:08.682823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:08.682823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:33:08.682823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:08.682823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:33:08.682823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:08.682823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:33:08.682823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:08.682823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:33:08.682823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:33:08.682823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:08.682823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:33:08.683381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:08.683381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:33:08.683381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:33:08.683381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:08.683381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:33:08.683381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:08.683381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:33:08.687824 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:33:08.688856 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:33:08.689191 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:33:08.689543 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:33:08.690255 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:33:08.690625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:08.691442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:33:08.691442 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:08.691442 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:33:08.691442 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:33:08.691442 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:08.691442 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:33:08.691442 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:08.691442 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:33:08.692504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:08.692504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:33:08.692504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:33:08.692504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:33:08.692504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:08.692504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:33:08.692504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:08.692504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:33:08.692504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:08.692504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:33:08.692504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:08.692504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:33:08.692504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:08.692504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:33:08.692504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:33:08.693061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:08.693061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:33:08.693061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:08.693061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:33:08.693061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:33:08.693061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:08.693061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:33:08.693061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:08.693061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:33:08.693511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:08.693511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:33:08.693511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:33:08.693511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:33:08.693511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:33:08.693511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:33:08.693511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:33:08.693511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:33:08.693511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:33:08.693511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:08.693511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:33:08.693911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:08.693911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:33:08.693911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:08.693911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:33:08.693911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:08.693911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:33:08.693911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:08.693911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:33:08.693911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:33:08.693911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:08.693911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:33:08.693911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:08.693911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:33:08.693911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:33:08.694314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:08.694314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:33:08.694314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:08.694314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:33:08.694897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:08.694897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:33:08.694897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:33:08.694897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:33:08.694897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:33:08.694897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:33:08.694897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:33:08.694897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:33:08.694897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:33:08.694897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:33:08.694897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:33:08.694897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:33:08.694897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:33:08.694897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:33:08.694897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:33:08.694897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:33:08.694897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:33:08.695422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:33:08.695422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:33:08.695422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:33:08.695422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:08.695422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:33:08.695422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:08.695422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:33:08.695422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:08.695422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:33:08.695422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:08.695422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:33:08.695422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:08.695422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:33:08.695422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:33:08.695422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:08.695422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:33:08.695422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:08.695422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:33:08.695422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:33:08.696040 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:08.696040 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:33:08.696040 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:08.696040 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:33:08.696040 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:33:08.696225 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:33:08.696391 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:33:08.696578 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:33:08.697029 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:33:08.697274 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:33:08.697465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:08.698073 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:33:08.698073 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:08.698073 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:33:08.698073 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:33:08.698073 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:08.698073 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:33:08.698073 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:08.698073 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:33:08.699025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:08.699025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:33:08.699025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:33:08.699025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:33:08.699025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:33:08.699025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:33:08.699025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:33:08.699025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:08.699025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:33:08.699025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:08.699025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:33:08.699025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:08.699385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:33:08.699385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:08.699385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:33:08.699385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:08.699385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:33:08.699385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:33:08.699385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:08.699385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:33:08.699385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:08.699385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:33:08.699385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:33:08.699385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:08.699385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:33:08.699792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:08.699792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:33:08.700267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:08.700267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:33:08.700267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:33:08.700267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:33:08.700267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:33:08.700267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:33:08.700267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:33:08.700267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:33:08.700267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:33:08.700267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:33:08.700267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:33:08.700267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:33:08.700267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:33:08.700267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:33:08.700267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:33:08.700267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:33:08.700712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:33:08.700712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:33:08.700712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:08.700712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:33:08.700712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:08.700712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:33:08.700712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:08.700712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:33:08.700712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:08.700712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:33:08.700712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:08.700712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:33:08.700712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:33:08.700712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:08.700712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:33:08.700712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:08.700712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:33:08.700712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:33:08.700712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:08.701429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:33:08.701429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:08.701429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:33:13.530921 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:33:13.531970 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:33:13.531970 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:33:13.532906 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:33:13.572588 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:33:13.573050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:13.574010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:33:13.574010 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:13.574010 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:33:13.574010 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:33:13.574010 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:13.574010 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:33:13.574010 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:13.574010 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:33:13.574875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.043 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:13.574875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:33:13.574875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:33:13.574875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:33:13.574875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:33:13.574875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:33:13.574875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:33:13.574875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:13.574875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:33:13.574875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:13.574875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:33:13.574875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:13.575573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:33:13.575573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:13.575573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:33:13.575573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:13.575573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:33:13.575573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:33:13.575573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:13.575573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:33:13.575573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:13.575573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:33:13.575573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:33:13.575573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:13.575573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:33:13.576701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:13.576701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:33:13.576701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:13.576701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:33:13.576701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:33:13.576701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:33:13.576701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:33:13.576701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:33:13.576701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:33:13.576701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:33:13.576701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:33:13.576701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:33:13.576701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:33:13.576701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:33:13.576701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:33:13.577467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:33:13.577467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:33:13.577467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:33:13.577467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:33:13.577467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:33:13.577467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:13.577467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:33:13.577467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:13.577467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:33:13.577467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:13.577467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:33:13.577467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:13.577467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:33:13.577467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:13.577467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:33:13.577467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:33:13.577467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:13.577467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:33:13.577467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:13.578598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:33:13.578598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:33:13.578598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:13.578598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:33:13.578598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:13.578598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:33:13.666542 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:33:13.667210 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:33:13.667409 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:33:13.668291 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:33:13.668419 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:33:13.668567 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:13.669224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:33:13.669224 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:13.669224 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:33:13.669224 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:33:13.669224 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:13.669224 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:33:13.669224 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:13.669224 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:33:13.671103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:13.671103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:33:13.671103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:33:13.671103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:33:13.671103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:33:13.671103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:33:13.671103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:33:13.671103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:13.671103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:33:13.671103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:13.671103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:33:13.671103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:13.671518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:33:13.671518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:13.671518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:33:13.671518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:13.671518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:33:13.671518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:33:13.671518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:13.671518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:33:13.671518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:13.671518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:33:13.671518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:33:13.671518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:13.671518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:33:13.671912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:13.671912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:33:13.672602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:13.672602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:33:13.672602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:33:13.672602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:33:13.672602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:33:13.672602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:33:13.672602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:33:13.672602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:33:13.672602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:33:13.672602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:33:13.672602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:33:13.672602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:33:13.672602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:33:13.672602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:33:13.672602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:33:13.672602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:33:13.673289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:33:13.673289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:33:13.673289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:13.673289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:33:13.673289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:13.673289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:33:13.673289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:13.673289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:33:13.673289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:13.673289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:33:13.673289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:13.673289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:33:13.673289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:33:13.673289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:13.673289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:33:13.673289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:13.673289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:33:13.673289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:33:13.673289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:13.673846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:33:13.673846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:13.673846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:33:13.676920 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:33:13.677257 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:33:13.677320 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:33:13.677449 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:33:13.677756 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:33:13.678012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:13.678712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:33:13.678712 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:13.678712 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:33:13.678712 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:33:13.678712 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:13.678712 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:33:13.678712 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:13.678712 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:33:13.679465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:13.679465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:33:13.679465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:33:13.679465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:33:13.679465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:13.679465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:33:13.679465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:13.679465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:33:13.679465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:13.679465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:33:13.679465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:13.679465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:33:13.679465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:13.679465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:33:13.679465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:33:13.680085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:13.680085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:33:13.680085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:13.680085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:33:13.680085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:33:13.680085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:13.680085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:33:13.680085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:13.680085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:33:13.680376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:13.680376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:33:13.680376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:33:13.680376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:33:13.680376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:33:13.680376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:33:13.680376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:33:13.680376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:33:13.680376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:33:13.680376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:33:13.680376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:13.680376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:33:13.680728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:13.680728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:33:13.680728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:13.680728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:33:13.680728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:13.680728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:33:13.680728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:13.680728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:33:13.680728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:33:13.680728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:13.680728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:33:13.680728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:13.680728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:33:13.680728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:33:13.681176 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:13.681176 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:33:13.681176 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:13.681176 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:33:13.681410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:13.681410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:33:13.681410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:33:13.681410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:33:13.681410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:33:13.681410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:33:13.681410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:33:13.681410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:33:13.681410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:33:13.681410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:33:13.681410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:33:13.681410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:33:13.681410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:33:13.681410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:33:13.681410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:33:13.681410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:33:13.681928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:33:13.681928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:33:13.681928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:33:13.681928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:33:13.681928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:33:13.681928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:13.681928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:33:13.681928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:13.681928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:33:13.681928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:13.681928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:33:13.681928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:13.681928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:33:13.681928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:13.681928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:33:13.681928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:33:13.681928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:13.681928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:33:13.682448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:13.682448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:33:13.682448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:33:13.682448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:13.682448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:33:13.682448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:13.682448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:33:13.688515 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:33:13.688820 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:33:13.688995 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:33:13.689155 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:33:13.689581 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:33:13.690018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:13.690731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:33:13.690731 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:13.690731 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:33:13.690731 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:33:13.690731 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:13.690731 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:33:13.690731 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:13.690731 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:33:13.691967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:13.691967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:33:13.691967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:33:13.691967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:33:13.691967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:13.691967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:33:13.691967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:13.691967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:33:13.691967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:13.691967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:33:13.691967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:13.691967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:33:13.691967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:13.691967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:33:13.691967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:33:13.692597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:13.692597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:33:13.692597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:13.692597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:33:13.692597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:33:13.692597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:13.692597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:33:13.692597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:13.692597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:33:13.693541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:13.693541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:33:13.693541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:33:13.693541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:33:13.693541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:33:13.693541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:33:13.693541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:33:13.693541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:33:13.693541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:33:13.693541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:13.693541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:33:13.694097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:13.694097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:33:13.694097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:13.694097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:33:13.694097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:13.694097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:33:13.694097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:13.694097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:33:13.694097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:33:13.694097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:13.694097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:33:13.694097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:13.694097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:33:13.694097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:33:13.694803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:13.694803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:33:13.694803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:13.694803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:33:13.695240 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:13.695240 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:33:13.695240 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:33:13.695240 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:33:13.695240 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:33:13.695240 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:33:13.695240 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:33:13.695240 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:33:13.695240 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:33:13.695240 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:33:13.695240 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:33:13.695240 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:33:13.695240 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:33:13.695240 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:33:13.695240 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:33:13.695240 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:33:13.695240 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:33:13.696082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:33:13.696082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:33:13.696082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:33:13.696082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:13.696082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:33:13.696082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:13.696082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:33:13.696082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:13.696082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:33:13.696082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:13.696082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:33:13.696082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:13.696082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:33:13.696082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:33:13.696082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:13.696082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:33:13.696082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:13.696082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:33:13.696082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:33:13.697009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:13.697009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:33:13.697009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:13.697009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:33:13.697009 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:33:13.697293 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:33:13.697458 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:33:13.697783 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:33:13.698277 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:33:13.698769 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:33:13.699123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:13.699934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:33:13.699934 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:13.699934 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:33:13.699934 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:33:13.699934 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:13.699934 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:33:13.699934 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:13.699934 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:33:13.701041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:13.701041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:33:13.701041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:33:13.701041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:33:13.701041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:33:13.701041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:33:13.701041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:33:13.701041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:13.701041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:33:13.701041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:13.701041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:33:13.701041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:13.701384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:33:13.701384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:13.701384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:33:13.701384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:13.701384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:33:13.701384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:33:13.701384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:13.701384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:33:13.701384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:13.701384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:33:13.701384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:33:13.701384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:13.701384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:33:13.701716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:13.701716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:33:13.702212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:13.702212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:33:13.702212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:33:13.702212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:33:13.702212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:33:13.702212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:33:13.702212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:33:13.702212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:33:13.702212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:33:13.702212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:33:13.702212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:33:13.702212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:33:13.702212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:33:13.702212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:33:13.702212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:33:13.702212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:33:13.702634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:33:13.702634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:33:13.702634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:13.702634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:33:13.702634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:13.702634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:33:13.702634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:13.702634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:33:13.702634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:13.702634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:33:13.702634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:13.702634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:33:13.702634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:33:13.702634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:13.702634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:33:13.702634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:13.702634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:33:13.702634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:33:13.702634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:13.703152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:33:13.703152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:13.703152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:33:18.531727 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:33:18.532664 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:33:18.532664 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:33:18.533376 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:33:18.606858 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:33:18.607172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:18.608146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:33:18.608146 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:18.608146 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:33:18.608146 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:33:18.608146 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:18.608146 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:33:18.608146 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:18.608146 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:33:18.609527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.076 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:18.609527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:33:18.609527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:33:18.609527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:33:18.609527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:33:18.609527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:33:18.609527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:33:18.609527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:18.609527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:33:18.609527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:18.609527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:33:18.609527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:18.610989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:33:18.610989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:18.610989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:33:18.610989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:18.610989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:33:18.610989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:33:18.610989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:18.610989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:33:18.610989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:18.610989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:33:18.610989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:33:18.610989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:18.610989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:33:18.611704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:18.611704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:33:18.611704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:18.611704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:33:18.611704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:33:18.611704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:33:18.611704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:33:18.611704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:33:18.611704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:33:18.611704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:33:18.611704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:33:18.611704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:33:18.611704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:33:18.611704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:33:18.611704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:33:18.612804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:33:18.612804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:33:18.612804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:33:18.612804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:33:18.612804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:33:18.612804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:18.612804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:33:18.612804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:18.612804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:33:18.612804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:18.612804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:33:18.612804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:18.612804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:33:18.612804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:18.612804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:33:18.612804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:33:18.612804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:18.612804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:33:18.612804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:18.614023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:33:18.614023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:33:18.614023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:18.614023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:33:18.614023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:18.614023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:33:18.667461 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:33:18.669522 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:33:18.669522 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:33:18.670263 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:33:18.670582 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:33:18.670814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:18.671788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:33:18.671788 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:18.671788 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:33:18.671788 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:33:18.671788 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:18.671788 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:33:18.671788 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:18.671788 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:33:18.673652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:18.673652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:33:18.673652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:33:18.673652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:33:18.673652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:33:18.673652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:33:18.673652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:33:18.673652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:18.673652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:33:18.673652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:18.673652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:33:18.673652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:18.674180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:33:18.674180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:18.674180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:33:18.674180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:18.674180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:33:18.674180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:33:18.674180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:18.674180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:33:18.674180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:18.674180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:33:18.674180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:33:18.674180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:18.674180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:33:18.674545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:18.674545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:33:18.675559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:18.675559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:33:18.675559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:33:18.675559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:33:18.675559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:33:18.675559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:33:18.675559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:33:18.675559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:33:18.675559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:33:18.675559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:33:18.675559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:33:18.675559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:33:18.675559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:33:18.675559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:33:18.675559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:33:18.675559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:33:18.675952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:33:18.675952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:33:18.675952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:18.675952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:33:18.675952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:18.675952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:33:18.675952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:18.675952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:33:18.675952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:18.675952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:33:18.675952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:18.675952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:33:18.675952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:33:18.675952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:18.675952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:33:18.675952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:18.675952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:33:18.675952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:33:18.675952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:18.676327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:33:18.676327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:18.676327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:33:18.677407 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:33:18.677736 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:33:18.677871 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:33:18.677994 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:33:18.678452 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:33:18.678730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:18.679712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:33:18.679712 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:18.679712 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:33:18.679712 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:33:18.679712 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:18.679712 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:33:18.679712 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:18.679712 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:33:18.680473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:18.680473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:33:18.680473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:33:18.680473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:33:18.680473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:18.680473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:33:18.680473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:18.680473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:33:18.680473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:18.680473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:33:18.680473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:18.680473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:33:18.680473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:18.680473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:33:18.680473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:33:18.680886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:18.680886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:33:18.680886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:18.680886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:33:18.680886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:33:18.680886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:18.680886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:33:18.680886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:18.680886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:33:18.681380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:18.681380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:33:18.681380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:33:18.681380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:33:18.681380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:33:18.681380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:33:18.681380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:33:18.681380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:33:18.681380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:33:18.681380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:33:18.681380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:18.681380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:33:18.682016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:18.682016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:33:18.682016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:18.682016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:33:18.682016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:18.682016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:33:18.682016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:18.682016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:33:18.682016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:33:18.682016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:18.682016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:33:18.682016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:18.682016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:33:18.682016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:33:18.682641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:18.682641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:33:18.682641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:18.682641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:33:18.682641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:18.682641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:33:18.682641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:33:18.682641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:33:18.682641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:33:18.682641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:33:18.682641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:33:18.682641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:33:18.682641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:33:18.682641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:33:18.683294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:33:18.683294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:33:18.683294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:33:18.683294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:33:18.683294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:33:18.683294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:33:18.683294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:33:18.683294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:33:18.683294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:33:18.683294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:33:18.683294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:33:18.683294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:18.683294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:33:18.683294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:18.683294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:33:18.683294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:18.683294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:33:18.683294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:18.683294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:33:18.684194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:18.684194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:33:18.684194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:33:18.684194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:18.684194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:33:18.684194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:18.684194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:33:18.684194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:33:18.684194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:18.684194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:33:18.684194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:18.684194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:33:18.689153 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:33:18.689435 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:33:18.689579 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:33:18.689716 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:33:18.690511 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:33:18.691028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:18.692072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:33:18.692072 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:18.692072 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:33:18.692072 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:33:18.692072 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:18.692072 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:33:18.692072 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:18.692072 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:33:18.692986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:18.692986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:33:18.692986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:33:18.692986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:33:18.692986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:18.692986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:33:18.692986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:18.692986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:33:18.692986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:18.692986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:33:18.692986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:18.692986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:33:18.692986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:18.692986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:33:18.692986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:33:18.693884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:18.693884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:33:18.693884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:18.693884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:33:18.693884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:33:18.693884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:18.693884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:33:18.693884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:18.693884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:33:18.694277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:18.694277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:33:18.694277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:33:18.694277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:33:18.694277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:33:18.694277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:33:18.694277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:33:18.694277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:33:18.694277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:33:18.694277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:18.694277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:33:18.694795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:18.694795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:33:18.694795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:18.694795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:33:18.694795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:18.694795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:33:18.694795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:18.694795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:33:18.694795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:33:18.694795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:18.694795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:33:18.694795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:18.694795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:33:18.694795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:33:18.695578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:18.695578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:33:18.695578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:18.695578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:33:18.695578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:18.695578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:33:18.695578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:33:18.695578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:33:18.695578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:33:18.695578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:33:18.695578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:33:18.695578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:33:18.695578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:33:18.695578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:33:18.696433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:33:18.696433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:33:18.696433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:33:18.696433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:33:18.696433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:33:18.696433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:33:18.696433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:33:18.696433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:33:18.696433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:33:18.696433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:33:18.696433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:18.696433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:33:18.696433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:18.696433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:33:18.696433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:18.696433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:33:18.696433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:18.696433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:33:18.696433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:18.697282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:33:18.697282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:33:18.697282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:18.697282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:33:18.697282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:18.697282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:33:18.697282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:33:18.697282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:18.697282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:33:18.697282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:18.697282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:33:18.697282 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:33:18.697668 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:33:18.697706 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:33:18.697903 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:33:18.698339 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:33:18.698607 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:33:18.698811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:18.699455 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:33:18.699455 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:18.699455 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:33:18.699455 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:33:18.699455 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:18.699455 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:33:18.699455 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:18.699455 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:33:18.700334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:18.700334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:33:18.700334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:33:18.700334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:33:18.700334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:33:18.700334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:33:18.700334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:33:18.700334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:18.700334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:33:18.700334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:18.700334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:33:18.700334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:18.700802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:33:18.700802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:18.700802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:33:18.700802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:18.700802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:33:18.700802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:33:18.700802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:18.700802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:33:18.700802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:18.700802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:33:18.700802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:33:18.700802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:18.700802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:33:18.701147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:18.701147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:33:18.701345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:18.701345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:33:18.701345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:33:18.701345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:33:18.701345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:33:18.701345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:33:18.701345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:33:18.701345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:33:18.701345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:33:18.701345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:33:18.701345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:33:18.701345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:33:18.701345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:33:18.701345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:33:18.701345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:33:18.701345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:33:18.701846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:33:18.701846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:33:18.701846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:18.701846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:33:18.701846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:18.701846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:33:18.701846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:18.701846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:33:18.701846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:18.701846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:33:18.701846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:18.701846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:33:18.701846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:33:18.701846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:18.701846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:33:18.701846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:18.701846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:33:18.701846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:33:18.701846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:18.702391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:33:18.702391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:18.702391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:33:23.532377 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:33:23.533345 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:33:23.533345 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:33:23.533945 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:33:23.573788 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:33:23.574070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:23.574896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:33:23.574896 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:23.574896 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:33:23.574896 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:33:23.574896 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:23.574896 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:33:23.574896 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:23.574896 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:33:23.575965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:23.575965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:33:23.575965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:33:23.575965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:33:23.575965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:33:23.575965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:33:23.575965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:33:23.575965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:23.575965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:33:23.575965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:23.575965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:33:23.575965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:23.576655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:33:23.576655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:23.576655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:33:23.576655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:23.576655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:33:23.576655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:33:23.576655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:23.576655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:33:23.576655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:23.576655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:33:23.576655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:33:23.576655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:23.576655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:33:23.577411 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:23.577411 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:33:23.577411 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:23.577411 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:33:23.577411 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:33:23.577411 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:33:23.577411 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:33:23.577411 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:33:23.577411 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:33:23.577411 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:33:23.577411 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:33:23.577411 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:33:23.577411 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:33:23.577411 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:33:23.577411 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:33:23.578213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:33:23.578213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:33:23.578213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:33:23.578213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:33:23.578213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:33:23.578213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:23.578213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:33:23.578213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:23.578213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:33:23.578213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:23.578213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:33:23.578213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:23.578213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:33:23.578213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:23.578213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:33:23.578213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:33:23.578213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:23.578213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:33:23.578213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:23.579119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:33:23.579119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:33:23.579119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:23.579119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:33:23.579119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:23.579119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:33:23.668101 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:33:23.668621 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:33:23.668784 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:33:23.669784 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:33:23.670161 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:33:23.670476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:23.671405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:33:23.671405 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:23.671405 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:33:23.671405 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:33:23.671405 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:23.671405 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:33:23.671405 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:23.671405 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:33:23.672470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:23.672470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:33:23.672470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:33:23.672470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:33:23.672470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:33:23.672470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:33:23.672470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:33:23.672470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:23.672470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:33:23.672470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:23.672470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:33:23.672470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:23.673214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:33:23.673214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:23.673214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:33:23.673214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:23.673214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:33:23.673214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:33:23.673214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:23.673214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:33:23.673214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:23.673214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:33:23.673214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:33:23.673214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:23.673214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:33:23.673778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:23.673778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:33:23.673778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:23.673778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:33:23.673778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:33:23.673778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:33:23.673778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:33:23.673778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:33:23.673778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:33:23.673778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:33:23.673778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:33:23.673778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:33:23.673778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:33:23.673778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:33:23.673778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:33:23.674414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:33:23.674414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:33:23.674414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:33:23.674414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:33:23.674414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:33:23.674414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:23.674414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:33:23.674414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:23.674414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:33:23.674414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:23.674414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:33:23.674414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:23.674414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:33:23.674414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:23.674414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:33:23.674414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:33:23.674414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:23.674414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:33:23.674414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:23.675203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:33:23.675203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:33:23.675203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:23.675203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:33:23.675203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:23.675203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:33:23.677875 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:33:23.678240 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:33:23.678333 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:33:23.678485 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:33:23.678928 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:33:23.679121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:23.679694 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:33:23.679694 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:23.679694 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:33:23.679694 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:33:23.679694 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:23.679694 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:33:23.679694 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:23.679694 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:33:23.680638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:23.680638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:33:23.680638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:33:23.680638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:33:23.680638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:23.680638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:33:23.680638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:23.680638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:33:23.680638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:23.680638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:33:23.680638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:23.680638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:33:23.680638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:23.680638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:33:23.680638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:33:23.681362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:23.681362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:33:23.681362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:23.681362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:33:23.681362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:33:23.681362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:23.681362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:33:23.681362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:23.681362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:33:23.681699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:23.681699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:33:23.681699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:33:23.681699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:33:23.681699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:33:23.681699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:33:23.681699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:33:23.681699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:33:23.681699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:33:23.681699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:33:23.681699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:23.681699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:33:23.682123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:23.682123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:33:23.682123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:23.682123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:33:23.682123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:23.682123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:33:23.682123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:23.682123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:33:23.682123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:33:23.682123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:23.682123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:33:23.682123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:23.682123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:33:23.682123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:33:23.682560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:23.682560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:33:23.682560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:23.682560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:33:23.682560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:23.682560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:33:23.682560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:33:23.682560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:33:23.682560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:33:23.682560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:33:23.682560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:33:23.682560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:33:23.682560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:33:23.682560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:33:23.683167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:33:23.683167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:33:23.683167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:33:23.683167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:33:23.683167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:33:23.683167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:33:23.683167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:33:23.683167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:33:23.683167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:33:23.683167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:33:23.683167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:33:23.683167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:23.683167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:33:23.683167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:23.683167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:33:23.683167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:23.683167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:33:23.683167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:23.683167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:33:23.683848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:23.683848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:33:23.683848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:33:23.683848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:23.683848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:33:23.683848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:23.683848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:33:23.683848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:33:23.683848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:23.683848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:33:23.683848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:23.683848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:33:23.689594 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:33:23.689969 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:33:23.690081 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:33:23.690219 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:33:23.690707 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:33:23.691090 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:23.691821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:33:23.691821 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:23.691821 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:33:23.691821 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:33:23.691821 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:23.691821 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:33:23.691821 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:23.691821 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:33:23.692668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:23.692668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:33:23.692668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:33:23.692668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:33:23.692668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:23.692668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:33:23.692668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:23.692668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:33:23.692668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:23.692668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:33:23.692668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:23.692668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:33:23.692668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:23.692668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:33:23.692668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:33:23.693147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:23.693147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:33:23.693147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:23.693147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:33:23.693147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:33:23.693147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:23.693147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:33:23.693147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:23.693147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:33:23.693586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:23.693586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:33:23.693586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:33:23.693586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:33:23.693586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:33:23.693586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:33:23.693586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:33:23.693586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:33:23.693586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:33:23.693586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:23.693586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:33:23.693973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:23.693973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:33:23.693973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:23.693973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:33:23.693973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:23.693973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:33:23.693973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:23.693973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:33:23.693973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:33:23.693973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:23.693973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:33:23.693973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:23.693973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:33:23.693973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:33:23.694435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:23.694435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:33:23.694435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:23.694435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:33:23.694726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:23.694726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:33:23.694726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:33:23.694726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:33:23.694726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:33:23.694726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:33:23.694726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:33:23.694726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:33:23.694726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:33:23.694726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:33:23.694726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:33:23.694726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:33:23.694726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:33:23.694726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:33:23.694726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:33:23.694726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:33:23.694726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:33:23.695561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:33:23.695561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:33:23.695561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:33:23.695561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:23.695561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:33:23.695561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:23.695561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:33:23.695561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:23.695561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:33:23.695561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:23.695561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:33:23.695561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:23.695561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:33:23.695561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:33:23.695561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:23.695561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:33:23.695561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:23.695561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:33:23.695561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:33:23.696473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:23.696473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:33:23.696473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:23.696473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:33:23.697790 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:33:23.698137 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:33:23.698251 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:33:23.698386 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:33:23.698696 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:33:23.699030 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:33:23.699227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:23.699919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:33:23.699919 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:23.699919 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:33:23.699919 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:33:23.699919 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:23.699919 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:33:23.699919 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:23.699919 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:33:23.700761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:23.700761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:33:23.700761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:33:23.700761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:33:23.700761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:33:23.700761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:33:23.700761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:33:23.700761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:23.700761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:33:23.700761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:23.700761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:33:23.700761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:23.701407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:33:23.701407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:23.701407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:33:23.701407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:23.701407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:33:23.701407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:33:23.701407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:23.701407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:33:23.701407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:23.701407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:33:23.701407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:33:23.701407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:23.701407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:33:23.702010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:23.702010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:33:23.702010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:23.702010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:33:23.702010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:33:23.702010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:33:23.702010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:33:23.702010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:33:23.702010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:33:23.702010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:33:23.702010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:33:23.702010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:33:23.702010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:33:23.702010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:33:23.702010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:33:23.702677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:33:23.702677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:33:23.702677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:33:23.702677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:33:23.702677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:33:23.702677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:23.702677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:33:23.702677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:23.702677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:33:23.702677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:23.702677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:33:23.702677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:23.702677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:33:23.702677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:23.702677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:33:23.702677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:33:23.702677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:23.702677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:33:23.702677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:23.703544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:33:23.703544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:33:23.703544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:23.703544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:33:23.703544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:23.703544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:33:28.533025 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:33:28.533700 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:33:28.533835 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:33:28.535123 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:33:28.577307 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:33:28.577307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:28.578064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:33:28.578064 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:28.578064 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:33:28.578064 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:33:28.578064 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:28.578064 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:33:28.578064 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:28.578064 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:33:28.578790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.044 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:28.578790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:33:28.578790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:33:28.578790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:33:28.578790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:33:28.578790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:33:28.578790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:33:28.578790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:28.578790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:33:28.578790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:28.578790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:33:28.578790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:28.579194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:33:28.579194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:28.579194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:33:28.579194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:28.579194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:33:28.579194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:33:28.579194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:28.579194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:33:28.579194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:28.579194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:33:28.579194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:33:28.579194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:28.579194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:33:28.579579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:28.579579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:33:28.579579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:28.579579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:33:28.579579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:33:28.579579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:33:28.579579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:33:28.579579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:33:28.579579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:33:28.579579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:33:28.579579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:33:28.579579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:33:28.579579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:33:28.579579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:33:28.579579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:33:28.580995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:33:28.580995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:33:28.580995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:33:28.580995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:33:28.580995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:33:28.580995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:28.580995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:33:28.580995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:28.580995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:33:28.580995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:28.580995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:33:28.580995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:28.580995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:33:28.580995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:28.580995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:33:28.580995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:33:28.580995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:28.580995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:33:28.580995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:28.581413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:33:28.581413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:33:28.581413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:28.581413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:33:28.581413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:28.581413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:33:28.668730 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:33:28.669337 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:33:28.669526 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:33:28.670916 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:33:28.670916 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:33:28.671089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:28.671926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:33:28.671926 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:28.671926 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:33:28.671926 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:33:28.671926 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:28.671926 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:33:28.671926 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:28.671926 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:33:28.673505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:28.673505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:33:28.673505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:33:28.673505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:33:28.673505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:33:28.673505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:33:28.673505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:33:28.673505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:28.673505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:33:28.673505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:28.673505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:33:28.673505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:28.674169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:33:28.674169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:28.674169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:33:28.674169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:28.674169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:33:28.674169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:33:28.674169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:28.674169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:33:28.674169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:28.674169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:33:28.674169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:33:28.674169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:28.674169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:33:28.674647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:28.674647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:33:28.678259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:28.678259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:33:28.678259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:33:28.678259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:33:28.678259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:33:28.678259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:33:28.678259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:33:28.678259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:33:28.678259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:33:28.678259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:33:28.678259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:33:28.678259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:33:28.678259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:33:28.678259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:33:28.678259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:33:28.678259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:33:28.678986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:33:28.678986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:33:28.678986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:28.678986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:33:28.678986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:28.678986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:33:28.678986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:28.678986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:33:28.678986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:28.678986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:33:28.678986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:28.678986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:33:28.678986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:33:28.678986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:28.678986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:33:28.678986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:28.678986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:33:28.678986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:33:28.678986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:28.679793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:33:28.679793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:28.679793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:33:28.679793 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:33:28.679793 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:33:28.679793 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:33:28.679793 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:33:28.680095 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:33:28.680405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:28.681319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:33:28.681319 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:28.681319 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:33:28.681319 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:33:28.681319 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:28.681319 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:33:28.681319 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:28.681319 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:33:28.682904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:28.682904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:33:28.682904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:33:28.682904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:33:28.682904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:28.682904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:33:28.682904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:28.682904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:33:28.682904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:28.682904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:33:28.682904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:28.682904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:33:28.682904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:28.682904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:33:28.682904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:33:28.683292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:28.683292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:33:28.683292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:28.683292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:33:28.683292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:33:28.683292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:28.683292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:33:28.683292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:28.683292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:33:28.684459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:28.684459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:33:28.684459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:33:28.684459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:33:28.684459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:33:28.684459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:33:28.684459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:33:28.684459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:33:28.684459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:33:28.684459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:33:28.684459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:28.684459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:33:28.684855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:28.684855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:33:28.684855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:28.684855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:33:28.684855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:28.684855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:33:28.684855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:28.684855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:33:28.684855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:33:28.684855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:28.684855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:33:28.684855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:28.684855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:33:28.684855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:33:28.685155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:28.685155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:33:28.685155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:28.685155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:33:28.685903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:28.685903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:33:28.685903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:33:28.685903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:33:28.685903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:33:28.685903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:33:28.685903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:33:28.685903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:33:28.685903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:33:28.685903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:33:28.685903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:33:28.685903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:33:28.685903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:33:28.685903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:33:28.685903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:33:28.685903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:33:28.686301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:33:28.686301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:33:28.686301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:33:28.686301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:33:28.686301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:33:28.686301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:28.686301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:33:28.686301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:28.686301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:33:28.686301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:28.686301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:33:28.686301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:28.686301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:33:28.686301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:28.686301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:33:28.686301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:33:28.686301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:28.686301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:33:28.686663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:28.686663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:33:28.686663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:33:28.686663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:28.686663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:33:28.686663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:28.686663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:33:28.689967 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:33:28.690255 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:33:28.690372 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:33:28.690510 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:33:28.691399 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:33:28.691862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:28.692952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:33:28.692952 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:28.692952 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:33:28.692952 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:33:28.692952 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:28.692952 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:33:28.692952 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:28.692952 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:33:28.693708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:28.693708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:33:28.693708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:33:28.693708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:33:28.693708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:28.693708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:33:28.693708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:28.693708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:33:28.693708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:28.693708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:33:28.693708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:28.693708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:33:28.693708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:28.693708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:33:28.693708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:33:28.694159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:28.694159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:33:28.694159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:28.694159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:33:28.694159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:33:28.694159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:28.694159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:33:28.694159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:28.694159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:33:28.694590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:28.694590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:33:28.694590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:33:28.694590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:33:28.694590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:33:28.694590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:33:28.694590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:33:28.694590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:33:28.694590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:33:28.694590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:28.694590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:33:28.694982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:28.694982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:33:28.694982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:28.694982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:33:28.694982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:28.694982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:33:28.694982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:28.694982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:33:28.694982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:33:28.694982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:28.694982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:33:28.694982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:28.694982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:33:28.694982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:33:28.695345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:28.695345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:33:28.695345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:28.695345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:33:28.695671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:28.695671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:33:28.695671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:33:28.695671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:33:28.695671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:33:28.695671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:33:28.695671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:33:28.695671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:33:28.695671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:33:28.695671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:33:28.695671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:33:28.695671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:33:28.695671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:33:28.695671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:33:28.695671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:33:28.695671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:33:28.695671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:33:28.696211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:33:28.696211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:33:28.696211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:33:28.696211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:28.696211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:33:28.696211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:28.696211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:33:28.696211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:28.696211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:33:28.696211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:28.696211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:33:28.696211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:28.696211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:33:28.696211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:33:28.696211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:28.696211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:33:28.696211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:28.696211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:33:28.696211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:33:28.696737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:28.696737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:33:28.696737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:28.696737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:33:28.698255 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:33:28.698547 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:33:28.698655 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:33:28.698832 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:33:28.699431 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:33:28.699881 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:33:28.700084 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:28.700868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:33:28.700868 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:28.700868 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:33:28.700868 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:33:28.700868 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:28.700868 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:33:28.700868 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:28.700868 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:33:28.701674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:28.701674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:33:28.701674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:33:28.701674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:33:28.701674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:33:28.701674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:33:28.701674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:33:28.701674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:28.701674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:33:28.701674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:28.701674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:33:28.701674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:28.702082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:33:28.702082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:28.702082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:33:28.702082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:28.702082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:33:28.702082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:33:28.702082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:28.702082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:33:28.702082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:28.702082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:33:28.702082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:33:28.702082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:28.702082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:33:28.702444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:28.702444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:33:28.702638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:28.702638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:33:28.702638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:33:28.702638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:33:28.702638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:33:28.702638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:33:28.702638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:33:28.702638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:33:28.702638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:33:28.702638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:33:28.702638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:33:28.702638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:33:28.702638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:33:28.702638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:33:28.702638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:33:28.702638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:33:28.703184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:33:28.703184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:33:28.703184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:28.703184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:33:28.703184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:28.703184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:33:28.703184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:28.703184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:33:28.703184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:28.703184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:33:28.703184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:28.703184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:33:28.703184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:33:28.703184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:28.703184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:33:28.703184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:28.703184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:33:28.703184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:33:28.703184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:28.703779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:33:28.703779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:28.703779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:33:33.533852 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:33:33.535157 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:33:33.535157 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:33:33.536575 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:33:33.575903 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:33:33.576150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:33.577008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:33:33.577008 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:33.577008 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:33:33.577008 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:33:33.577008 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:33.577008 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:33:33.577008 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:33.577008 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:33:33.578188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:33.578188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:33:33.578188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:33:33.578188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:33:33.578188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:33:33.578188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:33:33.578188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:33:33.578188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:33.578188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:33:33.578188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:33.578188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:33:33.578188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:33.578916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:33:33.578916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:33.578916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:33:33.578916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:33.578916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:33:33.578916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:33:33.578916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:33.578916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:33:33.578916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:33.578916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:33:33.578916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:33:33.578916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:33.578916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:33:33.579647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:33.579647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:33:33.579647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:33.579647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:33:33.579647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:33:33.579647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:33:33.579647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:33:33.579647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:33:33.579647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:33:33.579647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:33:33.579647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:33:33.579647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:33:33.579647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:33:33.579647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:33:33.579647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:33:33.580499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:33:33.580499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:33:33.580499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:33:33.580499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:33:33.580499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:33:33.580499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:33.580499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:33:33.580499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:33.580499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:33:33.580499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:33.580499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:33:33.580499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:33.580499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:33:33.580499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:33.580499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:33:33.580499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:33:33.580499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:33.580499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:33:33.580499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:33.582514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:33:33.582514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:33:33.582514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:33.582514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:33:33.582514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:33.582514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:33:33.673224 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:33:33.674364 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:33:33.674364 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:33:33.675356 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:33:33.675704 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:33:33.675918 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:33.676821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:33:33.676821 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:33.676821 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:33:33.676821 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:33:33.676821 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:33.676821 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:33:33.676821 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:33.676821 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:33:33.678878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:33.678878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:33:33.678878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:33:33.678878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:33:33.678878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:33:33.678878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:33:33.678878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:33:33.678878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:33.678878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:33:33.678878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:33.678878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:33:33.678878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:33.679288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:33:33.679288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:33.679288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:33:33.679288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:33.679288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:33:33.679288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:33:33.679288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:33.679288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:33:33.679288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:33.679288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:33:33.679288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:33:33.679288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:33.679288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:33:33.679640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:33.679640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:33:33.679640 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:33:33.681571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:33.681571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:33:33.681571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:33:33.681571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:33:33.681571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:33:33.681571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:33:33.681571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:33:33.681571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:33:33.681571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:33:33.681571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:33:33.681571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:33:33.681571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:33:33.681571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:33:33.681571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:33:33.681571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:33:33.681571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:33:33.682052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:33:33.682052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:33:33.682052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:33.682052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:33:33.682052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:33.682052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:33:33.682052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:33.682052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:33:33.682052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:33.682052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:33:33.682052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:33.682052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:33:33.682052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:33:33.682052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:33.682052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:33:33.682052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:33.682052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:33:33.682052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:33:33.682052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:33.682527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:33:33.682527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:33.682527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:33:33.682527 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:33:33.682527 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:33:33.682527 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:33:33.683281 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:33:33.683663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:33.684619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:33:33.684619 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:33.684619 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:33:33.684619 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:33:33.684619 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:33.684619 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:33:33.684619 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:33.684619 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:33:33.685503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:33.685503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:33:33.685503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:33:33.685503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:33:33.685503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:33.685503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:33:33.685503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:33.685503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:33:33.685503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:33.685503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:33:33.685503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:33.685503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:33:33.685503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:33.685503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:33:33.685503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:33:33.686086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:33.686086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:33:33.686086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:33.686086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:33:33.686086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:33:33.686086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:33.686086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:33:33.686086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:33.686086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:33:33.686490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:33.686490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:33:33.686490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:33:33.686490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:33:33.686490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:33:33.686490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:33:33.686490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:33:33.686490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:33:33.686490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:33:33.686490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:33:33.686490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:33.686490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:33:33.687476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:33.687476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:33:33.687476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:33.687476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:33:33.687476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:33.687476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:33:33.687476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:33.687476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:33:33.687476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:33:33.687476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:33.687476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:33:33.687476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:33.687476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:33:33.687476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:33:33.688015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:33.688015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:33:33.688015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:33.688015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:33:33.688015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:33.688015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:33:33.688015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:33:33.688015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:33:33.688015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:33:33.688015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:33:33.688015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:33:33.688015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:33:33.688015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:33:33.688015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:33:33.688409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:33:33.688409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:33:33.688409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:33:33.688409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:33:33.688409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:33:33.688409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:33:33.688409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:33:33.688409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:33:33.688409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:33:33.688409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:33:33.688409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:33:33.688409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:33.688409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:33:33.688409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:33.688409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:33:33.688409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:33.688409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:33:33.688409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:33.688409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:33:33.688967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:33.688967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:33:33.688967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:33:33.688967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:33.688967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:33:33.688967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:33.688967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:33:33.688967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:33:33.688967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:33.688967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:33:33.688967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:33.688967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:33:33.690505 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:33:33.690860 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:33:33.690982 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:33:33.691129 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:33:33.691634 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:33:33.692181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:33.693098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:33:33.693098 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:33.693098 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:33:33.693098 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:33:33.693098 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:33.693098 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:33:33.693098 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:33.693098 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:33:33.693842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:33.693842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:33:33.693842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:33:33.693842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:33:33.693842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:33.693842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:33:33.693842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:33.693842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:33:33.693842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:33.693842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:33:33.693842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:33.693842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:33:33.693842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:33.693842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:33:33.693842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:33:33.694210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:33.694210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:33:33.694210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:33.694210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:33:33.694210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:33:33.694210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:33.694210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:33:33.694210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:33.694210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:33:33.694891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:33.694891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:33:33.694891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:33:33.694891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:33:33.694891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:33:33.694891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:33:33.694891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:33:33.694891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:33:33.694891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:33:33.694891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:33.694891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:33:33.695561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:33.695561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:33:33.695561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:33.695561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:33:33.695561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:33.695561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:33:33.695561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:33.695561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:33:33.695561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:33:33.695561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:33.695561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:33:33.695561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:33.695561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:33:33.695561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:33:33.696305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:33.696305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:33:33.696305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:33.696305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:33:33.696305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:33.696305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:33:33.696305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:33:33.696305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:33:33.696305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:33:33.696305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:33:33.696305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:33:33.696305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:33:33.696305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:33:33.696305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:33:33.696684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:33:33.696684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:33:33.696684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:33:33.696684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:33:33.696684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:33:33.696684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:33:33.696684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:33:33.696684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:33:33.696684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:33:33.696684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:33:33.696684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:33.696684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:33:33.696684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:33.696684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:33:33.696684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:33.696684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:33:33.696684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:33.696684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:33:33.696684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:33.697174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:33:33.697174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:33:33.697174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:33.697174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:33:33.697174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:33.697174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:33:33.697174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:33:33.697174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:33.697174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:33:33.697174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:33.697174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:33:33.698766 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:33:33.699147 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:33:33.699265 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:33:33.699436 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:33:33.699946 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:33:33.700423 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:33:33.700686 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:33.701384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:33:33.701384 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:33.701384 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:33:33.701384 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:33:33.701384 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:33.701384 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:33:33.701384 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:33.701384 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:33:33.702228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:33.702228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:33:33.702228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:33:33.702228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:33:33.702228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:33:33.702228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:33:33.702228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:33:33.702228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:33.702228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:33:33.702228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:33.702228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:33:33.702228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:33.702554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:33:33.702554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:33.702554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:33:33.702554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:33.702554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:33:33.702554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:33:33.702554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:33.702554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:33:33.702554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:33.702554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:33:33.702554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:33:33.702554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:33.702554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:33:33.702885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:33.702885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:33:33.703521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:33.703521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:33:33.703521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:33:33.703521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:33:33.703521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:33:33.703521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:33:33.703521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:33:33.703521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:33:33.703521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:33:33.703521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:33:33.703521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:33:33.703521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:33:33.703521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:33:33.703521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:33:33.703521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:33:33.703521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:33:33.703975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:33:33.703975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:33:33.703975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:33.703975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:33:33.703975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:33.703975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:33:33.703975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:33.703975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:33:33.703975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:33.703975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:33:33.703975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:33.703975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:33:33.703975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:33:33.703975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:33.703975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:33:33.703975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:33.703975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:33:33.703975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:33:33.703975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:33.704485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:33:33.704485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:33.704485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:33:38.534829 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:33:38.535796 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:33:38.535796 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:33:38.536443 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:33:38.590314 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:33:38.590698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:38.591788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:33:38.591788 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:38.591788 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:33:38.591788 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:33:38.591788 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:38.591788 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:33:38.591788 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:38.591788 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:33:38.592711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.056 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:38.592711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:33:38.592711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:33:38.592711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:33:38.592711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:33:38.592711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:33:38.592711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:33:38.592711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:38.592711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:33:38.592711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:38.592711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:33:38.592711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:38.593461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:33:38.593461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:38.593461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:33:38.593461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:38.593461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:33:38.593461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:33:38.593461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:38.593461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:33:38.593461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:38.593461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:33:38.593461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:33:38.593461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:38.593461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:33:38.594200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:38.594200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:33:38.594200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:38.594200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:33:38.594200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:33:38.594200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:33:38.594200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:33:38.594200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:33:38.594200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:33:38.594200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:33:38.594200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:33:38.594200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:33:38.594200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:33:38.594200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:33:38.594200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:33:38.595125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:33:38.595125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:33:38.595125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:33:38.595125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:33:38.595125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:33:38.595125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:38.595125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:33:38.595125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:38.595125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:33:38.595125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:38.595125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:33:38.595125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:38.595125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:33:38.595125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:38.595125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:33:38.595125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:33:38.595125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:38.595125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:33:38.595125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:38.596174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:33:38.596174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:33:38.596174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:38.596174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:33:38.596174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:38.596174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:33:38.673902 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:33:38.674523 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:33:38.674656 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:33:38.676134 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:33:38.676560 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:33:38.676833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:38.679061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:33:38.679061 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:38.679061 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:33:38.679061 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:33:38.679061 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:38.679061 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:33:38.679061 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:38.679061 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:33:38.680587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:38.680587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:33:38.680587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:33:38.680587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:33:38.680587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:33:38.680587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:33:38.680587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:33:38.680587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:38.680587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:33:38.680587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:38.680587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:33:38.680587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:38.681320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:33:38.681320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:38.681320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:33:38.681320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:38.681320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:33:38.681320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:33:38.681320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:38.681320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:33:38.681320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:38.681320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:33:38.681320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:33:38.681320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:38.681320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:33:38.681904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:38.681904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:33:38.682447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:38.682447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:33:38.682447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:33:38.682447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:33:38.682447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:33:38.682447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:33:38.682447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:33:38.682447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:33:38.682447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:33:38.682447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:33:38.682447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:33:38.682447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:33:38.682447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:33:38.682447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:33:38.682447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:33:38.682447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:33:38.683217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:33:38.683217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:33:38.683217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:38.683217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:33:38.683217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:38.683217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:33:38.683217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:38.683217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:33:38.683217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:38.683217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:33:38.683217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:38.683217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:33:38.683217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:33:38.683217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:38.683217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:33:38.683217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:38.683217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:33:38.683217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:33:38.683217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:38.684056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:33:38.684056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:38.684056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:33:38.684056 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:33:38.684056 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:33:38.684056 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:33:38.684056 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:33:38.684056 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:33:38.684481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:38.684978 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:33:38.684978 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:38.684978 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:33:38.684978 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:33:38.684978 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:38.684978 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:33:38.684978 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:38.684978 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:33:38.685849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:38.685849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:33:38.685849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:33:38.685849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:33:38.685849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:38.685849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:33:38.685849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:38.685849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:33:38.685849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:38.685849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:33:38.685849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:38.685849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:33:38.685849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:38.685849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:33:38.685849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:33:38.686473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:38.686473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:33:38.686473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:38.686473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:33:38.686473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:33:38.686473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:38.686473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:33:38.686473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:38.686473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:33:38.687215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:38.687215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:33:38.687215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:33:38.687215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:33:38.687215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:33:38.687215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:33:38.687215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:33:38.687215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:33:38.687215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:33:38.687215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:33:38.687215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:38.687215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:33:38.687614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:38.687614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:33:38.687614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:38.687614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:33:38.687614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:38.687614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:33:38.687614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:38.687614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:33:38.687614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:33:38.687614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:38.687614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:33:38.687614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:38.687614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:33:38.687614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:33:38.688028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:38.688028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:33:38.688028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:38.688028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:33:38.688545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:38.688545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:33:38.688545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:33:38.688545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:33:38.688545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:33:38.688545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:33:38.688545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:33:38.688545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:33:38.688545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:33:38.688545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:33:38.688545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:33:38.688545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:33:38.688545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:33:38.688545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:33:38.688545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:33:38.688545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:33:38.689060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:33:38.689060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:33:38.689060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:33:38.689060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:33:38.689060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:33:38.689060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:38.689060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:33:38.689060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:38.689060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:33:38.689060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:38.689060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:33:38.689060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:38.689060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:33:38.689060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:38.689060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:33:38.689060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:33:38.689060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:38.689060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:33:38.689667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:38.689667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:33:38.689667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:33:38.689667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:38.689667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:33:38.689667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:38.689667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:33:38.691193 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:33:38.691560 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:33:38.691681 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:33:38.691849 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:33:38.693051 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:33:38.693282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:38.693871 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:33:38.693871 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:38.693871 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:33:38.693871 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:33:38.693871 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:38.693871 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:33:38.693871 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:38.693871 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:33:38.694705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:38.694705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:33:38.694705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:33:38.694705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:33:38.694705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:38.694705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:33:38.694705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:38.694705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:33:38.694705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:38.694705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:33:38.694705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:38.694705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:33:38.694705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:38.694705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:33:38.694705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:33:38.695138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:38.695138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:33:38.695138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:38.695138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:33:38.695138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:33:38.695138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:38.695138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:33:38.695138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:38.695138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:33:38.695595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:38.695595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:33:38.695595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:33:38.695595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:33:38.695595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:33:38.695595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:33:38.695595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:33:38.695595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:33:38.695595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:33:38.695595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:38.695595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:33:38.695958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:38.695958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:33:38.695958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:38.695958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:33:38.695958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:38.695958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:33:38.695958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:38.695958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:33:38.695958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:33:38.695958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:38.695958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:33:38.695958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:38.695958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:33:38.695958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:33:38.696324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:38.696324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:33:38.696324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:38.696324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:33:38.696704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:38.696704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:33:38.696704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:33:38.696704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:33:38.696704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:33:38.696704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:33:38.696704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:33:38.696704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:33:38.696704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:33:38.696704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:33:38.696704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:33:38.696704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:33:38.696704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:33:38.696704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:33:38.696704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:33:38.696704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:33:38.696704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:33:38.697228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:33:38.697228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:33:38.697228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:33:38.697228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:38.697228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:33:38.697228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:38.697228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:33:38.697228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:38.697228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:33:38.697228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:38.697228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:33:38.697228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:38.697228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:33:38.697228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:33:38.697228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:38.697228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:33:38.697228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:38.697228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:33:38.697228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:33:38.697735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:38.697735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:33:38.697735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:38.697735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:33:38.699293 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:33:38.699687 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:33:38.699848 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:33:38.700060 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:33:38.702134 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:33:38.702371 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:33:38.702521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:38.703091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:33:38.703091 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:38.703091 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:33:38.703091 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:33:38.703091 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:38.703091 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:33:38.703091 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:38.703091 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:33:38.704172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:38.704172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:33:38.704172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:33:38.704172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:33:38.704172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:33:38.704172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:33:38.704172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:33:38.704172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:38.704172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:33:38.704172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:38.704172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:33:38.704172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:38.704540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:33:38.704540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:38.704540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:33:38.704540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:38.704540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:33:38.704540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:33:38.704540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:38.704540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:33:38.704540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:38.704540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:33:38.704540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:33:38.704540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:38.704540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:33:38.704974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:38.704974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:33:38.705337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:38.705337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:33:38.705337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:33:38.705337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:33:38.705337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:33:38.705337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:33:38.705337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:33:38.705337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:33:38.705337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:33:38.705337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:33:38.705337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:33:38.705337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:33:38.705337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:33:38.705337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:33:38.705337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:33:38.705337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:33:38.705804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:33:38.705804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:33:38.705804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:38.705804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:33:38.705804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:38.705804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:33:38.705804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:38.705804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:33:38.705804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:38.705804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:33:38.705804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:38.705804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:33:38.705804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:33:38.705804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:38.705804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:33:38.705804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:38.705804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:33:38.705804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:33:38.705804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:38.706790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:33:38.706790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:38.706790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:33:43.535483 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:33:43.536661 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:33:43.536661 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:33:43.537725 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:33:43.575986 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:33:43.576492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:43.577676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:33:43.577676 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:43.577676 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:33:43.577676 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:33:43.577676 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:43.577676 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:33:43.577676 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:43.577676 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:33:43.578767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:43.578767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:33:43.578767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:33:43.578767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:33:43.578767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:33:43.578767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:33:43.578767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:33:43.578767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:43.578767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:33:43.578767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:43.578767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:33:43.578767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:43.579506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:33:43.579506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:43.579506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:33:43.579506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:43.579506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:33:43.579506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:33:43.579506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:43.579506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:33:43.579506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:43.579506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:33:43.579506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:33:43.579506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:43.579506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:33:43.580260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:43.580260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:33:43.580260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:43.580260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:33:43.580260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:33:43.580260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:33:43.580260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:33:43.580260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:33:43.580260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:33:43.580260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:33:43.580260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:33:43.580260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:33:43.580260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:33:43.580260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:33:43.580260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:33:43.581170 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:33:43.581170 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:33:43.581170 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:33:43.581170 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:33:43.581170 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:33:43.581170 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:43.581170 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:33:43.581170 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:43.581170 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:33:43.581170 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:43.581170 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:33:43.581170 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:43.581170 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:33:43.581170 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:43.581170 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:33:43.581170 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:33:43.581170 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:43.581170 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:33:43.581170 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:43.582184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:33:43.582184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:33:43.582184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:43.582184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:33:43.582184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:43.582184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:33:43.674875 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:33:43.675554 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:33:43.675895 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:33:43.677922 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:33:43.678529 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:33:43.678529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:43.679754 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:33:43.679754 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:43.679754 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:33:43.679754 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:33:43.679754 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:43.679754 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:33:43.679754 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:43.679754 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:33:43.680995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:43.680995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:33:43.680995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:33:43.680995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:33:43.680995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:33:43.680995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:33:43.680995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:33:43.680995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:43.680995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:33:43.680995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:43.680995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:33:43.680995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:43.681299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:33:43.681299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:43.681299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:33:43.681299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:43.681299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:33:43.681299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:33:43.681299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:43.681299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:33:43.681299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:43.681299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:33:43.681299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:33:43.681299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:43.681299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:33:43.681552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:43.681552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:33:43.682187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:43.682187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:33:43.682187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:33:43.682187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:33:43.682187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:33:43.682187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:33:43.682187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:33:43.682187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:33:43.682187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:33:43.682187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:33:43.682187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:33:43.682187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:33:43.682187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:33:43.682187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:33:43.682187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:33:43.682187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:33:43.683039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:33:43.683039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:33:43.683039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:43.683039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:33:43.683039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:43.683039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:33:43.683039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:43.683039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:33:43.683039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:43.683039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:33:43.683039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:43.683039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:33:43.683039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:33:43.683039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:43.683039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:33:43.683039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:43.683039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:33:43.683039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:33:43.683039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:43.683405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:33:43.683405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:43.683405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:33:43.683405 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:33:43.683405 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:33:43.683544 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:33:43.683647 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:33:43.684118 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:33:43.684404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:43.685100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:33:43.685100 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:43.685100 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:33:43.685100 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:33:43.685100 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:43.685100 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:33:43.685100 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:43.685100 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:33:43.686185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:43.686185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:33:43.686185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:33:43.686185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:33:43.686185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:43.686185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:33:43.686185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:43.686185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:33:43.686185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:43.686185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:33:43.686185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:43.686185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:33:43.686185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:43.686185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:33:43.686185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:33:43.686526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:43.686526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:33:43.686526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:43.686526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:33:43.686526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:33:43.686526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:43.686526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:33:43.686526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:43.686526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:33:43.687422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:43.687422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:33:43.687422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:33:43.687422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:33:43.687422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:33:43.687422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:33:43.687422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:33:43.687422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:33:43.687422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:33:43.687422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:33:43.687422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:43.687422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:33:43.687818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:43.687818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:33:43.687818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:43.687818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:33:43.687818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:43.687818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:33:43.687818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:43.687818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:33:43.687818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:33:43.687818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:43.687818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:33:43.687818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:43.687818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:33:43.687818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:33:43.688168 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:43.688168 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:33:43.688168 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:43.688168 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:33:43.689027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:43.689027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:33:43.689027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:33:43.689027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:33:43.689027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:33:43.689027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:33:43.689027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:33:43.689027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:33:43.689027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:33:43.689027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:33:43.689027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:33:43.689027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:33:43.689027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:33:43.689027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:33:43.689027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:33:43.689027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:33:43.689359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:33:43.689359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:33:43.689359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:33:43.689359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:33:43.689359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:33:43.689359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:43.689359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:33:43.689359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:43.689359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:33:43.689359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:43.689359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:33:43.689359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:43.689359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:33:43.689359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:43.689359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:33:43.689359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:33:43.689359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:43.689359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:33:43.689759 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:43.689759 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:33:43.689759 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:33:43.689759 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:43.689759 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:33:43.689759 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:43.689759 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:33:43.691548 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:33:43.691804 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:33:43.691970 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:33:43.692173 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:33:43.692646 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:33:43.693023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:43.693911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:33:43.693911 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:43.693911 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:33:43.693911 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:33:43.693911 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:43.693911 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:33:43.693911 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:43.693911 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:33:43.695002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:43.695002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:33:43.695002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:33:43.695002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:33:43.695002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:43.695002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:33:43.695002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:43.695002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:33:43.695002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:43.695002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:33:43.695002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:43.695002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:33:43.695002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:43.695002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:33:43.695002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:33:43.695391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:43.695391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:33:43.695391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:43.695391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:33:43.695391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:33:43.695391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:43.695391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:33:43.695391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:43.695391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:33:43.696153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:43.696153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:33:43.696153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:33:43.696153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:33:43.696153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:33:43.696153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:33:43.696153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:33:43.696153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:33:43.696153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:33:43.696153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:43.696153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:33:43.696637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:43.696637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:33:43.696637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:43.696637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:33:43.696637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:43.696637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:33:43.696637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:43.696637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:33:43.696637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:33:43.696637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:43.696637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:33:43.696637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:43.696637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:33:43.696637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:33:43.697015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:43.697015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:33:43.697015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:43.697015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:33:43.697355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:43.697355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:33:43.697355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:33:43.697355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:33:43.697355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:33:43.697355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:33:43.697355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:33:43.697355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:33:43.697355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:33:43.697355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:33:43.697355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:33:43.697355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:33:43.697355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:33:43.697355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:33:43.697355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:33:43.697355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:33:43.697355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:33:43.697844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:33:43.697844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:33:43.697844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:33:43.697844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:43.697844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:33:43.697844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:43.697844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:33:43.697844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:43.697844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:33:43.697844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:43.697844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:33:43.697844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:43.697844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:33:43.697844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:33:43.697844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:43.697844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:33:43.697844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:43.697844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:33:43.697844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:33:43.698339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:43.698339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:33:43.698339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:43.698339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:33:43.699677 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:33:43.699977 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:33:43.700136 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:33:43.700356 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:33:43.701454 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:33:43.702066 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:33:43.702436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:43.703454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:33:43.703454 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:43.703454 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:33:43.703454 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:33:43.703454 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:43.703454 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:33:43.703454 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:43.703454 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:33:43.704783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:43.704783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:33:43.704783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:33:43.704783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:33:43.704783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:33:43.704783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:33:43.704783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:33:43.704783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:43.704783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:33:43.704783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:43.704783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:33:43.704783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:43.705150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:33:43.705150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:43.705150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:33:43.705150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:43.705150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:33:43.705150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:33:43.705150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:43.705150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:33:43.705150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:43.705150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:33:43.705150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:33:43.705150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:43.705150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:33:43.705467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:43.705467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:33:43.706277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:43.706277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:33:43.706277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:33:43.706277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:33:43.706277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:33:43.706277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:33:43.706277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:33:43.706277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:33:43.706277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:33:43.706277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:33:43.706277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:33:43.706277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:33:43.706277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:33:43.706277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:33:43.706277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:33:43.706277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:33:43.706727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:33:43.706727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:33:43.706727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:43.706727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:33:43.706727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:43.706727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:33:43.706727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:43.706727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:33:43.706727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:43.706727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:33:43.706727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:43.706727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:33:43.706727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:33:43.706727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:43.706727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:33:43.706727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:43.706727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:33:43.706727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:33:43.706727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:43.707273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:33:43.707273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:43.707273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:33:48.535981 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:33:48.537317 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:33:48.537317 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:33:48.537317 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:33:48.572003 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:33:48.572003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:48.572348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:33:48.572348 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:48.572348 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:33:48.572348 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:33:48.572348 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:48.572348 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:33:48.572348 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:48.572348 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:33:48.573482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:48.573482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:33:48.573482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:33:48.573482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:33:48.573482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:33:48.573482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:33:48.573482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:33:48.573482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:48.573482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:33:48.573482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:48.573482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:33:48.573482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:48.574029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:33:48.574029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:48.574029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:33:48.574029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:48.574029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:33:48.574029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:33:48.574029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:48.574029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:33:48.574029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:48.574029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:33:48.574029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:33:48.574029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:48.574029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:33:48.574523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:48.574523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:33:48.574523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:48.574523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:33:48.574523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:33:48.574523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:33:48.574523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:33:48.574523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:33:48.574523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:33:48.574523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:33:48.574523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:33:48.574523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:33:48.574523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:33:48.574523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:33:48.574523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:33:48.575121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:33:48.575121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:33:48.575121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:33:48.575121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:33:48.575121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:33:48.575121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:48.575121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:33:48.575121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:48.575121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:33:48.575121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:48.575121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:33:48.575121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:48.575121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:33:48.575121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:48.575121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:33:48.575121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:33:48.575121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:48.575121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:33:48.575121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:48.575828 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:33:48.575828 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:33:48.575828 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:48.575828 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:33:48.575828 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:48.575828 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:33:48.675705 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:33:48.676122 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:33:48.676122 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:33:48.677922 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:33:48.677922 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:33:48.678427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:48.678965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:33:48.678965 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:48.678965 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:33:48.678965 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:33:48.678965 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:48.678965 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:33:48.678965 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:48.678965 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:33:48.680050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:48.680050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:33:48.680050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:33:48.680050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:33:48.680050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:33:48.680050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:33:48.680050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:33:48.680050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:48.680050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:33:48.680050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:48.680050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:33:48.680050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:48.680476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:33:48.680476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:48.680476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:33:48.680476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:48.680476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:33:48.680476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:33:48.680476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:48.680476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:33:48.680476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:48.680476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:33:48.680476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:33:48.680476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:48.680476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:33:48.680863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:48.680863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:33:48.681075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:48.681075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:33:48.681075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:33:48.681075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:33:48.681075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:33:48.681075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:33:48.681075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:33:48.681075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:33:48.681075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:33:48.681075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:33:48.681075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:33:48.681075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:33:48.681075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:33:48.681075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:33:48.681075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:33:48.681075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:33:48.681560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:33:48.681560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:33:48.681560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:48.681560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:33:48.681560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:48.681560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:33:48.681560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:48.681560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:33:48.681560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:48.681560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:33:48.681560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:48.681560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:33:48.681560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:33:48.681560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:48.681560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:33:48.681560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:48.681560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:33:48.681560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:33:48.681560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:48.682080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:33:48.682080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:48.682080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:33:48.683462 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:33:48.683660 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:33:48.683798 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:33:48.683929 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:33:48.684327 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:33:48.684612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:48.685332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:33:48.685332 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:48.685332 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:33:48.685332 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:33:48.685332 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:48.685332 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:33:48.685332 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:48.685332 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:33:48.686177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:48.686177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:33:48.686177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:33:48.686177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:33:48.686177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:48.686177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:33:48.686177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:48.686177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:33:48.686177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:48.686177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:33:48.686177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:48.686177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:33:48.686177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:48.686177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:33:48.686177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:33:48.686720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:48.686720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:33:48.686720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:48.686720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:33:48.686720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:33:48.686720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:48.686720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:33:48.686720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:48.686720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:33:48.687065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:48.687065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:33:48.687065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:33:48.687065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:33:48.687065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:33:48.687065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:33:48.687065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:33:48.687065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:33:48.687065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:33:48.687065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:33:48.687065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:48.687065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:33:48.687513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:48.687513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:33:48.687513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:48.687513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:33:48.687513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:48.687513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:33:48.687513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:48.687513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:33:48.687513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:33:48.687513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:48.687513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:33:48.687513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:48.687513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:33:48.687513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:33:48.688029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:48.688029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:33:48.688029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:48.688029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:33:48.688186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:48.688186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:33:48.688186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:33:48.688186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:33:48.688186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:33:48.688186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:33:48.688186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:33:48.688186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:33:48.688186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:33:48.688186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:33:48.688186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:33:48.688186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:33:48.688186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:33:48.688186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:33:48.688186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:33:48.688186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:33:48.688772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:33:48.688772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:33:48.688772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:33:48.688772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:33:48.688772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:33:48.688772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:48.688772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:33:48.688772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:48.688772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:33:48.688772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:48.688772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:33:48.688772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:48.688772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:33:48.688772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:48.688772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:33:48.688772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:33:48.688772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:48.688772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:33:48.689413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:48.689413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:33:48.689413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:33:48.689413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:48.689413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:33:48.689413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:48.689413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:33:48.691858 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:33:48.692107 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:33:48.692254 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:33:48.692426 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:33:48.692978 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:33:48.693336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:48.694102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:33:48.694102 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:48.694102 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:33:48.694102 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:33:48.694102 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:48.694102 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:33:48.694102 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:48.694102 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:33:48.694957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:48.694957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:33:48.694957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:33:48.694957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:33:48.694957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:48.694957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:33:48.694957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:48.694957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:33:48.694957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:48.694957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:33:48.694957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:48.694957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:33:48.694957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:48.694957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:33:48.694957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:33:48.695466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:48.695466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:33:48.695466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:48.695466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:33:48.695466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:33:48.695466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:48.695466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:33:48.695466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:48.695466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:33:48.695898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:48.695898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:33:48.695898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:33:48.695898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:33:48.695898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:33:48.695898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:33:48.695898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:33:48.695898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:33:48.695898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:33:48.695898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:48.695898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:33:48.696267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:48.696267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:33:48.696267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:48.696267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:33:48.696267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:48.696267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:33:48.696267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:48.696267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:33:48.696267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:33:48.696267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:48.696267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:33:48.696267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:48.696267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:33:48.696267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:33:48.696684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:48.696684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:33:48.696684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:48.696684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:33:48.697006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:48.697006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:33:48.697006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:33:48.697006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:33:48.697006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:33:48.697006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:33:48.697006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:33:48.697006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:33:48.697006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:33:48.697006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:33:48.697006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:33:48.697006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:33:48.697006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:33:48.697006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:33:48.697006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:33:48.697006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:33:48.697006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:33:48.697526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:33:48.697526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:33:48.697526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:33:48.697526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:48.697526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:33:48.697526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:48.697526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:33:48.697526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:48.697526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:33:48.697526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:48.697526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:33:48.697526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:48.697526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:33:48.697526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:33:48.697526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:48.697526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:33:48.697526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:48.697526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:33:48.697526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:33:48.698148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:48.698148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:33:48.698148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:48.698148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:33:48.700003 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:33:48.700188 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:33:48.700291 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:33:48.700462 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:33:48.700963 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:33:48.701307 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:33:48.701572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:48.702356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:33:48.702356 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:48.702356 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:33:48.702356 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:33:48.702356 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:48.702356 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:33:48.702356 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:48.702356 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:33:48.703275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:48.703275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:33:48.703275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:33:48.703275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:33:48.703275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:33:48.703275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:33:48.703275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:33:48.703275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:48.703275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:33:48.703275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:48.703275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:33:48.703275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:48.703726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:33:48.703726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:48.703726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:33:48.703726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:48.703726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:33:48.703726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:33:48.703726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:48.703726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:33:48.703726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:48.703726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:33:48.703726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:33:48.703726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:48.703726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:33:48.704147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:48.704147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:33:48.704334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:48.704334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:33:48.704334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:33:48.704334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:33:48.704334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:33:48.704334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:33:48.704334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:33:48.704334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:33:48.704334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:33:48.704334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:33:48.704334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:33:48.704334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:33:48.704334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:33:48.704334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:33:48.704334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:33:48.704334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:33:48.704887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:33:48.704887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:33:48.704887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:48.704887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:33:48.704887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:48.704887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:33:48.704887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:48.704887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:33:48.704887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:48.704887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:33:48.704887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:48.704887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:33:48.704887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:33:48.704887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:48.704887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:33:48.704887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:48.704887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:33:48.704887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:33:48.704887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:48.705617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:33:48.705617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:48.705617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:33:53.537060 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:33:53.538018 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:33:53.538018 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:33:53.538589 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:33:53.579614 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:33:53.580159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:53.581269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:33:53.581269 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:53.581269 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:33:53.581269 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:33:53.581269 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:53.581269 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:33:53.581269 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:53.581269 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:33:53.582172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.044 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:53.582172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:33:53.582172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:33:53.582172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:33:53.582172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:33:53.582172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:33:53.582172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:33:53.582172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:53.582172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:33:53.582172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:53.582172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:33:53.582172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:53.583020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:33:53.583020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:53.583020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:33:53.583020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:53.583020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:33:53.583020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:33:53.583020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:53.583020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:33:53.583020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:53.583020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:33:53.583020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:33:53.583020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:53.583020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:33:53.583837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:53.583837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:33:53.583837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:53.583837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:33:53.583837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:33:53.583837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:33:53.583837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:33:53.583837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:33:53.583837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:33:53.583837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:33:53.583837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:33:53.583837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:33:53.583837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:33:53.583837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:33:53.583837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:33:53.584598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:33:53.584598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:33:53.584598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:33:53.584598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:33:53.584598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:33:53.584598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:53.584598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:33:53.584598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:53.584598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:33:53.584598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:53.584598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:33:53.584598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:53.584598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:33:53.584598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:53.584598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:33:53.584598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:33:53.584598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:53.584598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:33:53.584598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:53.585726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:33:53.585726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:33:53.585726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:53.585726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:33:53.585726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:53.585726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:33:53.676298 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:33:53.676774 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:33:53.677036 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:33:53.678899 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:33:53.678899 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:33:53.679232 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:53.680215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:33:53.680215 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:53.680215 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:33:53.680215 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:33:53.680215 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:53.680215 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:33:53.680215 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:53.680215 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:33:53.682917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:53.682917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:33:53.682917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:33:53.682917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:33:53.682917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:33:53.682917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:33:53.682917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:33:53.682917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:53.682917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:33:53.682917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:53.682917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:33:53.682917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:53.683363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:33:53.683363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:53.683363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:33:53.683363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:53.683363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:33:53.683363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:33:53.683363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:53.683363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:33:53.683363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:53.683363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:33:53.683363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:33:53.683363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:53.683363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:33:53.683840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:53.683840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:33:53.684130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:53.684130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:33:53.684130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:33:53.684130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:33:53.684130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:33:53.684130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:33:53.684130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:33:53.684130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:33:53.684130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:33:53.684130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:33:53.684130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:33:53.684130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:33:53.684130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:33:53.684130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:33:53.684130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:33:53.684130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:33:53.685039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:33:53.685039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:33:53.685039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:53.685039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:33:53.685039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:53.685039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:33:53.685039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:53.685039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:33:53.685039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:53.685039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:33:53.685039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:53.685039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:33:53.685039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:33:53.685039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:53.685039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:33:53.685039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:53.685039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:33:53.685039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:33:53.685039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:53.685721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:33:53.685721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:53.685721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:33:53.685721 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:33:53.685721 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:33:53.685721 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:33:53.685721 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:33:53.685721 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:33:53.686090 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:53.686425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:33:53.686425 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:53.686425 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:33:53.686425 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:33:53.686425 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:53.686425 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:33:53.686425 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:53.686425 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:33:53.687201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:53.687201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:33:53.687201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:33:53.687201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:33:53.687201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:53.687201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:33:53.687201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:53.687201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:33:53.687201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:53.687201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:33:53.687201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:53.687201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:33:53.687201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:53.687201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:33:53.687201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:33:53.687998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:53.687998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:33:53.687998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:53.687998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:33:53.687998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:33:53.687998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:53.687998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:33:53.687998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:53.687998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:33:53.687998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:53.687998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:33:53.687998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:33:53.687998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:33:53.687998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:33:53.688501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:33:53.688501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:33:53.688501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:33:53.688501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:33:53.688501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:33:53.688501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:53.688501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:33:53.688501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:53.688501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:33:53.688501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:53.688501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:33:53.688501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:53.688501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:33:53.689011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:53.689011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:33:53.689011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:33:53.689011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:53.689011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:33:53.689011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:53.689011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:33:53.689011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:33:53.689011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:53.689011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:33:53.689011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:53.689011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:33:53.689532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:53.689532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:33:53.689532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:33:53.689532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:33:53.689532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:33:53.689532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:33:53.689532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:33:53.689532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:33:53.689532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:33:53.689532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:33:53.689532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:33:53.689532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:33:53.689532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:33:53.689532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:33:53.689532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:33:53.689532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:33:53.690133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:33:53.690133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:33:53.690133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:33:53.690133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:33:53.690133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:33:53.690133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:53.690133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:33:53.690133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:53.690133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:33:53.690133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:53.690133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:33:53.690133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:53.690133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:33:53.690133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:53.690133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:33:53.690133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:33:53.690133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:53.690133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:33:53.690765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:53.690765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:33:53.690765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:33:53.690765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:53.690765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:33:53.690765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:53.690765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:33:53.692199 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:33:53.692463 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:33:53.692580 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:33:53.692718 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:33:53.693126 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:33:53.693340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:53.693862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:33:53.693862 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:53.693862 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:33:53.693862 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:33:53.693862 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:53.693862 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:33:53.693862 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:53.693862 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:33:53.694621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:53.694621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:33:53.694621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:33:53.694621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:33:53.694621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:53.694621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:33:53.694621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:53.694621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:33:53.694621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:53.694621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:33:53.694621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:53.694621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:33:53.694621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:53.694621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:33:53.694621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:33:53.695077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:53.695077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:33:53.695077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:53.695077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:33:53.695077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:33:53.695077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:53.695077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:33:53.695077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:53.695077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:33:53.695549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:53.695549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:33:53.695549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:33:53.695549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:33:53.695549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:33:53.695549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:33:53.695549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:33:53.695549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:33:53.695549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:33:53.695549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:53.695549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:33:53.695914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:53.695914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:33:53.695914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:53.695914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:33:53.695914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:53.695914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:33:53.695914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:53.695914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:33:53.695914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:33:53.695914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:53.695914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:33:53.695914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:53.695914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:33:53.695914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:33:53.696281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:53.696281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:33:53.696281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:53.696281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:33:53.696581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:53.696581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:33:53.696581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:33:53.696581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:33:53.696581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:33:53.696581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:33:53.696581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:33:53.696581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:33:53.696581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:33:53.696581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:33:53.696581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:33:53.696581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:33:53.696581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:33:53.696581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:33:53.696581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:33:53.696581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:33:53.696581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:33:53.697151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:33:53.697151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:33:53.697151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:33:53.697151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:53.697151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:33:53.697151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:53.697151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:33:53.697151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:53.697151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:33:53.697151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:53.697151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:33:53.697151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:53.697151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:33:53.697151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:33:53.697151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:53.697151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:33:53.697151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:53.697151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:33:53.697151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:33:53.697605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:53.697605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:33:53.697605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:53.697605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:33:53.700357 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:33:53.700612 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:33:53.700722 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:33:53.700919 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:33:53.701644 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:33:53.702193 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:33:53.702461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:53.703310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:33:53.703310 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:53.703310 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:33:53.703310 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:33:53.703310 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:53.703310 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:33:53.703310 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:53.703310 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:33:53.704171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:53.704171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:33:53.704171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:33:53.704171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:33:53.704171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:33:53.704171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:33:53.704171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:33:53.704171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:53.704171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:33:53.704171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:53.704171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:33:53.704171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:53.704548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:33:53.704548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:53.704548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:33:53.704548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:53.704548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:33:53.704548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:33:53.704548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:53.704548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:33:53.704548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:53.704548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:33:53.704548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:33:53.704548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:53.704548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:33:53.704938 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:53.704938 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:33:53.705193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:53.705193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:33:53.705193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:33:53.705193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:33:53.705193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:33:53.705193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:33:53.705193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:33:53.705193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:33:53.705193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:33:53.705193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:33:53.705193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:33:53.705193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:33:53.705193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:33:53.705193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:33:53.705193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:33:53.705193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:33:53.705602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:33:53.705602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:33:53.705602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:53.705602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:33:53.705602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:53.705602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:33:53.705602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:53.705602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:33:53.705602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:53.705602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:33:53.705602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:53.705602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:33:53.705602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:33:53.705602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:53.705602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:33:53.705602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:53.705602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:33:53.705602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:33:53.705602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:53.706185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:33:53.706185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:53.706185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:33:58.537547 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:33:58.538699 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:33:58.538899 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:33:58.540173 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:33:58.542317 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG dbcounter [-] [106314] Writing DB stats neutron:SELECT=240 {{(pid=106314) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 09 09:33:58.583848 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:33:58.584362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:58.585434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:33:58.585434 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:58.585434 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:33:58.585434 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:33:58.585434 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:58.585434 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:33:58.585434 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:58.585434 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:33:58.586693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.047 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:58.586693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:33:58.586693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:33:58.586693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:33:58.586693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:33:58.586693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:33:58.586693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:33:58.586693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:58.586693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:33:58.586693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:58.586693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:33:58.586693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:58.587390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:33:58.587390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:58.587390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:33:58.587390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:58.587390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:33:58.587390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:33:58.587390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:58.587390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:33:58.587390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:58.587390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:33:58.587390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:33:58.587390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:58.587390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:33:58.587926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:58.587926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:33:58.587926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:58.587926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:33:58.587926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:33:58.587926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:33:58.587926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:33:58.587926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:33:58.587926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:33:58.587926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:33:58.587926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:33:58.587926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:33:58.587926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:33:58.587926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:33:58.587926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:33:58.588535 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:33:58.588535 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:33:58.588535 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:33:58.588535 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:33:58.588535 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:33:58.588535 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:58.588535 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:33:58.588535 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:58.588535 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:33:58.588535 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:58.588535 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:33:58.588535 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:58.588535 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:33:58.588535 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:58.588535 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:33:58.588535 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:33:58.588535 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:58.588535 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:33:58.588535 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:58.589287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:33:58.589287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:33:58.589287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:58.589287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:33:58.589287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:58.589287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:33:58.677161 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:33:58.678150 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:33:58.678423 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:33:58.680318 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:33:58.680765 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:33:58.681165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:58.682386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:33:58.682386 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:58.682386 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:33:58.682386 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:33:58.682386 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:58.682386 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:33:58.682386 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:58.682386 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:33:58.683904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:58.683904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:33:58.683904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:33:58.683904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:33:58.683904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:33:58.683904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:33:58.683904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:33:58.683904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:58.683904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:33:58.683904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:58.683904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:33:58.683904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:58.685411 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:33:58.685411 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:58.685411 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:33:58.685411 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:58.685411 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:33:58.685411 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:33:58.685411 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:58.685411 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:33:58.685411 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:58.685411 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:33:58.685411 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:33:58.685411 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:58.685411 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:33:58.685928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:58.685928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:33:58.685928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:58.685928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:33:58.685928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:33:58.685928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:33:58.685928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:33:58.685928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:33:58.685928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:33:58.685928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:33:58.685928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:33:58.685928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:33:58.685928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:33:58.685928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:33:58.685928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:33:58.686488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:33:58.686488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:33:58.686488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:33:58.686488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:33:58.686488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:33:58.686488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:58.686488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:33:58.686488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:58.686488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:33:58.686488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:58.686488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:33:58.686488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:58.686488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:33:58.686488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:58.686488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:33:58.686488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:33:58.686488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:58.686488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:33:58.686488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:58.687461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:33:58.687461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:33:58.687461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:58.687461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:33:58.687461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:58.687461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:33:58.687461 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:33:58.687461 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:33:58.687461 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:33:58.687461 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:33:58.690049 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:33:58.690049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:58.690049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:33:58.690049 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:58.690049 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:33:58.690049 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:33:58.690049 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:58.690049 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:33:58.690049 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:58.690049 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:33:58.690049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:58.690049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:33:58.690049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:33:58.690049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:33:58.691147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:58.691147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:33:58.691147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:58.691147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:33:58.691147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:58.691147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:33:58.691147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:58.691147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:33:58.691147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:58.691147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:33:58.691147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:33:58.691147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:58.691147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:33:58.691147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:58.691147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:33:58.691147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:33:58.691147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:58.691147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:33:58.692904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:58.692904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:33:58.692904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:58.692904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:33:58.692904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:33:58.692904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:33:58.692904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:33:58.692904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:33:58.692904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:33:58.692904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:33:58.692904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:33:58.692904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:33:58.693254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:58.693254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:33:58.693254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:58.693254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:33:58.693254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:58.693254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:33:58.693254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:58.693254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:33:58.693254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:58.693254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:33:58.693254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:33:58.693254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:58.693254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:33:58.693254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:58.693603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:33:58.693603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:33:58.693603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:58.693603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:33:58.693603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:58.693603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:33:58.693603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:58.693603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:33:58.693603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:33:58.693603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:33:58.693603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:33:58.693603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:33:58.693603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:33:58.693603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:33:58.693981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:33:58.693981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:33:58.693981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:33:58.693981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:33:58.693981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:33:58.693981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:33:58.693981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:33:58.693981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:33:58.693981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:33:58.693981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:33:58.693981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:33:58.693981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:33:58.693981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:33:58.693981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:58.693981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:33:58.693981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:58.693981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:33:58.693981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:58.693981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:33:58.694458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:58.694458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:33:58.694458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:58.694458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:33:58.694458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:33:58.694458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:58.694458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:33:58.694458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:58.694458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:33:58.694458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:33:58.694458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:58.694458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:33:58.694458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:58.694458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:33:58.694458 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:33:58.695576 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:33:58.695576 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:33:58.695576 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:33:58.695576 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:33:58.695576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:58.695576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:33:58.695576 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:58.695576 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:33:58.695576 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:33:58.696249 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:58.696249 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:33:58.696249 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:58.696249 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:33:58.696249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:58.696249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:33:58.696249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:33:58.696249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:33:58.696249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:58.696249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:33:58.696249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:58.696249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:33:58.696249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:58.696249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:33:58.696249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:58.696249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:33:58.697837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:58.697837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:33:58.697837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:33:58.697837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:58.697837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:33:58.697837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:58.697837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:33:58.697837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:33:58.697837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:58.697837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:33:58.697837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:58.697837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:33:58.697837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:58.697837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:33:58.698705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:33:58.698705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:33:58.698705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:33:58.698705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:33:58.698705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:33:58.698705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:33:58.698705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:33:58.698705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:58.698705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:33:58.698705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:58.698705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:33:58.698705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:58.698705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:33:58.699117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:58.699117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:33:58.699117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:58.699117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:33:58.699117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:33:58.699117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:58.699117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:33:58.699117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:58.699117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:33:58.699117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:33:58.699117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:58.699117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:33:58.699117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:58.699425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:33:58.699425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:58.699425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:33:58.699425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:33:58.699425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:33:58.699425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:33:58.699425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:33:58.699425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:33:58.699425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:33:58.699425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:33:58.699425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:33:58.699425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:33:58.699425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:33:58.699425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:33:58.699425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:33:58.699425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:33:58.699425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:33:58.700338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:33:58.700338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:33:58.700338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:33:58.700338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:33:58.700338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:58.700338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:33:58.700338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:58.700338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:33:58.700338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:58.700338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:33:58.700338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:58.700338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:33:58.700338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:58.700338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:33:58.700338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:33:58.700338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:58.700338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:33:58.700338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:58.700338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:33:58.701453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:33:58.701453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:58.701453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:33:58.701453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:58.701453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:33:58.701453 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:33:58.701453 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:33:58.701453 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:33:58.701453 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:33:58.702036 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:33:58.702437 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:33:58.702695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:58.703528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:33:58.703528 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:58.703528 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:33:58.703528 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:33:58.703528 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:58.703528 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:33:58.703528 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:58.703528 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:33:58.704483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:58.704483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:33:58.704483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:33:58.704483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:33:58.704483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:33:58.704483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:33:58.704483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:33:58.704483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:58.704483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:33:58.704483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:58.704483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:33:58.704483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:58.704737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:33:58.704737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:58.704737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:33:58.704737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:58.704737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:33:58.704737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:33:58.704737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:58.704737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:33:58.704737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:58.704737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:33:58.704737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:33:58.704737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:58.704737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:33:58.705000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:58.705000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:33:58.705515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:58.705515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:33:58.705515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:33:58.705515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:33:58.705515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:33:58.705515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:33:58.705515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:33:58.705515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:33:58.705515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:33:58.705515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:33:58.705515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:33:58.705515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:33:58.705515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:33:58.705515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:33:58.705515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:33:58.705515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:33:58.705881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:33:58.705881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:33:58.705881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:58.705881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:33:58.705881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:33:58.705881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:33:58.705881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:33:58.705881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:33:58.705881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:33:58.705881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:33:58.705881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:33:58.705881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:33:58.705881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:33:58.705881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:33:58.705881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:33:58.705881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:33:58.705881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:33:58.705881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:33:58.705881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:33:58.706292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:33:58.706292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:33:58.706292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:34:03.538519 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:34:03.539473 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:34:03.539779 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:34:03.541294 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:34:03.577779 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:34:03.578277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:03.579480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:34:03.579480 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:03.579480 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:34:03.579480 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:34:03.579480 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:03.579480 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:34:03.579480 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:03.579480 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:34:03.580680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:03.580680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:34:03.580680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:34:03.580680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:34:03.580680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:34:03.580680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:34:03.580680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:34:03.580680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:03.580680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:34:03.580680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:03.580680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:34:03.580680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:03.581601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:34:03.581601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:03.581601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:34:03.581601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:03.581601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:34:03.581601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:34:03.581601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:03.581601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:34:03.581601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:03.581601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:34:03.581601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:34:03.581601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:03.581601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:34:03.582157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:03.582157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:34:03.582157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:03.582157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:34:03.582157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:34:03.582157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:34:03.582157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:34:03.582157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:34:03.582157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:34:03.582157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:34:03.582157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:34:03.582157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:34:03.582157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:34:03.582157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:34:03.582157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:34:03.583408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:34:03.583408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:34:03.583408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:34:03.583408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:34:03.583408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:34:03.583408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:03.583408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:34:03.583408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:03.583408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:34:03.583408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:03.583408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:34:03.583408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:03.583408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:34:03.583408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:03.583408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:34:03.583408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:34:03.583408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:03.583408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:34:03.583408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:03.584298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:34:03.584298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:34:03.584298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:03.584298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:34:03.584298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:03.584298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:34:03.677942 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:34:03.678384 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:34:03.678507 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:34:03.680064 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:34:03.680593 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:34:03.681001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:03.682333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:34:03.682333 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:03.682333 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:34:03.682333 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:34:03.682333 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:03.682333 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:34:03.682333 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:03.682333 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:34:03.683945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:03.683945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:34:03.683945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:34:03.683945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:34:03.683945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:34:03.683945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:34:03.683945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:34:03.683945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:03.683945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:34:03.683945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:03.683945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:34:03.683945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:03.684573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:34:03.684573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:03.684573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:34:03.684573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:03.684573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:34:03.684573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:34:03.684573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:03.684573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:34:03.684573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:03.684573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:34:03.684573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:34:03.684573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:03.684573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:34:03.685176 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:03.685176 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:34:03.686211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:03.686211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:34:03.686211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:34:03.686211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:34:03.686211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:34:03.686211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:34:03.686211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:34:03.686211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:34:03.686211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:34:03.686211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:34:03.686211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:34:03.686211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:34:03.686211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:34:03.686211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:34:03.686211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:34:03.686211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:34:03.687524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:34:03.687524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:34:03.687524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:03.687524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:34:03.687524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:03.687524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:34:03.687524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:03.687524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:34:03.687524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:03.687524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:34:03.687524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:03.687524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:34:03.687524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:34:03.687524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:03.687524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:34:03.687524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:03.687524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:34:03.687524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:34:03.687524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:03.688416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:34:03.688416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:03.688416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:34:03.688416 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:34:03.688416 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:34:03.688416 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:34:03.688823 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:34:03.689246 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:34:03.689521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:03.690311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:34:03.690311 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:03.690311 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:34:03.690311 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:34:03.690311 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:03.690311 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:34:03.690311 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:03.690311 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:34:03.691484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:03.691484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:34:03.691484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:34:03.691484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:34:03.691484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:03.691484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:34:03.691484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:03.691484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:34:03.691484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:03.691484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:34:03.691484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:03.691484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:34:03.691484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:03.691484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:34:03.691484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:34:03.691939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:03.691939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:34:03.691939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:03.691939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:34:03.691939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:34:03.691939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:03.691939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:34:03.691939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:03.691939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:34:03.692589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:03.692589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:34:03.692589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:34:03.692589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:34:03.692589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:34:03.692589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:34:03.692589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:34:03.692589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:34:03.692589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:34:03.692589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:34:03.692589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:03.692589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:34:03.692969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:03.692969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:34:03.692969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:03.692969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:34:03.692969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:03.692969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:34:03.692969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:03.692969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:34:03.692969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:34:03.692969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:03.692969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:34:03.692969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:03.692969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:34:03.692969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:34:03.693326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:03.693326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:34:03.693326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:03.693326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:34:03.694028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:03.694028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:34:03.694028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:34:03.694028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:34:03.694028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:34:03.694028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:34:03.694028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:34:03.694028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:34:03.694028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:34:03.694028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:34:03.694028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:34:03.694028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:34:03.694028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:34:03.694028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:34:03.694028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:34:03.694028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:34:03.694476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:34:03.694476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:34:03.694476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:34:03.694476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:34:03.694476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:34:03.694476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:03.694476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:34:03.694476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:03.694476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:34:03.694476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:03.694476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:34:03.694476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:03.694476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:34:03.694476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:03.694476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:34:03.694476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:34:03.694476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:03.694476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:34:03.694984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:03.694984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:34:03.694984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:34:03.694984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:03.694984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:34:03.694984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:03.694984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:34:03.694984 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:34:03.694984 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:34:03.694984 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:34:03.695335 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:34:03.695335 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:34:03.695410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:03.695971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:34:03.695971 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:03.695971 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:34:03.695971 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:34:03.695971 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:03.695971 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:34:03.695971 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:03.695971 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:34:03.696818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:03.696818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:34:03.696818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:34:03.696818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:34:03.696818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:03.696818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:34:03.696818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:03.696818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:34:03.696818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:03.696818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:34:03.696818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:03.696818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:34:03.696818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:03.696818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:34:03.696818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:34:03.697204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:03.697204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:34:03.697204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:03.697204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:34:03.697204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:34:03.697204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:03.697204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:34:03.697204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:03.697204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:34:03.697972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:03.697972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:34:03.697972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:34:03.697972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:34:03.697972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:34:03.697972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:34:03.697972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:34:03.697972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:34:03.697972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:34:03.697972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:03.697972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:34:03.698282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:03.698282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:34:03.698282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:03.698282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:34:03.698282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:03.698282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:34:03.698282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:03.698282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:34:03.698282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:34:03.698282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:03.698282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:34:03.698282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:03.698282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:34:03.698282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:34:03.698699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:03.698699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:34:03.698699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:03.698699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:34:03.699315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:03.699315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:34:03.699315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:34:03.699315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:34:03.699315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:34:03.699315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:34:03.699315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:34:03.699315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:34:03.699315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:34:03.699315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:34:03.699315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:34:03.699315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:34:03.699315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:34:03.699315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:34:03.699315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:34:03.699315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:34:03.699315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:34:03.699785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:34:03.699785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:34:03.699785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:34:03.699785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:03.699785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:34:03.699785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:03.699785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:34:03.699785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:03.699785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:34:03.699785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:03.699785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:34:03.699785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:03.699785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:34:03.699785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:34:03.699785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:03.699785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:34:03.699785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:03.699785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:34:03.699785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:34:03.700454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:03.700454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:34:03.700454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:03.700454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:34:03.701119 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:34:03.701269 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:34:03.701365 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:34:03.701510 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:34:03.702071 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:34:03.703041 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:34:03.703320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:03.704193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:34:03.704193 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:03.704193 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:34:03.704193 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:34:03.704193 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:03.704193 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:34:03.704193 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:03.704193 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:34:03.704940 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:03.704940 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:34:03.704940 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:34:03.704940 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:34:03.704940 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:34:03.704940 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:34:03.704940 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:34:03.704940 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:03.704940 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:34:03.704940 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:03.704940 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:34:03.704940 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:03.705285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:34:03.705285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:03.705285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:34:03.705285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:03.705285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:34:03.705285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:34:03.705285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:03.705285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:34:03.705285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:03.705285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:34:03.705285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:34:03.705285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:03.705285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:34:03.705642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:03.705642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:34:03.705896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:03.705896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:34:03.705896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:34:03.705896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:34:03.705896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:34:03.705896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:34:03.705896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:34:03.705896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:34:03.705896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:34:03.705896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:34:03.705896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:34:03.705896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:34:03.705896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:34:03.705896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:34:03.705896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:34:03.705896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:34:03.706353 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:34:03.706353 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:34:03.706353 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:03.706353 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:34:03.706353 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:03.706353 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:34:03.706353 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:03.706353 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:34:03.706353 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:03.706353 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:34:03.706353 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:03.706353 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:34:03.706353 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:34:03.706353 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:03.706353 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:34:03.706353 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:03.706353 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:34:03.706353 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:34:03.706353 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:03.706929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:34:03.706929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:03.706929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:34:08.539467 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:34:08.540141 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:34:08.540141 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:34:08.541656 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:34:08.575932 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:34:08.577068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:08.578190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:34:08.578190 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:08.578190 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:34:08.578190 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:34:08.578190 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:08.578190 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:34:08.578190 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:08.578190 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:34:08.579310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:08.579310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:34:08.579310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:34:08.579310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:34:08.579310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:34:08.579310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:34:08.579310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:34:08.579310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:08.579310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:34:08.579310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:08.579310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:34:08.579310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:08.579725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:34:08.579725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:08.579725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:34:08.579725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:08.579725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:34:08.579725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:34:08.579725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:08.579725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:34:08.579725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:08.579725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:34:08.579725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:34:08.579725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:08.579725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:34:08.580267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:08.580267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:34:08.580512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:08.580512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:34:08.580512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:34:08.580512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:34:08.580512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:34:08.580512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:34:08.580512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:34:08.580512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:34:08.580512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:34:08.580512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:34:08.580512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:34:08.580512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:34:08.580512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:34:08.580512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:34:08.580512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:34:08.580512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:34:08.581597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:34:08.581597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:34:08.581597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:08.581597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:34:08.581597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:08.581597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:34:08.581597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:08.581597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:34:08.581597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:08.581597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:34:08.581597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:08.581597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:34:08.581597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:34:08.581597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:08.581597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:34:08.581597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:08.581597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:34:08.581597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:34:08.581597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:08.582301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:34:08.582301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:08.582301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:34:08.678503 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:34:08.679150 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:34:08.679299 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:34:08.680282 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:34:08.680502 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:34:08.680640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:08.681400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:34:08.681400 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:08.681400 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:34:08.681400 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:34:08.681400 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:08.681400 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:34:08.681400 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:08.681400 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:34:08.682560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:08.682560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:34:08.682560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:34:08.682560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:34:08.682560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:34:08.682560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:34:08.682560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:34:08.682560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:08.682560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:34:08.682560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:08.682560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:34:08.682560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:08.683146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:34:08.683146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:08.683146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:34:08.683146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:08.683146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:34:08.683146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:34:08.683146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:08.683146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:34:08.683146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:08.683146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:34:08.683146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:34:08.683146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:08.683146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:34:08.683513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:08.683513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:34:08.683612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:08.683612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:34:08.683612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:34:08.683612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:34:08.683612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:34:08.683612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:34:08.683612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:34:08.683612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:34:08.683612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:34:08.683612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:34:08.683612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:34:08.683612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:34:08.683612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:34:08.683612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:34:08.683612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:34:08.683612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:34:08.684270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:34:08.684270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:34:08.684270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:08.684270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:34:08.684270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:08.684270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:34:08.684270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:08.684270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:34:08.684270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:08.684270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:34:08.684270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:08.684270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:34:08.684270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:34:08.684270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:08.684270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:34:08.684270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:08.684270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:34:08.684270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:34:08.684270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:08.684896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:34:08.684896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:08.684896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:34:08.688031 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:34:08.688343 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:34:08.688456 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:34:08.688590 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:34:08.688880 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:34:08.689072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:08.689496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:34:08.689496 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:08.689496 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:34:08.689496 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:34:08.689496 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:08.689496 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:34:08.689496 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:08.689496 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:34:08.690204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:08.690204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:34:08.690204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:34:08.690204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:34:08.690204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:08.690204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:34:08.690204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:08.690204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:34:08.690204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:08.690204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:34:08.690204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:08.690204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:34:08.690204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:08.690204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:34:08.690204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:34:08.690875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:08.690875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:34:08.690875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:08.690875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:34:08.690875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:34:08.690875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:08.690875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:34:08.690875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:08.690875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:34:08.691159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:08.691159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:34:08.691159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:34:08.691159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:34:08.691159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:34:08.691159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:34:08.691159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:34:08.691159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:34:08.691159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:34:08.691159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:34:08.691159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:08.691159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:34:08.691485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:08.691485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:34:08.691485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:08.691485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:34:08.691485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:08.691485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:34:08.691485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:08.691485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:34:08.691485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:34:08.691485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:08.691485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:34:08.691485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:08.691485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:34:08.691485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:34:08.691934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:08.691934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:34:08.691934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:08.691934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:34:08.692157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:08.692157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:34:08.692157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:34:08.692157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:34:08.692157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:34:08.692157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:34:08.692157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:34:08.692157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:34:08.692157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:34:08.692157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:34:08.692157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:34:08.692157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:34:08.692157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:34:08.692157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:34:08.692157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:34:08.692157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:34:08.692587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:34:08.692587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:34:08.692587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:34:08.692587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:34:08.692587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:34:08.692587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:08.692587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:34:08.692587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:08.692587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:34:08.692587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:08.692587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:34:08.692587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:08.692587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:34:08.692587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:08.692587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:34:08.692587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:34:08.692587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:08.692587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:34:08.693071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:08.693071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:34:08.693071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:34:08.693071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:08.693071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:34:08.693071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:08.693071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:34:08.694653 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:34:08.695059 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:34:08.695197 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:34:08.695322 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:34:08.695705 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:34:08.695999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:08.696445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:34:08.696445 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:08.696445 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:34:08.696445 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:34:08.696445 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:08.696445 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:34:08.696445 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:08.696445 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:34:08.697151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:08.697151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:34:08.697151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:34:08.697151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:34:08.697151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:08.697151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:34:08.697151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:08.697151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:34:08.697151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:08.697151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:34:08.697151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:08.697151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:34:08.697151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:08.697151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:34:08.697151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:34:08.698209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:08.698209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:34:08.698209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:08.698209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:34:08.698209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:34:08.698209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:08.698209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:34:08.698209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:08.698209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:34:08.698209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:08.698209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:34:08.698209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:34:08.698209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:34:08.698209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:34:08.698911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:34:08.698911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:34:08.698911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:34:08.698911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:34:08.698911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:08.698911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:34:08.698911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:08.698911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:34:08.698911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:08.698911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:34:08.698911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:08.698911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:34:08.698911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:08.699268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:34:08.699268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:34:08.699268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:08.699268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:34:08.699268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:08.699268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:34:08.699268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:34:08.699268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:08.699268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:34:08.699268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:08.699268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:34:08.699268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:08.699602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:34:08.699602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:34:08.699602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:34:08.699602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:34:08.699602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:34:08.699602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:34:08.699602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:34:08.699602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:34:08.699602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:34:08.699602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:34:08.699602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:34:08.699602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:34:08.699602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:34:08.699602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:34:08.699602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:34:08.699602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:34:08.699602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:34:08.699602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:34:08.699602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:34:08.700158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:08.700158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:34:08.700158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:08.700158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:34:08.700158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:08.700158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:34:08.700158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:08.700158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:34:08.700158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:08.700158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:34:08.700158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:34:08.700158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:08.700158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:34:08.700158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:08.700158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:34:08.700158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:34:08.700158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:08.700158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:34:08.700158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:08.700158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:34:08.701449 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:34:08.701736 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:34:08.701860 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:34:08.702032 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:34:08.702276 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:34:08.702477 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:34:08.702618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:08.703393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:34:08.703393 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:08.703393 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:34:08.703393 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:34:08.703393 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:08.703393 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:34:08.703393 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:08.703393 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:34:08.704450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:08.704450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:34:08.704450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:34:08.704450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:34:08.704450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:34:08.704450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:34:08.704450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:34:08.704450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:08.704450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:34:08.704450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:08.704450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:34:08.704450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:08.704894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:34:08.704894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:08.704894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:34:08.704894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:08.704894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:34:08.704894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:34:08.704894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:08.704894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:34:08.704894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:08.704894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:34:08.704894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:34:08.704894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:08.704894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:34:08.705236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:08.705236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:34:08.705302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:08.705302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:34:08.705302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:34:08.705302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:34:08.705302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:34:08.705302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:34:08.705302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:34:08.705302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:34:08.705302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:34:08.705302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:34:08.705302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:34:08.705302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:34:08.705302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:34:08.705302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:34:08.705302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:34:08.705302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:34:08.705714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:34:08.705714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:34:08.705714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:08.705714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:34:08.705714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:08.705714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:34:08.705714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:08.705714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:34:08.705714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:08.705714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:34:08.705714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:08.705714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:34:08.705714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:34:08.705714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:08.705714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:34:08.705714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:08.705714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:34:08.705714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:34:08.705714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:08.706232 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:34:08.706232 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:08.706232 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:34:13.540305 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:34:13.541233 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:34:13.541233 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:34:13.542068 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:34:13.586217 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:34:13.587924 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:13.588223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:34:13.588223 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:13.588223 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:34:13.588223 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:34:13.588223 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:13.588223 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:34:13.588223 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:13.588223 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:34:13.589886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.047 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:13.589886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:34:13.589886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:34:13.589886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:34:13.589886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:34:13.589886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:34:13.589886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:34:13.589886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:13.589886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:34:13.589886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:13.589886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:34:13.589886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:13.590646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:34:13.590646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:13.590646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:34:13.590646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:13.590646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:34:13.590646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:34:13.590646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:13.590646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:34:13.590646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:13.590646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:34:13.590646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:34:13.590646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:13.590646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:34:13.591286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:13.591286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:34:13.591286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:13.591286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:34:13.591286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:34:13.591286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:34:13.591286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:34:13.591286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:34:13.591286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:34:13.591286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:34:13.591286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:34:13.591286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:34:13.591286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:34:13.591286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:34:13.591286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:34:13.591930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:34:13.591930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:34:13.591930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:34:13.591930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:34:13.591930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:34:13.591930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:13.591930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:34:13.591930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:13.591930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:34:13.591930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:13.591930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:34:13.591930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:13.591930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:34:13.591930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:13.591930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:34:13.591930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:34:13.591930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:13.591930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:34:13.591930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:13.592649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:34:13.592649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:34:13.592649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:13.592649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:34:13.592649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:13.592649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:34:13.679148 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:34:13.679674 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:34:13.679901 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:34:13.680866 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:34:13.681110 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:34:13.681289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:13.681986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:34:13.681986 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:13.681986 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:34:13.681986 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:34:13.681986 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:13.681986 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:34:13.681986 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:13.681986 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:34:13.683723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:13.683723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:34:13.683723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:34:13.683723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:34:13.683723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:34:13.683723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:34:13.683723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:34:13.683723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:13.683723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:34:13.683723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:13.683723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:34:13.683723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:13.684224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:34:13.684224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:13.684224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:34:13.684224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:13.684224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:34:13.684224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:34:13.684224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:13.684224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:34:13.684224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:13.684224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:34:13.684224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:34:13.684224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:13.684224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:34:13.684539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:13.684539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:34:13.685246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:13.685246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:34:13.685246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:34:13.685246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:34:13.685246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:34:13.685246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:34:13.685246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:34:13.685246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:34:13.685246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:34:13.685246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:34:13.685246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:34:13.685246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:34:13.685246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:34:13.685246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:34:13.685246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:34:13.685246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:34:13.685661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:34:13.685661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:34:13.685661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:13.685661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:34:13.685661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:13.685661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:34:13.685661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:13.685661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:34:13.685661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:13.685661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:34:13.685661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:13.685661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:34:13.685661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:34:13.685661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:13.685661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:34:13.685661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:13.685661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:34:13.685661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:34:13.685661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:13.686664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:34:13.686664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:13.686664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:34:13.688365 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:34:13.688564 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:34:13.688667 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:34:13.688821 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:34:13.689270 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:34:13.689533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:13.690174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:34:13.690174 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:13.690174 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:34:13.690174 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:34:13.690174 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:13.690174 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:34:13.690174 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:13.690174 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:34:13.690983 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:13.690983 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:34:13.690983 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:34:13.690983 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:34:13.690983 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:13.690983 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:34:13.690983 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:13.690983 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:34:13.690983 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:13.690983 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:34:13.690983 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:13.690983 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:34:13.690983 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:13.690983 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:34:13.690983 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:34:13.691369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:13.691369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:34:13.691369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:13.691369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:34:13.691369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:34:13.691369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:13.691369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:34:13.691369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:13.691369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:34:13.691863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:13.691863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:34:13.691863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:34:13.691863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:34:13.691863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:34:13.691863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:34:13.691863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:34:13.691863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:34:13.691863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:34:13.691863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:34:13.691863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:13.691863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:34:13.692196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:13.692196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:34:13.692196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:13.692196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:34:13.692196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:13.692196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:34:13.692196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:13.692196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:34:13.692196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:34:13.692196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:13.692196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:34:13.692196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:13.692196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:34:13.692196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:34:13.692546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:13.692546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:34:13.692546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:13.692546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:34:13.693194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:13.693194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:34:13.693194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:34:13.693194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:34:13.693194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:34:13.693194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:34:13.693194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:34:13.693194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:34:13.693194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:34:13.693194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:34:13.693194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:34:13.693194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:34:13.693194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:34:13.693194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:34:13.693194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:34:13.693194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:34:13.694143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:34:13.694143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:34:13.694143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:34:13.694143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:34:13.694143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:34:13.694143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:13.694143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:34:13.694143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:13.694143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:34:13.694143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:13.694143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:34:13.694143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:13.694143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:34:13.694143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:13.694143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:34:13.694143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:34:13.694143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:13.694143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:34:13.694956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:13.694956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:34:13.694956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:34:13.694956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:13.694956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:34:13.694956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:13.694956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:34:13.695178 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:34:13.695347 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:34:13.695491 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:34:13.696147 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:34:13.696854 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:34:13.697303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:13.698370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:34:13.698370 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:13.698370 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:34:13.698370 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:34:13.698370 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:13.698370 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:34:13.698370 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:13.698370 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:34:13.699650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:13.699650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:34:13.699650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:34:13.699650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:34:13.699650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:13.699650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:34:13.699650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:13.699650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:34:13.699650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:13.699650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:34:13.699650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:13.699650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:34:13.699650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:13.699650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:34:13.699650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:34:13.700465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:13.700465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:34:13.700465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:13.700465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:34:13.700465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:34:13.700465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:13.700465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:34:13.700465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:13.700465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:34:13.700884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:13.700884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:34:13.700884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:34:13.700884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:34:13.700884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:34:13.700884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:34:13.700884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:34:13.700884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:34:13.700884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:34:13.700884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:13.700884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:34:13.701407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:13.701407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:34:13.701407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:13.701407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:34:13.701407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:13.701407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:34:13.701407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:13.701407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:34:13.701407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:34:13.701407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:13.701407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:34:13.701407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:13.701407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:34:13.701407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:34:13.702015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:13.702015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:34:13.702015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:13.702015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:34:13.702015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:13.702015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:34:13.702015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:34:13.702015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:34:13.702015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:34:13.702015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:34:13.702015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:34:13.702015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:34:13.702015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:34:13.702015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:34:13.702699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:34:13.702699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:34:13.702699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:34:13.702699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:34:13.702699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:34:13.702699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:34:13.702699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:34:13.702699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:34:13.702699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:34:13.702699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:34:13.702699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:13.702699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:34:13.702699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:13.702699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:34:13.702699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:13.702699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:34:13.702699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:13.702699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:34:13.702699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:13.705827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:34:13.705827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:34:13.705827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:13.705827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:34:13.705827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:13.705827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:34:13.705827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:34:13.705827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:13.705827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:34:13.705827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:13.705827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:34:13.705827 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:34:13.705827 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:34:13.707177 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:34:13.707177 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:34:13.707177 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:34:13.707177 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:34:13.707177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:13.707177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:34:13.707177 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:13.707177 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:34:13.707177 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:34:13.708011 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:13.708011 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:34:13.708011 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:13.708011 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:34:13.708011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:13.708011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:34:13.708011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:34:13.708011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:34:13.708011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:34:13.708011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:34:13.708011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:34:13.708011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:13.708011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:34:13.708011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:13.708589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:34:13.708589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:13.708589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:34:13.708589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:13.708589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:34:13.708589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:13.708589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:34:13.708589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:34:13.708589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:13.708589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:34:13.708589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:13.708589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:34:13.708589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:34:13.708589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:13.709022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:34:13.709022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:13.709022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:34:13.709022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:13.709022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:34:13.709022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:34:13.709022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:34:13.709022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:34:13.709022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:34:13.709022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:34:13.709022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:34:13.709022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:34:13.709022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:34:13.709022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:34:13.709022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:34:13.709022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:34:13.709325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:34:13.709325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:34:13.709325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:34:13.709325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:34:13.709325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:34:13.709325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:13.709325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:34:13.709325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:13.709325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:34:13.709325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:13.709325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:34:13.709325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:13.709325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:34:13.709325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:13.709325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:34:13.709325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:34:13.709325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:13.709325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:34:13.709325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:13.709686 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:34:13.709686 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:34:13.709686 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:13.709686 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:34:13.709686 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:13.709686 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:34:18.541166 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:34:18.542072 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:34:18.542072 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:34:18.543015 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:34:18.576489 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:34:18.576802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:18.577365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:34:18.577365 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:18.577365 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:34:18.577365 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:34:18.577365 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:18.577365 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:34:18.577365 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:18.577365 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:34:18.578140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:18.578140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:34:18.578140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:34:18.578140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:34:18.578140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:34:18.578140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:34:18.578140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:34:18.578140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:18.578140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:34:18.578140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:18.578140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:34:18.578140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:18.578692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:34:18.578692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:18.578692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:34:18.578692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:18.578692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:34:18.578692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:34:18.578692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:18.578692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:34:18.578692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:18.578692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:34:18.578692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:34:18.578692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:18.578692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:34:18.579205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:18.579205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:34:18.579205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:18.579205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:34:18.579205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:34:18.579205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:34:18.579205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:34:18.579205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:34:18.579205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:34:18.579205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:34:18.579205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:34:18.579205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:34:18.579205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:34:18.579205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:34:18.579205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:34:18.579861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:34:18.579861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:34:18.579861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:34:18.579861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:34:18.579861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:34:18.579861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:18.579861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:34:18.579861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:18.579861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:34:18.579861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:18.579861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:34:18.579861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:18.579861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:34:18.579861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:18.579861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:34:18.579861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:34:18.579861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:18.579861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:34:18.579861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:18.580696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:34:18.580696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:34:18.580696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:18.580696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:34:18.580696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:18.580696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:34:18.679940 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:34:18.680206 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:34:18.680450 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:34:18.681284 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:34:18.681542 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:34:18.681683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:18.682847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:34:18.682847 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:18.682847 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:34:18.682847 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:34:18.682847 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:18.682847 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:34:18.682847 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:18.682847 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:34:18.684442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:18.684442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:34:18.684442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:34:18.684442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:34:18.684442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:34:18.684442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:34:18.684442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:34:18.684442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:18.684442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:34:18.684442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:18.684442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:34:18.684442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:18.685257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:34:18.685257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:18.685257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:34:18.685257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:18.685257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:34:18.685257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:34:18.685257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:18.685257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:34:18.685257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:18.685257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:34:18.685257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:34:18.685257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:18.685257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:34:18.685706 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:18.685706 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:34:18.686912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:18.686912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:34:18.686912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:34:18.686912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:34:18.686912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:34:18.686912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:34:18.686912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:34:18.686912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:34:18.686912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:34:18.686912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:34:18.686912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:34:18.686912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:34:18.686912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:34:18.686912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:34:18.686912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:34:18.686912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:34:18.688143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:34:18.688143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:34:18.688143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:18.688143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:34:18.688143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:18.688143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:34:18.688143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:18.688143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:34:18.688143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:18.688143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:34:18.688143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:18.688143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:34:18.688143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:34:18.688143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:18.688143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:34:18.688143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:18.688143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:34:18.688143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:34:18.688143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:18.689320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:34:18.689320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:18.689320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:34:18.689320 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:34:18.689603 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:34:18.689603 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:34:18.690672 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:34:18.690951 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:34:18.691535 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:18.692728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:34:18.692728 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:18.692728 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:34:18.692728 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:34:18.692728 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:18.692728 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:34:18.692728 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:18.692728 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:34:18.693304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:18.693304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:34:18.693304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:34:18.693304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:34:18.693304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:18.693304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:34:18.693304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:18.693304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:34:18.693304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:18.693304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:34:18.693304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:18.693304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:34:18.693304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:18.693304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:34:18.693304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:34:18.694553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:18.694553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:34:18.694553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:18.694553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:34:18.694553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:34:18.694553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:18.694553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:34:18.694553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:18.694553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:34:18.694553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:18.694553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:34:18.694553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:34:18.694553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:34:18.694553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:34:18.695590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:34:18.695590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:34:18.695590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:34:18.695590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:34:18.695590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:34:18.695590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:18.695590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:34:18.695590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:18.695590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:34:18.695590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:18.695590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:34:18.695590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:18.695590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:34:18.696464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:18.696464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:34:18.696464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:34:18.696464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:18.696464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:34:18.696464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:18.696464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:34:18.696464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:34:18.696464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:18.696464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:34:18.696464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:18.696464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:34:18.696887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:18.696887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:34:18.696887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:34:18.696887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:34:18.696887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:34:18.696887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:34:18.696887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:34:18.696887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:34:18.696887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:34:18.696887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:34:18.696887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:34:18.696887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:34:18.696887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:34:18.696887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:34:18.696887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:34:18.696887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:34:18.697269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:34:18.697269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:34:18.697269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:34:18.697269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:34:18.697269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:34:18.697269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:18.697269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:34:18.697269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:18.697269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:34:18.697269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:18.697269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:34:18.697269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:18.697269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:34:18.697269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:18.697269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:34:18.697269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:34:18.697269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:18.697269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:34:18.698413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:18.698413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:34:18.698413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:34:18.698413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:18.698413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:34:18.698413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:18.698413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:34:18.698413 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:34:18.698413 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:34:18.698413 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:34:18.699099 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:34:18.699099 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:34:18.699099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:18.699099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:34:18.699099 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:18.699099 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:34:18.699099 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:34:18.699099 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:18.699099 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:34:18.699099 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:18.699099 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:34:18.699099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:18.699922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:34:18.699922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:34:18.699922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:34:18.699922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:18.699922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:34:18.699922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:18.699922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:34:18.699922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:18.699922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:34:18.699922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:18.699922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:34:18.699922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:18.699922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:34:18.699922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:34:18.699922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:18.699922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:34:18.699922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:18.700503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:34:18.700503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:34:18.700503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:18.700503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:34:18.700503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:18.700503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:34:18.700503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:18.700503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:34:18.700503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:34:18.700503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:34:18.700503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:34:18.700503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:34:18.700503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:34:18.700870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:34:18.700870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:34:18.700870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:18.700870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:34:18.700870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:18.700870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:34:18.700870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:18.700870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:34:18.700870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:18.700870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:34:18.700870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:18.700870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:34:18.700870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:34:18.701949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:18.701949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:34:18.701949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:18.701949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:34:18.701949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:34:18.701949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:18.701949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:34:18.701949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:18.701949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:34:18.701949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:18.701949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:34:18.701949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:34:18.702192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:34:18.702192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:34:18.702192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:34:18.702192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:34:18.702192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:34:18.702192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:34:18.702192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:34:18.702192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:34:18.702192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:34:18.702192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:34:18.702192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:34:18.702192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:34:18.702192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:34:18.702192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:34:18.702192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:34:18.702192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:34:18.702192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:34:18.702192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:18.702192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:34:18.702830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:18.702830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:34:18.702830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:18.702830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:34:18.702830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:18.702830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:34:18.702830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:18.702830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:34:18.702830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:34:18.702830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:18.702830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:34:18.702830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:18.702830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:34:18.702830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:34:18.702830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:18.702830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:34:18.702830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:18.702830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:34:18.703202 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:34:18.703202 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:34:18.703202 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:34:18.703449 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:34:18.703941 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:34:18.704334 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:34:18.704554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:18.705375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:34:18.705375 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:18.705375 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:34:18.705375 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:34:18.705375 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:18.705375 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:34:18.705375 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:18.705375 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:34:18.706284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:18.706284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:34:18.706284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:34:18.706284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:34:18.706284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:34:18.706284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:34:18.706284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:34:18.706284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:18.706284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:34:18.706284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:18.706284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:34:18.706284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:18.706572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:34:18.706572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:18.706572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:34:18.706572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:18.706572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:34:18.706572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:34:18.706572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:18.706572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:34:18.706572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:18.706572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:34:18.706572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:34:18.706572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:18.706572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:34:18.706830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:18.706830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:34:18.707278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:18.707278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:34:18.707278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:34:18.707278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:34:18.707278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:34:18.707278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:34:18.707278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:34:18.707278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:34:18.707278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:34:18.707278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:34:18.707278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:34:18.707278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:34:18.707278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:34:18.707278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:34:18.707278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:34:18.707278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:34:18.707804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:34:18.707804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:34:18.707804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:18.707804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:34:18.707804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:18.707804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:34:18.707804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:18.707804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:34:18.707804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:18.707804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:34:18.707804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:18.707804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:34:18.707804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:34:18.707804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:18.707804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:34:18.707804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:18.707804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:34:18.707804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:34:18.707804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:18.708314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:34:18.708314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:18.708314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:34:23.541911 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:34:23.542982 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:34:23.542982 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:34:23.544473 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:34:23.580548 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:34:23.580910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:23.581895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:34:23.581895 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:23.581895 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:34:23.581895 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:34:23.581895 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:23.581895 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:34:23.581895 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:23.581895 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:34:23.583829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:23.583829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:34:23.583829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:34:23.583829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:34:23.583829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:34:23.583829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:34:23.583829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:34:23.583829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:23.583829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:34:23.583829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:23.583829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:34:23.583829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:23.584439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:34:23.584439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:23.584439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:34:23.584439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:23.584439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:34:23.584439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:34:23.584439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:23.584439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:34:23.584439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:23.584439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:34:23.584439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:34:23.584439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:23.584439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:34:23.585042 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:23.585042 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:34:23.585042 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:23.585042 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:34:23.585042 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:34:23.585042 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:34:23.585042 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:34:23.585042 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:34:23.585042 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:34:23.585042 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:34:23.585042 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:34:23.585042 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:34:23.585042 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:34:23.585042 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:34:23.585042 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:34:23.585704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:34:23.585704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:34:23.585704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:34:23.585704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:34:23.585704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:34:23.585704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:23.585704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:34:23.585704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:23.585704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:34:23.585704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:23.585704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:34:23.585704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:23.585704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:34:23.585704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:23.585704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:34:23.585704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:34:23.585704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:23.585704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:34:23.585704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:23.586658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:34:23.586658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:34:23.586658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:23.586658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:34:23.586658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:23.586658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:34:23.680578 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:34:23.680907 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:34:23.681005 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:34:23.681569 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:34:23.681776 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:34:23.681962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:23.682489 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:34:23.682489 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:23.682489 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:34:23.682489 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:34:23.682489 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:23.682489 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:34:23.682489 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:23.682489 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:34:23.683274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:23.683274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:34:23.683274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:34:23.683274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:34:23.683274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:34:23.683274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:34:23.683274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:34:23.683274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:23.683274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:34:23.683274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:23.683274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:34:23.683274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:23.683722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:34:23.683722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:23.683722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:34:23.683722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:23.683722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:34:23.683722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:34:23.683722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:23.683722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:34:23.683722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:23.683722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:34:23.683722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:34:23.683722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:23.683722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:34:23.684120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:23.684120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:34:23.684120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:23.684120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:34:23.684120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:34:23.684120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:34:23.684120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:34:23.684120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:34:23.684120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:34:23.684120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:34:23.684120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:34:23.684120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:34:23.684120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:34:23.684120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:34:23.684120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:34:23.684589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:34:23.684589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:34:23.684589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:34:23.684589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:34:23.684589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:34:23.684589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:23.684589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:34:23.684589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:23.684589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:34:23.684589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:23.684589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:34:23.684589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:23.684589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:34:23.684589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:23.684589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:34:23.684589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:34:23.684589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:23.684589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:34:23.684589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:23.685192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:34:23.685192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:34:23.685192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:23.685192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:34:23.685192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:23.685192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:34:23.689540 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:34:23.689824 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:34:23.689945 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:34:23.690088 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:34:23.690491 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:34:23.690757 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:23.691438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:34:23.691438 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:23.691438 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:34:23.691438 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:34:23.691438 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:23.691438 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:34:23.691438 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:23.691438 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:34:23.692450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:23.692450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:34:23.692450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:34:23.692450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:34:23.692450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:23.692450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:34:23.692450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:23.692450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:34:23.692450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:23.692450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:34:23.692450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:23.692450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:34:23.692450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:23.692450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:34:23.692450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:34:23.692977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:23.692977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:34:23.692977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:23.692977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:34:23.692977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:34:23.692977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:23.692977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:34:23.692977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:23.692977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:34:23.693652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:23.693652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:34:23.693652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:34:23.693652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:34:23.693652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:34:23.693652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:34:23.693652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:34:23.693652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:34:23.693652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:34:23.693652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:34:23.693652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:23.693652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:34:23.694056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:23.694056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:34:23.694056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:23.694056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:34:23.694056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:23.694056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:34:23.694056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:23.694056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:34:23.694056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:34:23.694056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:23.694056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:34:23.694056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:23.694056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:34:23.694056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:34:23.694417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:23.694417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:34:23.694417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:23.694417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:34:23.695341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:23.695341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:34:23.695341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:34:23.695341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:34:23.695341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:34:23.695341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:34:23.695341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:34:23.695341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:34:23.695341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:34:23.695341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:34:23.695341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:34:23.695341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:34:23.695341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:34:23.695341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:34:23.695341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:34:23.695341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:34:23.695793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:34:23.695793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:34:23.695793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:34:23.695793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:34:23.695793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:34:23.695793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:23.695793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:34:23.695793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:23.695793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:34:23.695793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:23.695793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:34:23.695793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:23.695793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:34:23.695793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:23.695793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:34:23.695793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:34:23.695793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:23.695793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:34:23.696270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:23.696270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:34:23.696270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:34:23.696270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:23.696270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:34:23.696270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:23.696270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:34:23.696270 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:34:23.696270 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:34:23.696270 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:34:23.696542 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:34:23.696871 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:34:23.697195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:23.697842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:34:23.697842 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:23.697842 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:34:23.697842 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:34:23.697842 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:23.697842 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:34:23.697842 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:23.697842 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:34:23.698628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:23.698628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:34:23.698628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:34:23.698628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:34:23.698628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:23.698628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:34:23.698628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:23.698628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:34:23.698628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:23.698628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:34:23.698628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:23.698628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:34:23.698628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:23.698628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:34:23.698628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:34:23.699073 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:23.699073 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:34:23.699073 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:23.699073 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:34:23.699073 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:34:23.699073 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:23.699073 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:34:23.699073 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:23.699073 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:34:23.699515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:23.699515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:34:23.699515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:34:23.699515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:34:23.699515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:34:23.699515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:34:23.699515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:34:23.699515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:34:23.699515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:34:23.699515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:23.699515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:34:23.699889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:23.699889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:34:23.699889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:23.699889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:34:23.699889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:23.699889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:34:23.699889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:23.699889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:34:23.699889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:34:23.699889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:23.699889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:34:23.699889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:23.699889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:34:23.699889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:34:23.700254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:23.700254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:34:23.700254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:23.700254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:34:23.700627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:23.700627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:34:23.700627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:34:23.700627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:34:23.700627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:34:23.700627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:34:23.700627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:34:23.700627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:34:23.700627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:34:23.700627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:34:23.700627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:34:23.700627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:34:23.700627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:34:23.700627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:34:23.700627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:34:23.700627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:34:23.700627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:34:23.701132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:34:23.701132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:34:23.701132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:34:23.701132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:23.701132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:34:23.701132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:23.701132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:34:23.701132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:23.701132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:34:23.701132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:23.701132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:34:23.701132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:23.701132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:34:23.701132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:34:23.701132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:23.701132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:34:23.701132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:23.701132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:34:23.701132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:34:23.701642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:23.701642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:34:23.701642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:23.701642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:34:23.703147 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:34:23.703399 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:34:23.703520 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:34:23.703739 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:34:23.704067 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:34:23.704298 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:34:23.704448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:23.704944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:34:23.704944 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:23.704944 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:34:23.704944 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:34:23.704944 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:23.704944 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:34:23.704944 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:23.704944 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:34:23.706030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:23.706030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:34:23.706030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:34:23.706030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:34:23.706030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:34:23.706030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:34:23.706030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:34:23.706030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:23.706030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:34:23.706030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:23.706030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:34:23.706030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:23.706432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:34:23.706432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:23.706432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:34:23.706432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:23.706432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:34:23.706432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:34:23.706432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:23.706432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:34:23.706432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:23.706432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:34:23.706432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:34:23.706432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:23.706432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:34:23.706815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:23.706815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:34:23.707341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:23.707341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:34:23.707341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:34:23.707341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:34:23.707341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:34:23.707341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:34:23.707341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:34:23.707341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:34:23.707341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:34:23.707341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:34:23.707341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:34:23.707341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:34:23.707341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:34:23.707341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:34:23.707341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:34:23.707341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:34:23.707857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:34:23.707857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:34:23.707857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:23.707857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:34:23.707857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:23.707857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:34:23.707857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:23.707857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:34:23.707857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:23.707857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:34:23.707857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:23.707857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:34:23.707857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:34:23.707857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:23.707857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:34:23.707857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:23.707857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:34:23.707857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:34:23.707857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:23.708538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:34:23.708538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:23.708538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:34:28.542654 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:34:28.543636 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:34:28.543636 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:34:28.544781 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:34:28.578294 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:34:28.578482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:28.579049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:34:28.579049 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:28.579049 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:34:28.579049 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:34:28.579049 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:28.579049 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:34:28.579049 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:28.579049 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:34:28.580019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:28.580019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:34:28.580019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:34:28.580019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:34:28.580019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:34:28.580019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:34:28.580019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:34:28.580019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:28.580019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:34:28.580019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:28.580019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:34:28.580019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:28.581080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:34:28.581080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:28.581080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:34:28.581080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:28.581080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:34:28.581080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:34:28.581080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:28.581080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:34:28.581080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:28.581080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:34:28.581080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:34:28.581080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:28.581080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:34:28.582083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:28.582083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:34:28.582083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:28.582083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:34:28.582083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:34:28.582083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:34:28.582083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:34:28.582083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:34:28.582083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:34:28.582083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:34:28.582083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:34:28.582083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:34:28.582083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:34:28.582083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:34:28.582083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:34:28.583257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:34:28.583257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:34:28.583257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:34:28.583257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:34:28.583257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:34:28.583257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:28.583257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:34:28.583257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:28.583257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:34:28.583257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:28.583257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:34:28.583257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:28.583257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:34:28.583257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:28.583257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:34:28.583257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:34:28.583257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:28.583257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:34:28.583257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:28.584556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:34:28.584556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:34:28.584556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:28.584556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:34:28.584556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:28.584556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:34:28.681224 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:34:28.681558 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:34:28.681687 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:34:28.682427 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:34:28.682648 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:34:28.682900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:28.683696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:34:28.683696 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:28.683696 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:34:28.683696 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:34:28.683696 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:28.683696 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:34:28.683696 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:28.683696 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:34:28.685152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:28.685152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:34:28.685152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:34:28.685152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:34:28.685152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:34:28.685152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:34:28.685152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:34:28.685152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:28.685152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:34:28.685152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:28.685152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:34:28.685152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:28.685631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:34:28.685631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:28.685631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:34:28.685631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:28.685631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:34:28.685631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:34:28.685631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:28.685631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:34:28.685631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:28.685631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:34:28.685631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:34:28.685631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:28.685631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:34:28.686142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:28.686142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:34:28.686792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:28.686792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:34:28.686792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:34:28.686792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:34:28.686792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:34:28.686792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:34:28.686792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:34:28.686792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:34:28.686792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:34:28.686792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:34:28.686792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:34:28.686792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:34:28.686792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:34:28.686792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:34:28.686792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:34:28.686792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:34:28.687426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:34:28.687426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:34:28.687426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:28.687426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:34:28.687426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:28.687426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:34:28.687426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:28.687426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:34:28.687426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:28.687426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:34:28.687426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:28.687426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:34:28.687426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:34:28.687426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:28.687426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:34:28.687426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:28.687426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:34:28.687426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:34:28.687426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:28.688130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:34:28.688130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:28.688130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:34:28.689993 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:34:28.690324 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:34:28.690573 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:34:28.690795 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:34:28.691014 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:34:28.691290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:28.691694 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:34:28.691694 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:28.691694 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:34:28.691694 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:34:28.691694 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:28.691694 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:34:28.691694 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:28.691694 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:34:28.692464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:28.692464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:34:28.692464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:34:28.692464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:34:28.692464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:28.692464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:34:28.692464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:28.692464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:34:28.692464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:28.692464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:34:28.692464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:28.692464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:34:28.692464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:28.692464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:34:28.692464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:34:28.693541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:28.693541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:34:28.693541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:28.693541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:34:28.693541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:34:28.693541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:28.693541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:34:28.693541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:28.693541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:34:28.693541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:28.693541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:34:28.693541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:34:28.693541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:34:28.693541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:34:28.694385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:34:28.694385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:34:28.694385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:34:28.694385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:34:28.694385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:34:28.694385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:28.694385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:34:28.694385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:28.694385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:34:28.694385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:28.694385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:34:28.694385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:28.694385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:34:28.694847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:28.694847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:34:28.694847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:34:28.694847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:28.694847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:34:28.694847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:28.694847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:34:28.694847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:34:28.694847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:28.694847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:34:28.694847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:28.694847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:34:28.695216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:28.695216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:34:28.695216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:34:28.695216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:34:28.695216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:34:28.695216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:34:28.695216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:34:28.695216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:34:28.695216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:34:28.695216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:34:28.695216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:34:28.695216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:34:28.695216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:34:28.695216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:34:28.695216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:34:28.695216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:34:28.695689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:34:28.695689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:34:28.695689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:34:28.695689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:34:28.695689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:34:28.695689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:28.695689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:34:28.695689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:28.695689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:34:28.695689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:28.695689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:34:28.695689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:28.695689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:34:28.695689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:28.695689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:34:28.695689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:34:28.695689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:28.695689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:34:28.696313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:28.696313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:34:28.696313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:34:28.696313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:28.696313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:34:28.696313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:28.696313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:34:28.696313 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:34:28.696566 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:34:28.696566 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:34:28.696635 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:34:28.696902 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:34:28.697272 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:28.697877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:34:28.697877 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:28.697877 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:34:28.697877 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:34:28.697877 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:28.697877 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:34:28.697877 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:28.697877 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:34:28.698824 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:28.698824 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:34:28.698824 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:34:28.698824 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:34:28.698824 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:28.698824 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:34:28.698824 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:28.698824 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:34:28.698824 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:28.698824 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:34:28.698824 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:28.698824 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:34:28.698824 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:28.698824 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:34:28.698824 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:34:28.699365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:28.699365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:34:28.699365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:28.699365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:34:28.699365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:34:28.699365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:28.699365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:34:28.699365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:28.699365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:34:28.699723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:28.699723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:34:28.699723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:34:28.699723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:34:28.699723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:34:28.699723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:34:28.699723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:34:28.699723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:34:28.699723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:34:28.699723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:28.699723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:34:28.700203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:28.700203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:34:28.700203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:28.700203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:34:28.700203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:28.700203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:34:28.700203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:28.700203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:34:28.700203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:34:28.700203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:28.700203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:34:28.700203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:28.700203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:34:28.700203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:34:28.700707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:28.700707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:34:28.700707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:28.700707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:34:28.700951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:28.700951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:34:28.700951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:34:28.700951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:34:28.700951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:34:28.700951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:34:28.700951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:34:28.700951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:34:28.700951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:34:28.700951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:34:28.700951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:34:28.700951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:34:28.700951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:34:28.700951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:34:28.700951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:34:28.700951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:34:28.700951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:34:28.701703 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:34:28.701703 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:34:28.701703 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:34:28.701703 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:28.701703 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:34:28.701703 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:28.701703 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:34:28.701703 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:28.701703 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:34:28.701703 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:28.701703 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:34:28.701703 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:28.701703 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:34:28.701703 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:34:28.701703 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:28.701703 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:34:28.701703 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:28.701703 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:34:28.701703 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:34:28.702513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:28.702513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:34:28.702513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:28.702513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:34:28.703405 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:34:28.703615 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:34:28.703692 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:34:28.703848 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:34:28.704007 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:34:28.704160 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:34:28.704258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:28.704637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:34:28.704637 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:28.704637 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:34:28.704637 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:34:28.704637 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:28.704637 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:34:28.704637 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:28.704637 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:34:28.705223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:28.705223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:34:28.705223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:34:28.705223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:34:28.705223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:34:28.705223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:34:28.705223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:34:28.705223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:28.705223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:34:28.705223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:28.705223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:34:28.705223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:28.705732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:34:28.705732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:28.705732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:34:28.705732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:28.705732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:34:28.705732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:34:28.705732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:28.705732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:34:28.705732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:28.705732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:34:28.705732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:34:28.705732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:28.705732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:34:28.706327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:28.706327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:34:28.706327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:28.706327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:34:28.706327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:34:28.706327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:34:28.706327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:34:28.706327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:34:28.706327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:34:28.706327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:34:28.706327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:34:28.706327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:34:28.706327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:34:28.706327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:34:28.706327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:34:28.706971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:34:28.706971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:34:28.706971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:34:28.706971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:34:28.706971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:34:28.706971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:28.706971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:34:28.706971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:28.706971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:34:28.706971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:28.706971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:34:28.706971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:28.706971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:34:28.706971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:28.706971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:34:28.706971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:34:28.706971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:28.706971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:34:28.706971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:28.707702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:34:28.707702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:34:28.707702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:28.707702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:34:28.707702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:28.707702 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:34:33.543349 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:34:33.544315 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:34:33.544457 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:34:33.545610 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:34:33.580592 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:34:33.580981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:33.581662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:34:33.581662 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:33.581662 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:34:33.581662 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:34:33.581662 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:33.581662 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:34:33.581662 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:33.581662 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:34:33.582646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:33.582646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:34:33.582646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:34:33.582646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:34:33.582646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:34:33.582646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:34:33.582646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:34:33.582646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:33.582646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:34:33.582646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:33.582646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:34:33.582646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:33.583077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:34:33.583077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:33.583077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:34:33.583077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:33.583077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:34:33.583077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:34:33.583077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:33.583077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:34:33.583077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:33.583077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:34:33.583077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:34:33.583077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:33.583077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:34:33.583457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:33.583457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:34:33.583758 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:33.583758 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:34:33.583758 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:34:33.583758 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:34:33.583758 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:34:33.583758 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:34:33.583758 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:34:33.583758 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:34:33.583758 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:34:33.583758 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:34:33.583758 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:34:33.583758 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:34:33.583758 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:34:33.583758 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:34:33.583758 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:34:33.583758 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:34:33.584235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:34:33.584235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:34:33.584235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:33.584235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:34:33.584235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:33.584235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:34:33.584235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:33.584235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:34:33.584235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:33.584235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:34:33.584235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:33.584235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:34:33.584235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:34:33.584235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:33.584235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:34:33.584235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:33.584235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:34:33.584235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:34:33.584235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:33.584822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:34:33.584822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:33.584822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:34:33.681880 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:34:33.682568 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:34:33.683118 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:34:33.684257 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:34:33.684616 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:34:33.684899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:33.685975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:34:33.685975 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:33.685975 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:34:33.685975 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:34:33.685975 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:33.685975 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:34:33.685975 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:33.685975 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:34:33.687306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:33.687306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:34:33.687306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:34:33.687306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:34:33.687306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:34:33.687306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:34:33.687306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:34:33.687306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:33.687306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:34:33.687306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:33.687306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:34:33.687306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:33.687939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:34:33.687939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:33.687939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:34:33.687939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:33.687939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:34:33.687939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:34:33.687939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:33.687939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:34:33.687939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:33.687939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:34:33.687939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:34:33.687939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:33.687939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:34:33.688487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:33.688487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:34:33.688713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:33.688713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:34:33.688713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:34:33.688713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:34:33.688713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:34:33.688713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:34:33.688713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:34:33.688713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:34:33.688713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:34:33.688713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:34:33.688713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:34:33.688713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:34:33.688713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:34:33.688713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:34:33.688713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:34:33.688713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:34:33.689539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:34:33.689539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:34:33.689539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:33.689539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:34:33.689539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:33.689539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:34:33.689539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:33.689539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:34:33.689539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:33.689539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:34:33.689539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:33.689539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:34:33.689539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:34:33.689539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:33.689539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:34:33.689539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:33.689539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:34:33.689539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:34:33.689539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:33.690095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:34:33.690095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:33.690095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:34:33.690410 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:34:33.690842 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:34:33.690952 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:34:33.691077 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:34:33.691430 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:34:33.691639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:33.692227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:34:33.692227 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:33.692227 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:34:33.692227 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:34:33.692227 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:33.692227 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:34:33.692227 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:33.692227 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:34:33.692909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:33.692909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:34:33.692909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:34:33.692909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:34:33.692909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:33.692909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:34:33.692909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:33.692909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:34:33.692909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:33.692909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:34:33.692909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:33.692909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:34:33.692909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:33.692909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:34:33.692909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:34:33.693451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:33.693451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:34:33.693451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:33.693451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:34:33.693451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:34:33.693451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:33.693451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:34:33.693451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:33.693451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:34:33.693779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:33.693779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:34:33.693779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:34:33.693779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:34:33.693779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:34:33.693779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:34:33.693779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:34:33.693779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:34:33.693779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:34:33.693779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:34:33.693779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:33.693779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:34:33.694141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:33.694141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:34:33.694141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:33.694141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:34:33.694141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:33.694141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:34:33.694141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:33.694141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:34:33.694141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:34:33.694141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:33.694141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:34:33.694141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:33.694141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:34:33.694141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:34:33.694531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:33.694531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:34:33.694531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:33.694531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:34:33.694531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:33.694531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:34:33.694531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:34:33.694531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:34:33.694531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:34:33.694531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:34:33.694531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:34:33.694531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:34:33.694531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:34:33.694531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:34:33.694977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:34:33.694977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:34:33.694977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:34:33.694977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:34:33.694977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:34:33.694977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:34:33.694977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:34:33.694977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:34:33.694977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:34:33.694977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:34:33.694977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:34:33.694977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:33.694977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:34:33.694977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:33.694977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:34:33.694977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:33.694977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:34:33.694977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:33.694977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:34:33.695521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:33.695521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:34:33.695521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:34:33.695521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:33.695521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:34:33.695521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:33.695521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:34:33.695521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:34:33.695521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:33.695521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:34:33.695521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:33.695521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:34:33.696596 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:34:33.697867 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:34:33.698048 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:34:33.698959 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:34:33.699555 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:34:33.699796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:33.700665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:34:33.700665 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:33.700665 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:34:33.700665 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:34:33.700665 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:33.700665 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:34:33.700665 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:33.700665 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:34:33.702332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:33.702332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:34:33.702332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:34:33.702332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:34:33.702332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:33.702332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:34:33.702332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:33.702332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:34:33.702332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:33.702332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:34:33.702332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:33.702332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:34:33.702332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:33.702332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:34:33.702332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:34:33.703107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:33.703107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:34:33.703107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:33.703107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:34:33.703107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:34:33.703107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:33.703107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:34:33.703107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:33.703107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:34:33.703905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:33.703905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:34:33.703905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:34:33.703905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:34:33.703905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:34:33.703905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:34:33.703905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:34:33.703905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:34:33.703905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:34:33.703905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:33.703905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:34:33.704479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:33.704479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:34:33.704479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:33.704479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:34:33.704479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:33.704479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:34:33.704479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:33.704479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:34:33.704479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:34:33.704479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:33.704479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:34:33.704479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:33.704479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:34:33.704479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:34:33.705199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:33.705199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:34:33.705199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:33.705199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:34:33.706220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:33.706220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:34:33.706220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:34:33.706220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:34:33.706220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:34:33.706220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:34:33.706220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:34:33.706220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:34:33.706220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:34:33.706220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:34:33.706220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:34:33.706220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:34:33.706220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:34:33.706220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:34:33.706220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:34:33.706220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:34:33.706220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:34:33.706723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:34:33.706723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:34:33.706723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:34:33.706723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:33.706723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:34:33.706723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:33.706723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:34:33.706723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:33.706723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:34:33.706723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:33.706723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:34:33.706723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:33.706723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:34:33.706723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:34:33.706723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:33.706723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:34:33.706723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:33.706723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:34:33.706723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:34:33.707281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:33.707281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:34:33.707281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:33.707281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:34:33.707281 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:34:33.707281 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:34:33.707281 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:34:33.707281 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:34:33.707536 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:34:33.707757 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:34:33.707922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:33.708459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:34:33.708459 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:33.708459 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:34:33.708459 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:34:33.708459 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:33.708459 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:34:33.708459 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:33.708459 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:34:33.709371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:33.709371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:34:33.709371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:34:33.709371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:34:33.709371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:34:33.709371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:34:33.709371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:34:33.709371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:33.709371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:34:33.709371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:33.709371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:34:33.709371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:33.709783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:34:33.709783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:33.709783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:34:33.709783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:33.709783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:34:33.709783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:34:33.709783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:33.709783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:34:33.709783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:33.709783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:34:33.709783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:34:33.709783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:33.709783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:34:33.710167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:33.710167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:34:33.710423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:33.710423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:34:33.710423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:34:33.710423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:34:33.710423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:34:33.710423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:34:33.710423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:34:33.710423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:34:33.710423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:34:33.710423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:34:33.710423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:34:33.710423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:34:33.710423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:34:33.710423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:34:33.710423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:34:33.710423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:34:33.710889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:34:33.710889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:34:33.710889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:33.710889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:34:33.710889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:33.710889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:34:33.710889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:33.710889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:34:33.710889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:33.710889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:34:33.710889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:33.710889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:34:33.710889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:34:33.710889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:33.710889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:34:33.710889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:33.710889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:34:33.710889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:34:33.710889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:33.711640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:34:33.711640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:33.711640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:34:38.544545 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:34:38.545286 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:34:38.545286 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:34:38.546002 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:34:38.590102 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:34:38.590318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:38.590849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:34:38.590849 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:38.590849 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:34:38.590849 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:34:38.590849 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:38.590849 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:34:38.590849 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:38.590849 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:34:38.591898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.046 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:38.591898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:34:38.591898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:34:38.591898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:34:38.591898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:34:38.591898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:34:38.591898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:34:38.591898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:38.591898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:34:38.591898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:38.591898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:34:38.591898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:38.592369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:34:38.592369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:38.592369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:34:38.592369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:38.592369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:34:38.592369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:34:38.592369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:38.592369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:34:38.592369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:38.592369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:34:38.592369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:34:38.592369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:38.592369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:34:38.593122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:38.593122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:34:38.593122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:38.593122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:34:38.593122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:34:38.593122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:34:38.593122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:34:38.593122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:34:38.593122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:34:38.593122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:34:38.593122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:34:38.593122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:34:38.593122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:34:38.593122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:34:38.593122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:34:38.593545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:34:38.593545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:34:38.593545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:34:38.593545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:34:38.593545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:34:38.593545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:38.593545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:34:38.593545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:38.593545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:34:38.593545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:38.593545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:34:38.593545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:38.593545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:34:38.593545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:38.593545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:34:38.593545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:34:38.593545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:38.593545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:34:38.593545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:38.594122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:34:38.594122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:34:38.594122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:38.594122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:34:38.594122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:38.594122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:34:38.682521 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:34:38.683010 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:34:38.683142 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:34:38.683843 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:34:38.684177 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:34:38.684271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:38.685669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:34:38.685669 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:38.685669 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:34:38.685669 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:34:38.685669 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:38.685669 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:34:38.685669 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:38.685669 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:34:38.686969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:38.686969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:34:38.686969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:34:38.686969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:34:38.686969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:34:38.686969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:34:38.686969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:34:38.686969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:38.686969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:34:38.686969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:38.686969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:34:38.686969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:38.687695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:34:38.687695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:38.687695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:34:38.687695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:38.687695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:34:38.687695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:34:38.687695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:38.687695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:34:38.687695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:38.687695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:34:38.687695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:34:38.687695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:38.687695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:34:38.688336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:38.688336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:34:38.688336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:38.688336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:34:38.688336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:34:38.688336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:34:38.688336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:34:38.688336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:34:38.688336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:34:38.688336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:34:38.688336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:34:38.688336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:34:38.688336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:34:38.688336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:34:38.688336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:34:38.689021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:34:38.689021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:34:38.689021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:34:38.689021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:34:38.689021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:34:38.689021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:38.689021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:34:38.689021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:38.689021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:34:38.689021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:38.689021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:34:38.689021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:38.689021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:34:38.689021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:38.689021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:34:38.689021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:34:38.689021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:38.689021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:34:38.689021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:38.691735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:34:38.691735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:34:38.691735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:38.691735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:34:38.691735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:38.691735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:34:38.691735 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:34:38.691735 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:34:38.692769 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:34:38.692769 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:34:38.692769 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:34:38.693015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:38.693510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:34:38.693510 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:38.693510 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:34:38.693510 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:34:38.693510 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:38.693510 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:34:38.693510 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:38.693510 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:34:38.694491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:38.694491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:34:38.694491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:34:38.694491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:34:38.694491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:38.694491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:34:38.694491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:38.694491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:34:38.694491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:38.694491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:34:38.694491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:38.694491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:34:38.694491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:38.694491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:34:38.694491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:34:38.695339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:38.695339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:34:38.695339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:38.695339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:34:38.695339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:34:38.695339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:38.695339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:34:38.695339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:38.695339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:34:38.695856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:38.695856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:34:38.695856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:34:38.695856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:34:38.695856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:34:38.695856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:34:38.695856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:34:38.695856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:34:38.695856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:34:38.695856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:34:38.695856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:38.695856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:34:38.696961 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:38.696961 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:34:38.696961 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:38.696961 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:34:38.696961 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:38.696961 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:34:38.696961 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:38.696961 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:34:38.696961 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:34:38.696961 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:38.696961 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:34:38.696961 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:38.696961 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:34:38.696961 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:34:38.698180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:38.698180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:34:38.698180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:38.698180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:34:38.698180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:38.698180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:34:38.698180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:34:38.698180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:34:38.698180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:34:38.698180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:34:38.698180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:34:38.698180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:34:38.698180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:34:38.698180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:34:38.698905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:34:38.698905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:34:38.698905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:34:38.698905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:34:38.698905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:34:38.698905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:34:38.698905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:34:38.698905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:34:38.698905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:34:38.698905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:34:38.698905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:34:38.698905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:38.698905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:34:38.698905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:38.698905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:34:38.698905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:38.698905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:34:38.698905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:38.698905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:34:38.700509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:38.700509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:34:38.700509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:34:38.700509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:38.700509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:34:38.700509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:38.700509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:34:38.700509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:34:38.700509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:38.700509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:34:38.700509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:38.700509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:34:38.700509 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:34:38.700509 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:34:38.701155 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:34:38.701155 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:34:38.701155 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:34:38.701155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:38.701155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:34:38.701155 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:38.701155 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:34:38.701155 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:34:38.701155 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:38.701155 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:34:38.701155 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:38.701155 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:34:38.701384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:38.701384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:34:38.701384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:34:38.701384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:34:38.701384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:38.701384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:34:38.701384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:38.701384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:34:38.701384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:38.701384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:34:38.701384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:38.701384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:34:38.701384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:38.701384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:34:38.701384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:34:38.702391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:38.702391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:34:38.702391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:38.702391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:34:38.702391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:34:38.702391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:38.702391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:34:38.702391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:38.702391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:34:38.702391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:38.702391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:34:38.702391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:34:38.702391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:34:38.702391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:34:38.703598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:34:38.703598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:34:38.703598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:34:38.703598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:34:38.703598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:38.703598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:34:38.703598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:38.703598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:34:38.703598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:38.703598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:34:38.703598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:38.703598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:34:38.703598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:38.703855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:34:38.703855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:34:38.703855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:38.703855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:34:38.703855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:38.703855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:34:38.703855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:34:38.703855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:38.703855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:34:38.703855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:38.703855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:34:38.703855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:38.704088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:34:38.704088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:34:38.704088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:34:38.704088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:34:38.704088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:34:38.704088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:34:38.704088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:34:38.704088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:34:38.704088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:34:38.704088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:34:38.704088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:34:38.704088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:34:38.704088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:34:38.704088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:34:38.704088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:34:38.704088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:34:38.704088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:34:38.704088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:34:38.704088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:34:38.704441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:38.704441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:34:38.704441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:38.704441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:34:38.704441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:38.704441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:34:38.704441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:38.704441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:34:38.704441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:38.704441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:34:38.704441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:34:38.704441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:38.704441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:34:38.704441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:38.704441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:34:38.704441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:34:38.704441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:38.704441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:34:38.704441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:38.704441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:34:38.707015 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:34:38.707094 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:34:38.707162 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:34:38.707299 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:34:38.707605 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:34:38.707880 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:34:38.708085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:38.708673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:34:38.708673 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:38.708673 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:34:38.708673 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:34:38.708673 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:38.708673 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:34:38.708673 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:38.708673 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:34:38.709453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:38.709453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:34:38.709453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:34:38.709453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:34:38.709453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:34:38.709453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:34:38.709453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:34:38.709453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:38.709453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:34:38.709453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:38.709453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:34:38.709453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:38.710253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:34:38.710253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:38.710253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:34:38.710253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:38.710253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:34:38.710253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:34:38.710253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:38.710253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:34:38.710253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:38.710253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:34:38.710253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:34:38.710253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:38.710253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:34:38.711295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:38.711295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:34:38.711295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:38.711295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:34:38.711295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:34:38.711295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:34:38.711295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:34:38.711295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:34:38.711295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:34:38.711295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:34:38.711295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:34:38.711295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:34:38.711295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:34:38.711295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:34:38.711295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:34:38.711685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:34:38.711685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:34:38.711685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:34:38.711685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:34:38.711685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:34:38.711685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:38.711685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:34:38.711685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:38.711685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:34:38.711685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:38.711685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:34:38.711685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:38.711685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:34:38.711685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:38.711685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:34:38.711685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:34:38.711685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:38.711685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:34:38.711685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:38.712221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:34:38.712221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:34:38.712221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:38.712221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:34:38.712221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:38.712221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:34:43.545242 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:34:43.546391 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:34:43.546391 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:34:43.547460 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:34:43.584055 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:34:43.584308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:43.585220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:34:43.585220 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:43.585220 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:34:43.585220 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:34:43.585220 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:43.585220 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:34:43.585220 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:43.585220 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:34:43.586275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:43.586275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:34:43.586275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:34:43.586275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:34:43.586275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:34:43.586275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:34:43.586275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:34:43.586275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:43.586275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:34:43.586275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:43.586275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:34:43.586275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:43.587247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:34:43.587247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:43.587247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:34:43.587247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:43.587247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:34:43.587247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:34:43.587247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:43.587247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:34:43.587247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:43.587247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:34:43.587247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:34:43.587247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:43.587247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:34:43.588010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:43.588010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:34:43.588010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:43.588010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:34:43.588010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:34:43.588010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:34:43.588010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:34:43.588010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:34:43.588010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:34:43.588010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:34:43.588010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:34:43.588010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:34:43.588010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:34:43.588010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:34:43.588010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:34:43.588974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:34:43.588974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:34:43.588974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:34:43.588974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:34:43.588974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:34:43.588974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:43.588974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:34:43.588974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:43.588974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:34:43.588974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:43.588974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:34:43.588974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:43.588974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:34:43.588974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:43.588974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:34:43.588974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:34:43.588974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:43.588974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:34:43.588974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:43.590071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:34:43.590071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:34:43.590071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:43.590071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:34:43.590071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:43.590071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:34:43.683320 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:34:43.684199 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:34:43.684488 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:34:43.686173 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:34:43.686581 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:34:43.686879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:43.688471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:34:43.688471 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:43.688471 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:34:43.688471 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:34:43.688471 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:43.688471 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:34:43.688471 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:43.688471 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:34:43.691108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:43.691108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:34:43.691108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:34:43.691108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:34:43.691108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:34:43.691108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:34:43.691108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:34:43.691108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:43.691108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:34:43.691108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:43.691108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:34:43.691108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:43.691435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:34:43.691435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:43.691435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:34:43.691435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:43.691435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:34:43.691435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:34:43.691435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:43.691435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:34:43.691435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:43.691435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:34:43.691435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:34:43.691435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:43.691435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:34:43.691768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:43.691768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:34:43.694218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:43.694218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:34:43.694218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:34:43.694218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:34:43.694218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:34:43.694218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:34:43.694218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:34:43.694218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:34:43.694218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:34:43.694218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:34:43.694218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:34:43.694218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:34:43.694218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:34:43.694218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:34:43.694218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:34:43.694218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:34:43.694701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:34:43.694701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:34:43.694701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:43.694701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:34:43.694701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:43.694701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:34:43.694701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:43.694701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:34:43.694701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:43.694701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:34:43.694701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:43.694701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:34:43.694701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:34:43.694701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:43.694701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:34:43.694701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:43.694701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:34:43.694701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:34:43.694701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:43.695354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:34:43.695354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:43.695354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:34:43.695354 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:34:43.695354 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:34:43.695597 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:34:43.695637 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:34:43.696147 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:34:43.696464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:43.697203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:34:43.697203 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:43.697203 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:34:43.697203 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:34:43.697203 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:43.697203 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:34:43.697203 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:43.697203 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:34:43.698551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:43.698551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:34:43.698551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:34:43.698551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:34:43.698551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:43.698551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:34:43.698551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:43.698551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:34:43.698551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:43.698551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:34:43.698551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:43.698551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:34:43.698551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:43.698551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:34:43.698551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:34:43.699302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:43.699302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:34:43.699302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:43.699302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:34:43.699302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:34:43.699302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:43.699302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:34:43.699302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:43.699302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:34:43.699302 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:34:43.700194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:43.700194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:34:43.700194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:34:43.700194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:34:43.700194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:34:43.700194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:34:43.700194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:34:43.700194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:34:43.700194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:34:43.700194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:34:43.700194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:43.700194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:34:43.700609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:43.700609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:34:43.700609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:43.700609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:34:43.700609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:43.700609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:34:43.700609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:43.700609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:34:43.700609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:34:43.700609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:43.700609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:34:43.700609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:43.700609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:34:43.700609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:34:43.701036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:43.701036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:34:43.701036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:43.701036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:34:43.701483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:43.701483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:34:43.701483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:34:43.701483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:34:43.701483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:34:43.701483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:34:43.701483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:34:43.701483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:34:43.701483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:34:43.701483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:34:43.701483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:34:43.701483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:34:43.701483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:34:43.701483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:34:43.701483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:34:43.701483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:34:43.702012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:34:43.702012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:34:43.702012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:34:43.702012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:34:43.702012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:34:43.702012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:43.702012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:34:43.702012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:43.702012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:34:43.702012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:43.702012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:34:43.702012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:43.702012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:34:43.702012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:43.702012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:34:43.702012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:34:43.702012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:43.702012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:34:43.702545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:43.702545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:34:43.702545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:34:43.702545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:43.702545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:34:43.702545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:43.702545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:34:43.702545 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:34:43.702545 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:34:43.702545 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:34:43.702885 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:34:43.703052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:43.703723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:34:43.703723 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:43.703723 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:34:43.703723 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:34:43.703723 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:43.703723 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:34:43.703723 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:43.703723 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:34:43.704805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:43.704805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:34:43.704805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:34:43.704805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:34:43.704805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:43.704805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:34:43.704805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:43.704805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:34:43.704805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:43.704805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:34:43.704805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:43.704805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:34:43.704805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:43.704805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:34:43.704805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:34:43.705515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:43.705515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:34:43.705515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:43.705515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:34:43.705515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:34:43.705515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:43.705515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:34:43.705515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:43.705515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:34:43.705812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:43.705812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:34:43.705812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:34:43.705812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:34:43.705812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:34:43.705812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:34:43.705812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:34:43.705812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:34:43.705812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:34:43.705812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:43.705812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:34:43.706219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:43.706219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:34:43.706219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:43.706219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:34:43.706219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:43.706219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:34:43.706219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:43.706219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:34:43.706219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:34:43.706219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:43.706219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:34:43.706219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:43.706219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:34:43.706219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:34:43.706587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:43.706587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:34:43.706587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:43.706587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:34:43.707030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:43.707030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:34:43.707030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:34:43.707030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:34:43.707030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:34:43.707030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:34:43.707030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:34:43.707030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:34:43.707030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:34:43.707030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:34:43.707030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:34:43.707030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:34:43.707030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:34:43.707030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:34:43.707030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:34:43.707030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:34:43.707030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:34:43.707521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:34:43.707521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:34:43.707521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:34:43.707521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:43.707521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:34:43.707521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:43.707521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:34:43.707521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:43.707521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:34:43.707521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:43.707521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:34:43.707521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:43.707521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:34:43.707521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:34:43.707521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:43.707521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:34:43.707521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:43.707521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:34:43.707521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:34:43.708138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:43.708138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:34:43.708138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:43.708138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:34:43.708138 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:34:43.708138 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:34:43.708138 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:34:43.708138 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:34:43.708467 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:34:43.708654 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:34:43.708852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:43.709438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:34:43.709438 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:43.709438 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:34:43.709438 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:34:43.709438 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:43.709438 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:34:43.709438 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:43.709438 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:34:43.710425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:43.710425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:34:43.710425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:34:43.710425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:34:43.710425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:34:43.710425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:34:43.710425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:34:43.710425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:43.710425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:34:43.710425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:43.710425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:34:43.710425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:43.710826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:34:43.710826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:43.710826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:34:43.710826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:43.710826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:34:43.710826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:34:43.710826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:43.710826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:34:43.710826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:43.710826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:34:43.710826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:34:43.710826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:43.710826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:34:43.711166 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:43.711166 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:34:43.711996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:43.711996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:34:43.711996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:34:43.711996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:34:43.711996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:34:43.711996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:34:43.711996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:34:43.711996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:34:43.711996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:34:43.711996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:34:43.711996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:34:43.711996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:34:43.711996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:34:43.711996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:34:43.711996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:34:43.711996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:34:43.712483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:34:43.712483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:34:43.712483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:43.712483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:34:43.712483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:43.712483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:34:43.712483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:43.712483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:34:43.712483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:43.712483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:34:43.712483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:43.712483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:34:43.712483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:34:43.712483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:43.712483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:34:43.712483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:43.712483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:34:43.712483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:34:43.712483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:43.713165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:34:43.713165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:43.713165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:34:47.982357 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_for_inconsistencies' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:34:47.983472 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_for_inconsistencies {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:34:47.983703 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Checking Neutron and OVN revision consistency. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:34:47.994329 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Maintenance task: Synchronizing Neutron and OVN databases started {{(pid=106314) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:440}}
May 09 09:34:47.994651 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Maintenance task: Number of inconsistencies found at create/update: networks=4, routers=1, router_ports=1 {{(pid=106314) _log /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:419}}
May 09 09:34:47.994950 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Maintenance task: Fixing resource f8a38fcb-e8a2-421a-a340-bf7f2f5cc72b (type: networks) at create/update {{(pid=106314) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 09 09:34:48.036449 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-f8a38fcb-e8a2-421a-a340-bf7f2f5cc72b, resource={'id': 'f8a38fcb-e8a2-421a-a340-bf7f2f5cc72b', 'name': 'private', 'tenant_id': 'f4d54dfe001d43fbb344093a1ccabd6a', 'project_id': 'f4d54dfe001d43fbb344093a1ccabd6a', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['96e72a03-97b8-412a-b2b9-1bf048ce1066', 'bbd6957a-953f-4fc1-8761-5d93128564d9'], 'standard_attr_id': 12, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-05-09T09:04:34Z', 'updated_at': '2026-05-09T09:04:37Z', 'revision_number': 3, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 3730}, resource_type=networks, if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:34:48.036997 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-f8a38fcb-e8a2-421a-a340-bf7f2f5cc72b, col_values=(('external_ids', {'neutron:network_name': 'private', 'neutron:mtu': '1372', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:34:48.037660 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-78a32c7c-7637-4b83-b0ab-62aee8bb3038, external_ids_update=None, columns={'tag': 3730}, if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:34:48.037786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:48.038956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:34:48.038956 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:48.038956 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:34:48.038956 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:34:48.038956 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:48.038956 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:34:48.038956 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:48.038956 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:34:48.040643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Maintenance task: Failed to fix resource f8a38fcb-e8a2-421a-a340-bf7f2f5cc72b (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:48.040643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:34:48.040643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 09 09:34:48.040643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 09 09:34:48.040643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 09 09:34:48.040643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 09 09:34:48.040643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2332, in update_network
May 09 09:34:48.040643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:34:48.040643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:48.040643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:34:48.040643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:48.041583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:34:48.041583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:48.041583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:34:48.041583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:48.041583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:34:48.041583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:48.041583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:34:48.041583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:34:48.041583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:48.041583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:34:48.041583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:48.041583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:34:48.041583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:34:48.041583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:48.042501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:34:48.042501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:48.042501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:34:48.042501 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Maintenance task: Fixing resource d18a9fac-2ebf-4d1e-a86c-0f33fba8eeda (type: networks) at create/update {{(pid=106314) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 09 09:34:48.077477 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-d18a9fac-2ebf-4d1e-a86c-0f33fba8eeda, resource={'id': 'd18a9fac-2ebf-4d1e-a86c-0f33fba8eeda', 'name': 'public', 'tenant_id': 'f4d54dfe001d43fbb344093a1ccabd6a', 'project_id': 'f4d54dfe001d43fbb344093a1ccabd6a', 'admin_state_up': True, 'mtu': 1430, 'status': 'ACTIVE', 'subnets': ['088bbd0f-7e81-4a76-8399-24d15c984009', 'bbd6a8cc-80e6-4336-a70b-1c962c561590'], 'standard_attr_id': 18, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': True, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'is_default': True, 'tags': [], 'created_at': '2026-05-09T09:04:40Z', 'updated_at': '2026-05-09T09:04:50Z', 'revision_number': 3, 'provider:network_type': 'flat', 'provider:physical_network': 'public', 'provider:segmentation_id': None}, resource_type=networks, if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:34:48.078053 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-d18a9fac-2ebf-4d1e-a86c-0f33fba8eeda, col_values=(('external_ids', {'neutron:network_name': 'public', 'neutron:mtu': '1430', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'flat', 'neutron:provnet-physical-network': 'public'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false', 'fdb_age_threshold': '300', 'broadcast-arps-to-all-routers': 'true'})), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:34:48.078503 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-1163f525-70cd-4b84-ae9b-7d56378305e9, external_ids_update=None, columns={'tag': []}, if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:34:48.078832 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:48.079726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:34:48.079726 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:48.079726 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:34:48.079726 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:34:48.079726 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:48.079726 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:34:48.079726 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:48.079726 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:34:48.081255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Maintenance task: Failed to fix resource d18a9fac-2ebf-4d1e-a86c-0f33fba8eeda (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:48.081255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:34:48.081255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 09 09:34:48.081255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 09 09:34:48.081255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 09 09:34:48.081255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 09 09:34:48.081255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2332, in update_network
May 09 09:34:48.081255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:34:48.081255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:48.081255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:34:48.081255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:48.081621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:34:48.081621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:48.081621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:34:48.081621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:48.081621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:34:48.081621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:48.081621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:34:48.081621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:34:48.081621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:48.081621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:34:48.081621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:48.081621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:34:48.081621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:34:48.081621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:48.082063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:34:48.082063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:48.082063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:34:48.082063 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Maintenance task: Fixing resource 417da935-2882-452f-8ba7-9076d68fcd2b (type: networks) at create/update {{(pid=106314) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 09 09:34:48.110711 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-417da935-2882-452f-8ba7-9076d68fcd2b, resource={'id': '417da935-2882-452f-8ba7-9076d68fcd2b', 'name': 'admin_net', 'tenant_id': '38e536cfc9c94b6cbf529caf55cc7739', 'project_id': '38e536cfc9c94b6cbf529caf55cc7739', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['d3c53d14-f490-4615-a986-3715a315f827'], 'standard_attr_id': 42, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-05-09T09:10:38Z', 'updated_at': '2026-05-09T09:10:39Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 32823}, resource_type=networks, if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:34:48.111051 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-417da935-2882-452f-8ba7-9076d68fcd2b, col_values=(('external_ids', {'neutron:network_name': 'admin_net', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:34:48.111394 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-5738ab2a-b471-447b-9292-7f8c6bc97168, external_ids_update=None, columns={'tag': 32823}, if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:34:48.111517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:48.112427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:34:48.112427 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:48.112427 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:34:48.112427 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:34:48.112427 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:48.112427 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:34:48.112427 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:48.112427 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:34:48.113995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Maintenance task: Failed to fix resource 417da935-2882-452f-8ba7-9076d68fcd2b (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:48.113995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:34:48.113995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 09 09:34:48.113995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 09 09:34:48.113995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 09 09:34:48.113995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 09 09:34:48.113995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2332, in update_network
May 09 09:34:48.113995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:34:48.113995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:48.113995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:34:48.113995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:48.114391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:34:48.114391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:48.114391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:34:48.114391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:48.114391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:34:48.114391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:48.114391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:34:48.114391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:34:48.114391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:48.114391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:34:48.114391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:48.114391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:34:48.114391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:34:48.114391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:48.114768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:34:48.114768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:48.114768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:34:48.114768 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Maintenance task: Fixing resource 433617e8-cdf8-414a-8cc6-63a7b5d0c678 (type: networks) at create/update {{(pid=106314) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 09 09:34:48.149068 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-433617e8-cdf8-414a-8cc6-63a7b5d0c678, resource={'id': '433617e8-cdf8-414a-8cc6-63a7b5d0c678', 'name': 'k8s-clusterapi-cluster-magnum-system-kube-whtb8', 'tenant_id': '3e013d45a44544c3b50eb0e83e4b1efc', 'project_id': '3e013d45a44544c3b50eb0e83e4b1efc', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['9187f065-9264-427a-8df8-672a6981191d'], 'standard_attr_id': 52, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-05-09T09:11:20Z', 'updated_at': '2026-05-09T09:11:20Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 32644}, resource_type=networks, if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:34:48.149352 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-433617e8-cdf8-414a-8cc6-63a7b5d0c678, col_values=(('external_ids', {'neutron:network_name': 'k8s-clusterapi-cluster-magnum-system-kube-whtb8', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:34:48.149626 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-8c3fe586-c182-47ba-9845-d7608113cb71, external_ids_update=None, columns={'tag': 32644}, if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:34:48.149766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:48.150409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:34:48.150409 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:48.150409 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:34:48.150409 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:34:48.150409 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:48.150409 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:34:48.150409 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:48.150409 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:34:48.151560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Maintenance task: Failed to fix resource 433617e8-cdf8-414a-8cc6-63a7b5d0c678 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:48.151560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:34:48.151560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 09 09:34:48.151560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 09 09:34:48.151560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 09 09:34:48.151560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 09 09:34:48.151560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2332, in update_network
May 09 09:34:48.151560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:34:48.151560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:48.151560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:34:48.151560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:48.151906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:34:48.151906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:48.151906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:34:48.151906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:48.151906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:34:48.151906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:48.151906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:34:48.151906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:34:48.151906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:48.151906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:34:48.151906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:48.151906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:34:48.151906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:34:48.151906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:48.152285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:34:48.152285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:48.152285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:34:48.152285 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Maintenance task: Fixing resource cab10384-14c5-492c-8bd0-061f5838310c (type: routers) at create/update {{(pid=106314) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 09 09:34:48.419343 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-cab10384-14c5-492c-8bd0-061f5838310c, resource={'id': 'cab10384-14c5-492c-8bd0-061f5838310c', 'name': 'k8s-clusterapi-cluster-magnum-system-kube-whtb8', 'project_id': '3e013d45a44544c3b50eb0e83e4b1efc', 'admin_state_up': True, 'status': 'ACTIVE', 'enable_snat': True, 'external_gateway_info': {'network_id': 'd18a9fac-2ebf-4d1e-a86c-0f33fba8eeda', 'external_fixed_ips': [{'subnet_id': 'bbd6a8cc-80e6-4336-a70b-1c962c561590', 'ip_address': '172.24.5.116'}, {'subnet_id': '088bbd0f-7e81-4a76-8399-24d15c984009', 'ip_address': '2001:db8::1b7'}], 'enable_snat': True}, 'gw_port_id': 'af118f11-dc01-429a-b20a-e97b77b8d5ec', 'description': 'Created by cluster-api-provider-openstack cluster magnum-system-kube-whtb8', 'availability_zones': [], 'distributed': False, 'ha': True, 'ha_vr_id': 0, 'availability_zone_hints': [], 'enable_default_route_ecmp': False, 'enable_default_route_bfd': False, 'external_gateways': [{'network_id': 'd18a9fac-2ebf-4d1e-a86c-0f33fba8eeda', 'external_fixed_ips': [{'ip_address': '172.24.5.116', 'subnet_id': 'bbd6a8cc-80e6-4336-a70b-1c962c561590'}, {'ip_address': '2001:db8::1b7', 'subnet_id': '088bbd0f-7e81-4a76-8399-24d15c984009'}]}], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2026-05-09T09:11:21Z', 'updated_at': '2026-05-09T09:11:24Z', 'revision_number': 3, 'tenant_id': '3e013d45a44544c3b50eb0e83e4b1efc'}, resource_type=routers, if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:34:48.419612 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterCommand(_result=None, name=neutron-cab10384-14c5-492c-8bd0-061f5838310c, columns={'external_ids': {'neutron:router_name': 'k8s-clusterapi-cluster-magnum-system-kube-whtb8', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': ''}, 'enabled': True}, if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:34:48.419612 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): QoSDelCommand(_result=None, switch=neutron-d18a9fac-2ebf-4d1e-a86c-0f33fba8eeda, conditions=[('direction', '=', 'from-lport'), ('priority', '=', 2002), ('match', '=', 'inport == "af118f11-dc01-429a-b20a-e97b77b8d5ec"')], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:34:48.419831 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): QoSDelCommand(_result=None, switch=neutron-d18a9fac-2ebf-4d1e-a86c-0f33fba8eeda, conditions=[('direction', '=', 'to-lport'), ('priority', '=', 2002), ('match', '=', 'outport == "af118f11-dc01-429a-b20a-e97b77b8d5ec"')], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:34:48.419831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:48.420213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:34:48.420213 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:48.420213 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:34:48.420213 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:34:48.420213 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:48.420213 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:34:48.420213 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:48.420213 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:34:48.420511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_client [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Unable to update router cab10384-14c5-492c-8bd0-061f5838310c. Error: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:48.421374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Maintenance task: Failed to fix resource cab10384-14c5-492c-8bd0-061f5838310c (type: routers): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:48.421374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:34:48.421374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 09 09:34:48.421374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 09 09:34:48.421374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 09 09:34:48.421374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 09 09:34:48.421374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1635, in update_router
May 09 09:34:48.421374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with excutils.save_and_reraise_exception():
May 09 09:34:48.421374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 271, in __exit__
May 09 09:34:48.421374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.force_reraise()
May 09 09:34:48.421374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 233, in force_reraise
May 09 09:34:48.421842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise self.value
May 09 09:34:48.421842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1569, in update_router
May 09 09:34:48.421842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:34:48.421842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:48.421842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:34:48.421842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:48.421842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:34:48.421842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:48.421842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:34:48.421842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:48.421842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:34:48.421842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:48.421842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:34:48.421842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:34:48.422463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:48.422463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:34:48.422463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:48.422463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:34:48.422463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:34:48.422463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:48.422463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:34:48.422463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:48.422463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:34:48.422463 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Maintenance task: Fixing resource af118f11-dc01-429a-b20a-e97b77b8d5ec (type: router_ports) at create/update {{(pid=106314) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 09 09:34:48.530911 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=lrp-af118f11-dc01-429a-b20a-e97b77b8d5ec, resource={'id': 'af118f11-dc01-429a-b20a-e97b77b8d5ec', 'name': '', 'network_id': 'd18a9fac-2ebf-4d1e-a86c-0f33fba8eeda', 'project_id': '', 'mac_address': 'fa:16:3e:51:04:94', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'cab10384-14c5-492c-8bd0-061f5838310c', 'device_owner': 'network:router_gateway', 'standard_attr_id': 57, 'fixed_ips': [{'subnet_id': 'bbd6a8cc-80e6-4336-a70b-1c962c561590', 'ip_address': '172.24.5.116'}, {'subnet_id': '088bbd0f-7e81-4a76-8399-24d15c984009', 'ip_address': '2001:db8::1b7'}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'np0000173033', 'binding:vif_type': 'ovs', 'binding:vif_details': {'port_filter': True, 'connectivity': 'l2', 'ovs_create_tap': False, 'bridge_name': 'br-int', 'datapath_type': 'system', 'bound_drivers': {'0': 'ovn'}}, 'port_security_enabled': False, 'tags': [], 'created_at': '2026-05-09T09:11:21Z', 'updated_at': '2026-05-09T09:11:24Z', 'revision_number': 5, 'tenant_id': ''}, resource_type=router_ports, if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:34:48.531151 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterPortCommand(_result=None, name=lrp-af118f11-dc01-429a-b20a-e97b77b8d5ec, columns={'external_ids': {'neutron:revision_number': '5', 'neutron:subnet_ids': 'bbd6a8cc-80e6-4336-a70b-1c962c561590 088bbd0f-7e81-4a76-8399-24d15c984009', 'neutron:network_name': 'neutron-d18a9fac-2ebf-4d1e-a86c-0f33fba8eeda', 'neutron:is_ext_gw': 'True', 'neutron:router_name': 'neutron-cab10384-14c5-492c-8bd0-061f5838310c'}, 'options': {'reside-on-redirect-chassis': 'true', 'gateway_mtu': '1372'}, 'networks': ['172.24.5.116/24', '2001:db8::1b7/64'], 'ipv6_ra_configs': {}}, if_exists=False) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:34:48.531339 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLRouterPortInLSwitchPortCommand(_result=None, lswitch_port=af118f11-dc01-429a-b20a-e97b77b8d5ec, lrouter_port=lrp-af118f11-dc01-429a-b20a-e97b77b8d5ec, is_gw_port=True, if_exists=True, lsp_address=router) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:34:48.531553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:48.532226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:34:48.532226 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:48.532226 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:34:48.532226 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:34:48.532226 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:48.532226 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:34:48.532226 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:48.532226 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:34:48.533313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Maintenance task: Failed to fix resource af118f11-dc01-429a-b20a-e97b77b8d5ec (type: router_ports): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:48.533313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:34:48.533313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 09 09:34:48.533313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 09 09:34:48.533313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 09 09:34:48.533313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 09 09:34:48.533313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1976, in update_router_port
May 09 09:34:48.533313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:34:48.533313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:48.533313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:34:48.533313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:48.533575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:34:48.533575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:48.533575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:34:48.533575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:48.533575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:34:48.533575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:48.533575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:34:48.533575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:34:48.533575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:48.533575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:34:48.533575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:48.533575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:34:48.533575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:34:48.533575 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:48.533970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:34:48.533970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:48.533970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:34:48.533970 np0000173033 neutron-ovn-maintenance-worker[106314]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_for_inconsistencies finished in 0.550 seconds
May 09 09:34:48.546005 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:34:48.546434 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:34:48.546564 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:34:48.547472 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:34:48.577391 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:34:48.577601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:48.578028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:34:48.578028 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:48.578028 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:34:48.578028 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:34:48.578028 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:48.578028 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:34:48.578028 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:48.578028 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:34:48.578898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.031 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:48.578898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:34:48.578898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:34:48.578898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:34:48.578898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:34:48.578898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:34:48.578898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:34:48.578898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:48.578898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:34:48.578898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:48.578898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:34:48.578898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:48.579843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:34:48.579843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:48.579843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:34:48.579843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:48.579843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:34:48.579843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:34:48.579843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:48.579843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:34:48.579843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:48.579843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:34:48.579843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:34:48.579843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:48.579843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:34:48.580705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:48.580705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:34:48.580705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:48.580705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:34:48.580705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:34:48.580705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:34:48.580705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:34:48.580705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:34:48.580705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:34:48.580705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:34:48.580705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:34:48.580705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:34:48.580705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:34:48.580705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:34:48.580705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:34:48.582091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:34:48.582091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:34:48.582091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:34:48.582091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:34:48.582091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:34:48.582091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:48.582091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:34:48.582091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:48.582091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:34:48.582091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:48.582091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:34:48.582091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:48.582091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:34:48.582091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:48.582091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:34:48.582091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:34:48.582091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:48.582091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:34:48.582091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:48.583398 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:34:48.583398 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:34:48.583398 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:48.583398 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:34:48.583398 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:48.583398 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:34:48.684088 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:34:48.684420 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:34:48.684575 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:34:48.685702 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:34:48.685895 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:34:48.686050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:48.686730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:34:48.686730 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:48.686730 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:34:48.686730 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:34:48.686730 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:48.686730 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:34:48.686730 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:48.686730 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:34:48.687713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:48.687713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:34:48.687713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:34:48.687713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:34:48.687713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:34:48.687713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:34:48.687713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:34:48.687713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:48.687713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:34:48.687713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:48.687713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:34:48.687713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:48.688378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:34:48.688378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:48.688378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:34:48.688378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:48.688378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:34:48.688378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:34:48.688378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:48.688378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:34:48.688378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:48.688378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:34:48.688378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:34:48.688378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:48.688378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:34:48.689002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:48.689002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:34:48.689002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:48.689002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:34:48.689002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:34:48.689002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:34:48.689002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:34:48.689002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:34:48.689002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:34:48.689002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:34:48.689002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:34:48.689002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:34:48.689002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:34:48.689002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:34:48.689002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:34:48.689777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:34:48.689777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:34:48.689777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:34:48.689777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:34:48.689777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:34:48.689777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:48.689777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:34:48.689777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:48.689777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:34:48.689777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:48.689777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:34:48.689777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:48.689777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:34:48.689777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:48.689777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:34:48.689777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:34:48.689777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:48.689777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:34:48.689777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:48.690719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:34:48.690719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:34:48.690719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:48.690719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:34:48.690719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:48.690719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:34:48.695206 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:34:48.695471 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:34:48.695601 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:34:48.695723 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:34:48.696082 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:34:48.696330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:48.696932 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:34:48.696932 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:48.696932 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:34:48.696932 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:34:48.696932 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:48.696932 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:34:48.696932 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:48.696932 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:34:48.697612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:48.697612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:34:48.697612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:34:48.697612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:34:48.697612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:48.697612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:34:48.697612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:48.697612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:34:48.697612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:48.697612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:34:48.697612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:48.697612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:34:48.697612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:48.697612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:34:48.697612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:34:48.698012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:48.698012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:34:48.698012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:48.698012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:34:48.698012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:34:48.698012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:48.698012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:34:48.698012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:48.698012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:34:48.698448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:48.698448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:34:48.698448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:34:48.698448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:34:48.698448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:34:48.698448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:34:48.698448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:34:48.698448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:34:48.698448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:34:48.698448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:34:48.698448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:48.698448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:34:48.698756 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:48.698756 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:34:48.698756 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:48.698756 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:34:48.698756 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:48.698756 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:34:48.698756 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:48.698756 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:34:48.698756 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:34:48.698756 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:48.698756 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:34:48.698756 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:48.698756 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:34:48.698756 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:34:48.699049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:48.699049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:34:48.699049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:48.699049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:34:48.699497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:48.699497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:34:48.699497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:34:48.699497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:34:48.699497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:34:48.699497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:34:48.699497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:34:48.699497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:34:48.699497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:34:48.699497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:34:48.699497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:34:48.699497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:34:48.699497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:34:48.699497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:34:48.699497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:34:48.699497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:34:48.699827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:34:48.699827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:34:48.699827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:34:48.699827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:34:48.699827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:34:48.699827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:48.699827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:34:48.699827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:48.699827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:34:48.699827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:48.699827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:34:48.699827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:48.699827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:34:48.699827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:48.699827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:34:48.699827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:34:48.699827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:48.699827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:34:48.700213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:48.700213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:34:48.700213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:34:48.700213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:48.700213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:34:48.700213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:48.700213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:34:48.701759 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:34:48.702087 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:34:48.702135 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:34:48.702338 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:34:48.702476 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:34:48.702774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:48.703272 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:34:48.703272 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:48.703272 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:34:48.703272 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:34:48.703272 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:48.703272 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:34:48.703272 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:48.703272 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:34:48.704237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:48.704237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:34:48.704237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:34:48.704237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:34:48.704237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:48.704237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:34:48.704237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:48.704237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:34:48.704237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:48.704237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:34:48.704237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:48.704237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:34:48.704237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:48.704237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:34:48.704237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:34:48.705085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:48.705085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:34:48.705085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:48.705085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:34:48.705085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:34:48.705085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:48.705085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:34:48.705085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:48.705085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:34:48.705085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:48.705085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:34:48.705085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:34:48.705085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:34:48.705085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:34:48.705848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:34:48.705848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:34:48.705848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:34:48.705848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:34:48.705848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:48.705848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:34:48.705848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:48.705848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:34:48.705848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:48.705848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:34:48.705848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:48.705848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:34:48.705848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:48.706476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:34:48.706476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:34:48.706476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:48.706476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:34:48.706476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:48.706476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:34:48.706476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:34:48.706476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:48.706476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:34:48.706476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:48.706476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:34:48.706476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:48.707181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:34:48.707181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:34:48.707181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:34:48.707181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:34:48.707181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:34:48.707181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:34:48.707181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:34:48.707181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:34:48.707181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:34:48.707181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:34:48.707181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:34:48.707181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:34:48.707181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:34:48.707181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:34:48.707181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:34:48.707181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:34:48.707181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:34:48.707181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:34:48.707181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:34:48.707696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:48.707696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:34:48.707696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:48.707696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:34:48.707696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:48.707696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:34:48.707696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:48.707696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:34:48.707696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:48.707696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:34:48.707696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:34:48.707696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:48.707696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:34:48.707696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:48.707696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:34:48.707696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:34:48.707696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:48.707696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:34:48.707696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:48.707696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:34:48.708354 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:34:48.708354 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:34:48.708354 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:34:48.708462 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:34:48.708658 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:34:48.708888 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:34:48.709031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:48.709512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:34:48.709512 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:48.709512 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:34:48.709512 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:34:48.709512 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:48.709512 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:34:48.709512 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:48.709512 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:34:48.710408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:48.710408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:34:48.710408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:34:48.710408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:34:48.710408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:34:48.710408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:34:48.710408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:34:48.710408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:48.710408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:34:48.710408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:48.710408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:34:48.710408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:48.710851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:34:48.710851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:48.710851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:34:48.710851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:48.710851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:34:48.710851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:34:48.710851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:48.710851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:34:48.710851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:48.710851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:34:48.710851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:34:48.710851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:48.710851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:34:48.711196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:48.711196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:34:48.711464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:48.711464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:34:48.711464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:34:48.711464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:34:48.711464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:34:48.711464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:34:48.711464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:34:48.711464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:34:48.711464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:34:48.711464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:34:48.711464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:34:48.711464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:34:48.711464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:34:48.711464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:34:48.711464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:34:48.711464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:34:48.711894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:34:48.711894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:34:48.711894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:48.711894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:34:48.711894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:48.711894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:34:48.711894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:48.711894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:34:48.711894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:48.711894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:34:48.711894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:48.711894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:34:48.711894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:34:48.711894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:48.711894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:34:48.711894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:48.711894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:34:48.711894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:34:48.711894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:48.712400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:34:48.712400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:48.712400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:34:53.546615 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:34:53.547720 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:34:53.547720 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:34:53.548248 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:34:53.587626 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:34:53.588040 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:53.588971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:34:53.588971 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:53.588971 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:34:53.588971 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:34:53.588971 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:53.588971 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:34:53.588971 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:53.588971 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:34:53.590079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:53.590079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:34:53.590079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:34:53.590079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:34:53.590079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:34:53.590079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:34:53.590079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:34:53.590079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:53.590079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:34:53.590079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:53.590079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:34:53.590079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:53.590559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:34:53.590559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:53.590559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:34:53.590559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:53.590559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:34:53.590559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:34:53.590559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:53.590559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:34:53.590559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:53.590559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:34:53.590559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:34:53.590559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:53.590559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:34:53.591220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:53.591220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:34:53.591331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:53.591331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:34:53.591331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:34:53.591331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:34:53.591331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:34:53.591331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:34:53.591331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:34:53.591331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:34:53.591331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:34:53.591331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:34:53.591331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:34:53.591331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:34:53.591331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:34:53.591331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:34:53.591331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:34:53.591331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:34:53.591991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:34:53.591991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:34:53.591991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:53.591991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:34:53.591991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:53.591991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:34:53.591991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:53.591991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:34:53.591991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:53.591991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:34:53.591991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:53.591991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:34:53.591991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:34:53.591991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:53.591991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:34:53.591991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:53.591991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:34:53.591991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:34:53.591991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:53.592683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:34:53.592683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:53.592683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:34:53.684794 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:34:53.685565 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:34:53.685777 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:34:53.687112 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:34:53.687587 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:34:53.687946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:53.689214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:34:53.689214 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:53.689214 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:34:53.689214 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:34:53.689214 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:53.689214 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:34:53.689214 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:53.689214 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:34:53.690982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:53.690982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:34:53.690982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:34:53.690982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:34:53.690982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:34:53.690982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:34:53.690982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:34:53.690982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:53.690982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:34:53.690982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:53.690982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:34:53.690982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:53.691969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:34:53.691969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:53.691969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:34:53.691969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:53.691969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:34:53.691969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:34:53.691969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:53.691969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:34:53.691969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:53.691969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:34:53.691969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:34:53.691969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:53.691969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:34:53.692660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:53.692660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:34:53.693017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:53.693017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:34:53.693017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:34:53.693017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:34:53.693017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:34:53.693017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:34:53.693017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:34:53.693017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:34:53.693017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:34:53.693017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:34:53.693017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:34:53.693017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:34:53.693017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:34:53.693017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:34:53.693017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:34:53.693017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:34:53.694107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:34:53.694107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:34:53.694107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:53.694107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:34:53.694107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:53.694107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:34:53.694107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:53.694107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:34:53.694107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:53.694107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:34:53.694107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:53.694107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:34:53.694107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:34:53.694107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:53.694107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:34:53.694107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:53.694107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:34:53.694107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:34:53.694107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:53.695582 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:34:53.695582 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:53.695582 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:34:53.695993 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:34:53.696170 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:34:53.696364 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:34:53.696479 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:34:53.696930 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:34:53.697160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:53.697773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:34:53.697773 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:53.697773 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:34:53.697773 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:34:53.697773 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:53.697773 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:34:53.697773 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:53.697773 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:34:53.698646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:53.698646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:34:53.698646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:34:53.698646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:34:53.698646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:53.698646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:34:53.698646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:53.698646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:34:53.698646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:53.698646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:34:53.698646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:53.698646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:34:53.698646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:53.698646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:34:53.698646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:34:53.699182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:53.699182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:34:53.699182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:53.699182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:34:53.699182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:34:53.699182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:53.699182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:34:53.699182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:53.699182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:34:53.699494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:53.699494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:34:53.699494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:34:53.699494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:34:53.699494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:34:53.699494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:34:53.699494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:34:53.699494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:34:53.699494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:34:53.699494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:34:53.699494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:53.699494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:34:53.699910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:53.699910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:34:53.699910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:53.699910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:34:53.699910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:53.699910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:34:53.699910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:53.699910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:34:53.699910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:34:53.699910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:53.699910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:34:53.699910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:53.699910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:34:53.699910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:34:53.700332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:53.700332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:34:53.700332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:53.700332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:34:53.700652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:53.700652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:34:53.700652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:34:53.700652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:34:53.700652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:34:53.700652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:34:53.700652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:34:53.700652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:34:53.700652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:34:53.700652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:34:53.700652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:34:53.700652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:34:53.700652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:34:53.700652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:34:53.700652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:34:53.700652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:34:53.701137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:34:53.701137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:34:53.701137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:34:53.701137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:34:53.701137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:34:53.701137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:53.701137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:34:53.701137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:53.701137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:34:53.701137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:53.701137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:34:53.701137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:53.701137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:34:53.701137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:53.701137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:34:53.701137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:34:53.701137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:53.701137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:34:53.701676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:53.701676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:34:53.701676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:34:53.701676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:53.701676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:34:53.701676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:53.701676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:34:53.702196 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:34:53.702601 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:34:53.702688 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:34:53.702876 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:34:53.703397 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:34:53.703793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:53.704639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:34:53.704639 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:53.704639 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:34:53.704639 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:34:53.704639 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:53.704639 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:34:53.704639 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:53.704639 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:34:53.705466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:53.705466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:34:53.705466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:34:53.705466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:34:53.705466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:53.705466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:34:53.705466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:53.705466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:34:53.705466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:53.705466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:34:53.705466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:53.705466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:34:53.705466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:53.705466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:34:53.705466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:34:53.705977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:53.705977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:34:53.705977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:53.705977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:34:53.705977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:34:53.705977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:53.705977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:34:53.705977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:53.705977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:34:53.706451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:53.706451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:34:53.706451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:34:53.706451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:34:53.706451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:34:53.706451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:34:53.706451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:34:53.706451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:34:53.706451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:34:53.706451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:53.706451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:34:53.706903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:53.706903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:34:53.706903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:53.706903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:34:53.706903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:53.706903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:34:53.706903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:53.706903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:34:53.706903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:34:53.706903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:53.706903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:34:53.706903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:53.706903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:34:53.706903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:34:53.707299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:53.707299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:34:53.707299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:53.707299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:34:53.707619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:53.707619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:34:53.707619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:34:53.707619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:34:53.707619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:34:53.707619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:34:53.707619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:34:53.707619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:34:53.707619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:34:53.707619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:34:53.707619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:34:53.707619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:34:53.707619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:34:53.707619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:34:53.707619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:34:53.707619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:34:53.707619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:34:53.708238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:34:53.708238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:34:53.708238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:34:53.708238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:53.708238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:34:53.708238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:53.708238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:34:53.708238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:53.708238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:34:53.708238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:53.708238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:34:53.708238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:53.708238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:34:53.708238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:34:53.708238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:53.708238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:34:53.708238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:53.708238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:34:53.708238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:34:53.708918 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:53.708918 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:34:53.708918 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:53.708918 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:34:53.708918 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:34:53.708918 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:34:53.708918 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:34:53.708918 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:34:53.709320 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:34:53.709771 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:34:53.710062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:53.710979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:34:53.710979 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:53.710979 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:34:53.710979 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:34:53.710979 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:53.710979 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:34:53.710979 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:53.710979 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:34:53.711854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:53.711854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:34:53.711854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:34:53.711854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:34:53.711854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:34:53.711854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:34:53.711854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:34:53.711854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:53.711854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:34:53.711854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:53.711854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:34:53.711854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:53.712305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:34:53.712305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:53.712305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:34:53.712305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:53.712305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:34:53.712305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:34:53.712305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:53.712305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:34:53.712305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:53.712305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:34:53.712305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:34:53.712305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:53.712305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:34:53.712888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:53.712888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:34:53.712888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:53.712888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:34:53.712888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:34:53.712888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:34:53.712888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:34:53.712888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:34:53.712888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:34:53.712888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:34:53.712888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:34:53.712888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:34:53.712888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:34:53.712888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:34:53.712888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:34:53.713384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:34:53.713384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:34:53.713384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:34:53.713384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:34:53.713384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:34:53.713384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:53.713384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:34:53.713384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:53.713384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:34:53.713384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:53.713384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:34:53.713384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:53.713384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:34:53.713384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:53.713384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:34:53.713384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:34:53.713384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:53.713384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:34:53.713384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:53.713968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:34:53.713968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:34:53.713968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:53.713968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:34:53.713968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:53.713968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:34:58.547504 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:34:58.548597 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:34:58.549398 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:34:58.550139 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:34:58.554531 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG dbcounter [-] [106314] Writing DB stats neutron:SELECT=434 {{(pid=106314) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 09 09:34:58.590129 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:34:58.590677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:58.591796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:34:58.591796 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:58.591796 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:34:58.591796 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:34:58.591796 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:58.591796 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:34:58.591796 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:58.591796 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:34:58.592596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.043 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:58.592596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:34:58.592596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:34:58.592596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:34:58.592596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:34:58.592596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:34:58.592596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:34:58.592596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:58.592596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:34:58.592596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:58.592596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:34:58.592596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:58.593099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:34:58.593099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:58.593099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:34:58.593099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:58.593099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:34:58.593099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:34:58.593099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:58.593099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:34:58.593099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:58.593099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:34:58.593099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:34:58.593099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:58.593099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:34:58.593631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:58.593631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:34:58.593631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:58.593631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:34:58.593631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:34:58.593631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:34:58.593631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:34:58.593631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:34:58.593631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:34:58.593631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:34:58.593631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:34:58.593631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:34:58.593631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:34:58.593631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:34:58.593631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:34:58.594164 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:34:58.594164 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:34:58.594164 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:34:58.594164 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:34:58.594164 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:34:58.594164 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:58.594164 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:34:58.594164 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:58.594164 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:34:58.594164 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:58.594164 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:34:58.594164 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:58.594164 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:34:58.594164 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:58.594164 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:34:58.594164 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:34:58.594164 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:58.594164 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:34:58.594164 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:58.594832 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:34:58.594832 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:34:58.594832 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:58.594832 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:34:58.594832 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:58.594832 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:34:58.685555 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:34:58.686421 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:34:58.686776 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:34:58.687812 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:34:58.688260 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:34:58.688382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:58.689073 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:34:58.689073 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:58.689073 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:34:58.689073 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:34:58.689073 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:58.689073 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:34:58.689073 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:58.689073 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:34:58.690992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:58.690992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:34:58.690992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:34:58.690992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:34:58.690992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:34:58.690992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:34:58.690992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:34:58.690992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:58.690992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:34:58.690992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:58.690992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:34:58.690992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:58.691431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:34:58.691431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:58.691431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:34:58.691431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:58.691431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:34:58.691431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:34:58.691431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:58.691431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:34:58.691431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:58.691431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:34:58.691431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:34:58.691431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:58.691431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:34:58.691828 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:58.691828 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:34:58.692912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:58.692912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:34:58.692912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:34:58.692912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:34:58.692912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:34:58.692912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:34:58.692912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:34:58.692912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:34:58.692912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:34:58.692912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:34:58.692912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:34:58.692912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:34:58.692912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:34:58.692912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:34:58.692912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:34:58.692912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:34:58.693607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:34:58.693607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:34:58.693607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:58.693607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:34:58.693607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:58.693607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:34:58.693607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:58.693607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:34:58.693607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:58.693607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:34:58.693607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:58.693607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:34:58.693607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:34:58.693607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:58.693607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:34:58.693607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:58.693607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:34:58.693607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:34:58.693607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:58.694789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:34:58.694789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:58.694789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:34:58.696328 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:34:58.696617 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:34:58.696754 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:34:58.696923 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:34:58.697394 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:34:58.697997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:58.698823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:34:58.698823 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:58.698823 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:34:58.698823 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:34:58.698823 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:58.698823 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:34:58.698823 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:58.698823 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:34:58.699520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:58.699520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:34:58.699520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:34:58.699520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:34:58.699520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:58.699520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:34:58.699520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:58.699520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:34:58.699520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:58.699520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:34:58.699520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:58.699520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:34:58.699520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:58.699520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:34:58.699520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:34:58.700035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:58.700035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:34:58.700035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:58.700035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:34:58.700035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:34:58.700035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:58.700035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:34:58.700035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:58.700035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:34:58.700730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:58.700730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:34:58.700730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:34:58.700730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:34:58.700730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:34:58.700730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:34:58.700730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:34:58.700730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:34:58.700730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:34:58.700730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:34:58.700730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:58.700730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:34:58.701114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:58.701114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:34:58.701114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:58.701114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:34:58.701114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:58.701114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:34:58.701114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:58.701114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:34:58.701114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:34:58.701114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:58.701114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:34:58.701114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:58.701114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:34:58.701114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:34:58.701477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:58.701477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:34:58.701477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:58.701477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:34:58.702259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:58.702259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:34:58.702259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:34:58.702259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:34:58.702259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:34:58.702259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:34:58.702259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:34:58.702259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:34:58.702259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:34:58.702259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:34:58.702259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:34:58.702259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:34:58.702259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:34:58.702259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:34:58.702259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:34:58.702259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:34:58.702838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:34:58.702838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:34:58.702838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:34:58.702838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:34:58.702838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:34:58.702838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:58.702838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:34:58.702838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:58.702838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:34:58.702838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:58.702838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:34:58.702838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:58.702838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:34:58.702838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:58.702838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:34:58.702838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:34:58.702838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:58.702838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:34:58.703323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:58.703323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:34:58.703323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:34:58.703323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:58.703323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:34:58.703323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:58.703323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:34:58.703323 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:34:58.703323 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:34:58.703323 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:34:58.703709 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:34:58.703929 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:34:58.704235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:58.704928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:34:58.704928 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:58.704928 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:34:58.704928 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:34:58.704928 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:58.704928 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:34:58.704928 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:58.704928 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:34:58.706007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:58.706007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:34:58.706007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:34:58.706007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:34:58.706007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:58.706007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:34:58.706007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:58.706007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:34:58.706007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:58.706007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:34:58.706007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:58.706007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:34:58.706007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:58.706007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:34:58.706007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:34:58.706554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:58.706554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:34:58.706554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:58.706554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:34:58.706554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:34:58.706554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:58.706554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:34:58.706554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:58.706554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:34:58.707457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:58.707457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:34:58.707457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:34:58.707457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:34:58.707457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:34:58.707457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:34:58.707457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:34:58.707457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:34:58.707457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:34:58.707457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:58.707457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:34:58.708000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:58.708000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:34:58.708000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:58.708000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:34:58.708000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:58.708000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:34:58.708000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:58.708000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:34:58.708000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:34:58.708000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:58.708000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:34:58.708000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:58.708000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:34:58.708000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:34:58.708539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:58.708539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:34:58.708539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:58.708539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:34:58.708985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:58.708985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:34:58.708985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:34:58.708985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:34:58.708985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:34:58.708985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:34:58.708985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:34:58.708985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:34:58.708985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:34:58.708985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:34:58.708985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:34:58.708985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:34:58.708985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:34:58.708985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:34:58.708985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:34:58.708985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:34:58.708985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:34:58.709555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:34:58.709555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:34:58.709555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:34:58.709555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:58.709555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:34:58.709555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:58.709555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:34:58.709555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:58.709555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:34:58.709555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:58.709555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:34:58.709555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:58.709555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:34:58.709555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:34:58.709555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:58.709555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:34:58.709555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:58.709555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:34:58.709555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:34:58.710365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:58.710365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:34:58.710365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:58.710365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:34:58.710365 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:34:58.710365 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:34:58.710365 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:34:58.710606 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:34:58.711142 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:34:58.711391 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:34:58.711554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:58.712417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:34:58.712417 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:58.712417 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:34:58.712417 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:34:58.712417 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:58.712417 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:34:58.712417 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:58.712417 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:34:58.714123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:58.714123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:34:58.714123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:34:58.714123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:34:58.714123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:34:58.714123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:34:58.714123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:34:58.714123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:58.714123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:34:58.714123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:58.714123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:34:58.714123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:58.714511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:34:58.714511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:58.714511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:34:58.714511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:58.714511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:34:58.714511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:34:58.714511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:58.714511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:34:58.714511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:58.714511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:34:58.714511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:34:58.714511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:58.714511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:34:58.714898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:58.714898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:34:58.715377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:58.715377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:34:58.715377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:34:58.715377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:34:58.715377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:34:58.715377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:34:58.715377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:34:58.715377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:34:58.715377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:34:58.715377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:34:58.715377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:34:58.715377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:34:58.715377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:34:58.715377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:34:58.715377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:34:58.715377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:34:58.715892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:34:58.715892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:34:58.715892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:58.715892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:34:58.715892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:34:58.715892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:34:58.715892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:34:58.715892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:34:58.715892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:34:58.715892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:34:58.715892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:34:58.715892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:34:58.715892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:34:58.715892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:34:58.715892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:34:58.715892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:34:58.715892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:34:58.715892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:34:58.715892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:34:58.716412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:34:58.716412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:34:58.716412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:35:03.548483 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:35:03.549379 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:35:03.549379 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:35:03.550029 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:35:03.584377 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:35:03.584762 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:03.585470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:35:03.585470 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:03.585470 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:35:03.585470 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:35:03.585470 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:03.585470 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:35:03.585470 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:03.585470 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:35:03.587313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:03.587313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:35:03.587313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:35:03.587313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:35:03.587313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:35:03.587313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:35:03.587313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:35:03.587313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:03.587313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:35:03.587313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:03.587313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:35:03.587313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:03.588016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:35:03.588016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:03.588016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:35:03.588016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:03.588016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:35:03.588016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:35:03.588016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:03.588016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:35:03.588016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:03.588016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:35:03.588016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:35:03.588016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:03.588016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:35:03.588699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:03.588699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:35:03.588699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:03.588699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:35:03.588699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:35:03.588699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:35:03.588699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:35:03.588699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:35:03.588699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:35:03.588699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:35:03.588699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:35:03.588699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:35:03.588699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:35:03.588699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:35:03.588699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:35:03.589433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:35:03.589433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:35:03.589433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:35:03.589433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:35:03.589433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:35:03.589433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:03.589433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:35:03.589433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:03.589433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:35:03.589433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:03.589433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:35:03.589433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:03.589433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:35:03.589433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:03.589433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:35:03.589433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:35:03.589433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:03.589433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:35:03.589433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:03.591674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:35:03.591674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:35:03.591674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:03.591674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:35:03.591674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:03.591674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:35:03.686227 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:35:03.687113 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:35:03.687374 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:35:03.688635 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:35:03.688881 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:35:03.689063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:03.689851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:35:03.689851 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:03.689851 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:35:03.689851 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:35:03.689851 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:03.689851 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:35:03.689851 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:03.689851 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:35:03.691152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:03.691152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:35:03.691152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:35:03.691152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:35:03.691152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:35:03.691152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:35:03.691152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:35:03.691152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:03.691152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:35:03.691152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:03.691152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:35:03.691152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:03.693186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:35:03.693186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:03.693186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:35:03.693186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:03.693186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:35:03.693186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:35:03.693186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:03.693186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:35:03.693186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:03.693186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:35:03.693186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:35:03.693186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:03.693186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:35:03.695492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:03.695492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:35:03.695492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:03.695492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:35:03.695492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:35:03.695492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:35:03.695492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:35:03.695492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:35:03.695492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:35:03.695492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:35:03.695492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:35:03.695492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:35:03.695492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:35:03.695492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:35:03.695492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:35:03.697620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:35:03.697620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:35:03.697620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:35:03.697620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:35:03.697620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:35:03.697620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:03.697620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:35:03.697620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:03.697620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:35:03.697620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:03.697620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:35:03.697620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:03.697620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:35:03.697620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:03.697620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:35:03.697620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:35:03.697620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:03.697620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:35:03.697620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:03.698643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:35:03.698643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:35:03.698643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:03.698643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:35:03.698643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:03.698643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:35:03.698643 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:35:03.698643 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:35:03.698643 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:35:03.698643 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:35:03.699309 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:35:03.699309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:03.699309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:35:03.699309 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:03.699309 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:35:03.699309 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:35:03.699309 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:03.699309 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:35:03.699309 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:03.699309 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:35:03.699899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:03.699899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:35:03.699899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:35:03.699899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:35:03.699899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:03.699899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:35:03.699899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:03.699899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:35:03.699899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:03.699899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:35:03.699899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:03.699899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:35:03.699899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:03.699899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:35:03.699899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:35:03.700530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:03.700530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:35:03.700530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:03.700530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:35:03.700530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:35:03.700530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:03.700530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:35:03.700530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:03.700530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:35:03.701086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:03.701086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:35:03.701086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:35:03.701086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:35:03.701086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:35:03.701086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:35:03.701086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:35:03.701086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:35:03.701086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:35:03.701086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:35:03.701086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:03.701086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:35:03.701509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:03.701509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:35:03.701509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:03.701509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:35:03.701509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:03.701509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:35:03.701509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:03.701509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:35:03.701509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:35:03.701509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:03.701509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:35:03.701509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:03.701509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:35:03.701509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:35:03.701972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:03.701972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:35:03.701972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:03.701972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:35:03.702442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:03.702442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:35:03.702442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:35:03.702442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:35:03.702442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:35:03.702442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:35:03.702442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:35:03.702442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:35:03.702442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:35:03.702442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:35:03.702442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:35:03.702442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:35:03.702442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:35:03.702442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:35:03.702442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:35:03.702442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:35:03.703424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:35:03.703424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:35:03.703424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:35:03.703424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:35:03.703424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:35:03.703424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:03.703424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:35:03.703424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:03.703424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:35:03.703424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:03.703424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:35:03.703424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:03.703424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:35:03.703424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:03.703424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:35:03.703424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:35:03.703424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:03.703424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:35:03.704252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:03.704252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:35:03.704252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:35:03.704252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:03.704252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:35:03.704252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:03.704252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:35:03.704252 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:35:03.704252 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:35:03.704252 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:35:03.704581 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:35:03.704581 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:35:03.704581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:03.704688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:35:03.704688 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:03.704688 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:35:03.704688 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:35:03.704688 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:03.704688 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:35:03.704688 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:03.704688 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:35:03.705537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:03.705537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:35:03.705537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:35:03.705537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:35:03.705537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:03.705537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:35:03.705537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:03.705537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:35:03.705537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:03.705537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:35:03.705537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:03.705537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:35:03.705537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:03.705537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:35:03.705537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:35:03.705946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:03.705946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:35:03.705946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:03.705946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:35:03.705946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:35:03.705946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:03.705946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:35:03.705946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:03.705946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:35:03.706435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:03.706435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:35:03.706435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:35:03.706435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:35:03.706435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:35:03.706435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:35:03.706435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:35:03.706435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:35:03.706435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:35:03.706435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:03.706435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:35:03.706696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:03.706696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:35:03.706696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:03.706696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:35:03.706696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:03.706696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:35:03.706696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:03.706696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:35:03.706696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:35:03.706696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:03.706696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:35:03.706696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:03.706696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:35:03.706696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:35:03.707039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:03.707039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:35:03.707039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:03.707039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:35:03.707424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:03.707424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:35:03.707424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:35:03.707424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:35:03.707424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:35:03.707424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:35:03.707424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:35:03.707424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:35:03.707424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:35:03.707424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:35:03.707424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:35:03.707424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:35:03.707424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:35:03.707424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:35:03.707424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:35:03.707424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:35:03.707424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:35:03.707829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:35:03.707829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:35:03.707829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:35:03.707829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:03.707829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:35:03.707829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:03.707829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:35:03.707829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:03.707829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:35:03.707829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:03.707829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:35:03.707829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:03.707829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:35:03.707829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:35:03.707829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:03.707829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:35:03.707829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:03.707829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:35:03.707829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:35:03.708206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:03.708206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:35:03.708206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:03.708206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:35:03.709883 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:35:03.710241 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:35:03.710340 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:35:03.710495 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:35:03.710982 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:35:03.711310 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:35:03.711526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:03.712141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:35:03.712141 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:03.712141 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:35:03.712141 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:35:03.712141 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:03.712141 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:35:03.712141 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:03.712141 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:35:03.712927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:03.712927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:35:03.712927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:35:03.712927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:35:03.712927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:35:03.712927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:35:03.712927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:35:03.712927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:03.712927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:35:03.712927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:03.712927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:35:03.712927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:03.713248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:35:03.713248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:03.713248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:35:03.713248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:03.713248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:35:03.713248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:35:03.713248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:03.713248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:35:03.713248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:03.713248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:35:03.713248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:35:03.713248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:03.713248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:35:03.713529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:03.713529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:35:03.714005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:03.714005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:35:03.714005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:35:03.714005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:35:03.714005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:35:03.714005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:35:03.714005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:35:03.714005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:35:03.714005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:35:03.714005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:35:03.714005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:35:03.714005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:35:03.714005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:35:03.714005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:35:03.714005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:35:03.714005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:35:03.714482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:35:03.714482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:35:03.714482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:03.714482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:35:03.714482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:03.714482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:35:03.714482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:03.714482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:35:03.714482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:03.714482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:35:03.714482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:03.714482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:35:03.714482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:35:03.714482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:03.714482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:35:03.714482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:03.714482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:35:03.714482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:35:03.714482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:03.715156 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:35:03.715156 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:03.715156 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:35:08.549253 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:35:08.550135 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:35:08.550135 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:35:08.551540 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:35:08.590319 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:35:08.590489 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:08.591186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:35:08.591186 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:08.591186 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:35:08.591186 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:35:08.591186 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:08.591186 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:35:08.591186 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:08.591186 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:35:08.592327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:08.592327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:35:08.592327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:35:08.592327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:35:08.592327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:35:08.592327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:35:08.592327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:35:08.592327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:08.592327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:35:08.592327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:08.592327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:35:08.592327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:08.593157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:35:08.593157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:08.593157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:35:08.593157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:08.593157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:35:08.593157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:35:08.593157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:08.593157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:35:08.593157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:08.593157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:35:08.593157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:35:08.593157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:08.593157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:35:08.593955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:08.593955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:35:08.593955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:08.593955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:35:08.593955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:35:08.593955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:35:08.593955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:35:08.593955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:35:08.593955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:35:08.593955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:35:08.593955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:35:08.593955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:35:08.593955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:35:08.593955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:35:08.593955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:35:08.595076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:35:08.595076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:35:08.595076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:35:08.595076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:35:08.595076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:35:08.595076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:08.595076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:35:08.595076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:08.595076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:35:08.595076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:08.595076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:35:08.595076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:08.595076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:35:08.595076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:08.595076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:35:08.595076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:35:08.595076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:08.595076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:35:08.595076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:08.596255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:35:08.596255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:35:08.596255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:08.596255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:35:08.596255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:08.596255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:35:08.687087 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:35:08.687738 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:35:08.687908 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:35:08.689199 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:35:08.689535 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:35:08.689900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:08.690706 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:35:08.690706 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:08.690706 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:35:08.690706 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:35:08.690706 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:08.690706 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:35:08.690706 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:08.690706 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:35:08.692240 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:08.692240 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:35:08.692240 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:35:08.692240 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:35:08.692240 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:35:08.692240 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:35:08.692240 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:35:08.692240 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:08.692240 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:35:08.692240 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:08.692240 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:35:08.692240 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:08.693259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:35:08.693259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:08.693259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:35:08.693259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:08.693259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:35:08.693259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:35:08.693259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:08.693259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:35:08.693259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:08.693259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:35:08.693259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:35:08.693259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:08.693259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:35:08.694199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:08.694199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:35:08.694199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:08.694199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:35:08.694199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:35:08.694199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:35:08.694199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:35:08.694199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:35:08.694199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:35:08.694199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:35:08.694199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:35:08.694199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:35:08.694199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:35:08.694199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:35:08.694199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:35:08.695527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:35:08.695527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:35:08.695527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:35:08.695527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:35:08.695527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:35:08.695527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:08.695527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:35:08.695527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:08.695527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:35:08.695527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:08.695527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:35:08.695527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:08.695527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:35:08.695527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:08.695527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:35:08.695527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:35:08.695527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:08.695527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:35:08.695527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:08.696833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:35:08.696833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:35:08.696833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:08.696833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:35:08.696833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:08.696833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:35:08.697401 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:35:08.697712 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:35:08.697844 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:35:08.697962 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:35:08.698344 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:35:08.698574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:08.699160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:35:08.699160 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:08.699160 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:35:08.699160 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:35:08.699160 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:08.699160 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:35:08.699160 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:08.699160 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:35:08.699916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:08.699916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:35:08.699916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:35:08.699916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:35:08.699916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:08.699916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:35:08.699916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:08.699916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:35:08.699916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:08.699916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:35:08.699916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:08.699916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:35:08.699916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:08.699916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:35:08.699916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:35:08.700207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:08.700207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:35:08.700207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:08.700207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:35:08.700207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:35:08.700207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:08.700207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:35:08.700207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:08.700207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:35:08.700823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:08.700823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:35:08.700823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:35:08.700823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:35:08.700823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:35:08.700823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:35:08.700823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:35:08.700823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:35:08.700823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:35:08.700823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:35:08.700823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:08.700823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:35:08.701076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:08.701076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:35:08.701076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:08.701076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:35:08.701076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:08.701076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:35:08.701076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:08.701076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:35:08.701076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:35:08.701076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:08.701076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:35:08.701076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:08.701076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:35:08.701076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:35:08.701324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:08.701324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:35:08.701324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:08.701324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:35:08.701852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:08.701852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:35:08.701852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:35:08.701852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:35:08.701852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:35:08.701852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:35:08.701852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:35:08.701852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:35:08.701852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:35:08.701852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:35:08.701852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:35:08.701852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:35:08.701852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:35:08.701852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:35:08.701852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:35:08.701852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:35:08.702394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:35:08.702394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:35:08.702394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:35:08.702394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:35:08.702394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:35:08.702394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:08.702394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:35:08.702394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:08.702394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:35:08.702394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:08.702394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:35:08.702394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:08.702394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:35:08.702394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:08.702394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:35:08.702394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:35:08.702394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:08.702394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:35:08.702775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:08.702775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:35:08.702775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:35:08.702775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:08.702775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:35:08.702775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:08.702775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:35:08.703558 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:35:08.703940 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:35:08.704051 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:35:08.704429 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:35:08.704909 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:35:08.705171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:08.705708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:35:08.705708 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:08.705708 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:35:08.705708 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:35:08.705708 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:08.705708 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:35:08.705708 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:08.705708 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:35:08.706459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:08.706459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:35:08.706459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:35:08.706459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:35:08.706459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:08.706459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:35:08.706459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:08.706459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:35:08.706459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:08.706459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:35:08.706459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:08.706459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:35:08.706459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:08.706459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:35:08.706459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:35:08.706940 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:08.706940 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:35:08.706940 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:08.706940 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:35:08.706940 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:35:08.706940 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:08.706940 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:35:08.706940 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:08.706940 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:35:08.707314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:08.707314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:35:08.707314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:35:08.707314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:35:08.707314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:35:08.707314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:35:08.707314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:35:08.707314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:35:08.707314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:35:08.707314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:08.707314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:35:08.707642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:08.707642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:35:08.707642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:08.707642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:35:08.707642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:08.707642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:35:08.707642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:08.707642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:35:08.707642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:35:08.707642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:08.707642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:35:08.707642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:08.707642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:35:08.707642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:35:08.708078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:08.708078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:35:08.708078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:08.708078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:35:08.708363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:08.708363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:35:08.708363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:35:08.708363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:35:08.708363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:35:08.708363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:35:08.708363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:35:08.708363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:35:08.708363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:35:08.708363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:35:08.708363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:35:08.708363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:35:08.708363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:35:08.708363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:35:08.708363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:35:08.708363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:35:08.708363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:35:08.708932 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:35:08.708932 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:35:08.708932 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:35:08.708932 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:08.708932 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:35:08.708932 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:08.708932 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:35:08.708932 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:08.708932 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:35:08.708932 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:08.708932 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:35:08.708932 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:08.708932 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:35:08.708932 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:35:08.708932 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:08.708932 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:35:08.708932 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:08.708932 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:35:08.708932 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:35:08.709458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:08.709458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:35:08.709458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:08.709458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:35:08.710438 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:35:08.710767 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:35:08.710893 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:35:08.711043 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:35:08.713354 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:35:08.713354 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:35:08.713354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:08.713548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:35:08.713548 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:08.713548 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:35:08.713548 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:35:08.713548 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:08.713548 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:35:08.713548 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:08.713548 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:35:08.714379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:08.714379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:35:08.714379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:35:08.714379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:35:08.714379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:35:08.714379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:35:08.714379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:35:08.714379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:08.714379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:35:08.714379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:08.714379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:35:08.714379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:08.714788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:35:08.714788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:08.714788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:35:08.714788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:08.714788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:35:08.714788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:35:08.714788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:08.714788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:35:08.714788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:08.714788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:35:08.714788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:35:08.714788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:08.714788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:35:08.715153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:08.715153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:35:08.715346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:08.715346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:35:08.715346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:35:08.715346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:35:08.715346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:35:08.715346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:35:08.715346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:35:08.715346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:35:08.715346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:35:08.715346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:35:08.715346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:35:08.715346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:35:08.715346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:35:08.715346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:35:08.715346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:35:08.715346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:35:08.715842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:35:08.715842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:35:08.715842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:08.715842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:35:08.715842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:08.715842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:35:08.715842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:08.715842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:35:08.715842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:08.715842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:35:08.715842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:08.715842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:35:08.715842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:35:08.715842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:08.715842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:35:08.715842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:08.715842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:35:08.715842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:35:08.715842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:08.716375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:35:08.716375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:08.716375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:35:13.550169 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:35:13.551033 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:35:13.551033 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:35:13.551662 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:35:13.587928 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:35:13.588105 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:13.588551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:35:13.588551 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:13.588551 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:35:13.588551 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:35:13.588551 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:13.588551 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:35:13.588551 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:13.588551 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:35:13.589290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:13.589290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:35:13.589290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:35:13.589290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:35:13.589290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:35:13.589290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:35:13.589290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:35:13.589290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:13.589290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:35:13.589290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:13.589290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:35:13.589290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:13.589967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:35:13.589967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:13.589967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:35:13.589967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:13.589967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:35:13.589967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:35:13.589967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:13.589967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:35:13.589967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:13.589967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:35:13.589967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:35:13.589967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:13.589967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:35:13.590532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:13.590532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:35:13.590532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:13.590532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:35:13.590532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:35:13.590532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:35:13.590532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:35:13.590532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:35:13.590532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:35:13.590532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:35:13.590532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:35:13.590532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:35:13.590532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:35:13.590532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:35:13.590532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:35:13.591197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:35:13.591197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:35:13.591197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:35:13.591197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:35:13.591197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:35:13.591197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:13.591197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:35:13.591197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:13.591197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:35:13.591197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:13.591197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:35:13.591197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:13.591197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:35:13.591197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:13.591197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:35:13.591197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:35:13.591197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:13.591197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:35:13.591197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:13.592021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:35:13.592021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:35:13.592021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:13.592021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:35:13.592021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:13.592021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:35:13.687737 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:35:13.688294 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:35:13.688443 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:35:13.689571 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:35:13.690134 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:35:13.690321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:13.691086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:35:13.691086 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:13.691086 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:35:13.691086 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:35:13.691086 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:13.691086 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:35:13.691086 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:13.691086 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:35:13.692663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:13.692663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:35:13.692663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:35:13.692663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:35:13.692663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:35:13.692663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:35:13.692663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:35:13.692663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:13.692663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:35:13.692663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:13.692663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:35:13.692663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:13.693323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:35:13.693323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:13.693323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:35:13.693323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:13.693323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:35:13.693323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:35:13.693323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:13.693323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:35:13.693323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:13.693323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:35:13.693323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:35:13.693323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:13.693323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:35:13.694012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:13.694012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:35:13.694372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:13.694372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:35:13.694372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:35:13.694372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:35:13.694372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:35:13.694372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:35:13.694372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:35:13.694372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:35:13.694372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:35:13.694372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:35:13.694372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:35:13.694372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:35:13.694372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:35:13.694372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:35:13.694372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:35:13.694372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:35:13.694793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:35:13.694793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:35:13.694793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:13.694793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:35:13.694793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:13.694793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:35:13.694793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:13.694793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:35:13.694793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:13.694793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:35:13.694793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:13.694793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:35:13.694793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:35:13.694793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:13.694793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:35:13.694793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:13.694793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:35:13.694793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:35:13.694793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:13.695276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:35:13.695276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:13.695276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:35:13.697968 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:35:13.698369 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:35:13.698494 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:35:13.698665 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:35:13.699081 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:35:13.699523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:13.700445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:35:13.700445 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:13.700445 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:35:13.700445 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:35:13.700445 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:13.700445 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:35:13.700445 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:13.700445 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:35:13.701308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:13.701308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:35:13.701308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:35:13.701308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:35:13.701308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:13.701308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:35:13.701308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:13.701308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:35:13.701308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:13.701308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:35:13.701308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:13.701308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:35:13.701308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:13.701308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:35:13.701308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:35:13.701722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:13.701722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:35:13.701722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:13.701722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:35:13.701722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:35:13.701722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:13.701722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:35:13.701722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:13.701722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:35:13.702389 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:13.702389 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:35:13.702389 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:35:13.702389 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:35:13.702389 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:35:13.702389 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:35:13.702389 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:35:13.702389 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:35:13.702389 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:35:13.702389 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:35:13.702389 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:13.702389 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:35:13.702736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:13.702736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:35:13.702736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:13.702736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:35:13.702736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:13.702736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:35:13.702736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:13.702736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:35:13.702736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:35:13.702736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:13.702736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:35:13.702736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:13.702736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:35:13.702736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:35:13.703138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:13.703138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:35:13.703138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:13.703138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:35:13.703657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:13.703657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:35:13.703657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:35:13.703657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:35:13.703657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:35:13.703657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:35:13.703657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:35:13.703657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:35:13.703657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:35:13.703657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:35:13.703657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:35:13.703657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:35:13.703657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:35:13.703657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:35:13.703657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:35:13.703657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:35:13.704088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:35:13.704088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:35:13.704088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:35:13.704088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:35:13.704088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:35:13.704088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:13.704088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:35:13.704088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:13.704088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:35:13.704088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:13.704088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:35:13.704088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:13.704088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:35:13.704088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:13.704088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:35:13.704088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:35:13.704088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:13.704088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:35:13.704559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:13.704559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:35:13.704559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:35:13.704559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:13.704559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:35:13.704559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:13.704559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:35:13.704559 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:35:13.704559 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:35:13.704559 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:35:13.704856 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:35:13.705555 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:35:13.706088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:13.707012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:35:13.707012 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:13.707012 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:35:13.707012 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:35:13.707012 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:13.707012 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:35:13.707012 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:13.707012 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:35:13.707835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:13.707835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:35:13.707835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:35:13.707835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:35:13.707835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:13.707835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:35:13.707835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:13.707835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:35:13.707835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:13.707835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:35:13.707835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:13.707835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:35:13.707835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:13.707835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:35:13.707835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:35:13.708246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:13.708246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:35:13.708246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:13.708246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:35:13.708246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:35:13.708246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:13.708246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:35:13.708246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:13.708246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:35:13.708900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:13.708900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:35:13.708900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:35:13.708900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:35:13.708900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:35:13.708900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:35:13.708900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:35:13.708900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:35:13.708900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:35:13.708900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:13.708900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:35:13.709236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:13.709236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:35:13.709236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:13.709236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:35:13.709236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:13.709236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:35:13.709236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:13.709236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:35:13.709236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:35:13.709236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:13.709236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:35:13.709236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:13.709236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:35:13.709236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:35:13.709606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:13.709606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:35:13.709606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:13.709606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:35:13.710309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:13.710309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:35:13.710309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:35:13.710309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:35:13.710309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:35:13.710309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:35:13.710309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:35:13.710309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:35:13.710309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:35:13.710309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:35:13.710309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:35:13.710309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:35:13.710309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:35:13.710309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:35:13.710309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:35:13.710309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:35:13.710309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:35:13.710804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:35:13.710804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:35:13.710804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:35:13.710804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:13.710804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:35:13.710804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:13.710804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:35:13.710804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:13.710804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:35:13.710804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:13.710804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:35:13.710804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:13.710804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:35:13.710804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:35:13.710804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:13.710804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:35:13.710804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:13.710804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:35:13.710804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:35:13.711307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:13.711307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:35:13.711307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:13.711307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:35:13.711307 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:35:13.711307 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:35:13.711307 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:35:13.711515 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:35:13.711915 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:35:13.712148 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:35:13.712305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:13.716325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:35:13.716325 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:13.716325 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:35:13.716325 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:35:13.716325 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:13.716325 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:35:13.716325 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:13.716325 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:35:13.717194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:13.717194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:35:13.717194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:35:13.717194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:35:13.717194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:35:13.717194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:35:13.717194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:35:13.717194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:13.717194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:35:13.717194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:13.717194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:35:13.717194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:13.717562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:35:13.717562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:13.717562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:35:13.717562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:13.717562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:35:13.717562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:35:13.717562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:13.717562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:35:13.717562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:13.717562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:35:13.717562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:35:13.717562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:13.717562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:35:13.718030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:13.718030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:35:13.718470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:13.718470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:35:13.718470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:35:13.718470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:35:13.718470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:35:13.718470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:35:13.718470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:35:13.718470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:35:13.718470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:35:13.718470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:35:13.718470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:35:13.718470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:35:13.718470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:35:13.718470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:35:13.718470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:35:13.718470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:35:13.718905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:35:13.718905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:35:13.718905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:13.718905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:35:13.718905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:13.718905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:35:13.718905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:13.718905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:35:13.718905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:13.718905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:35:13.718905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:13.718905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:35:13.718905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:35:13.718905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:13.718905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:35:13.718905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:13.718905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:35:13.718905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:35:13.718905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:13.719388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:35:13.719388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:13.719388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:35:18.550738 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:35:18.551928 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:35:18.551928 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:35:18.553058 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:35:18.585467 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:35:18.585655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:18.586643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:35:18.586643 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:18.586643 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:35:18.586643 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:35:18.586643 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:18.586643 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:35:18.586643 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:18.586643 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:35:18.587609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:18.587609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:35:18.587609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:35:18.587609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:35:18.587609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:35:18.587609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:35:18.587609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:35:18.587609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:18.587609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:35:18.587609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:18.587609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:35:18.587609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:18.588298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:35:18.588298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:18.588298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:35:18.588298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:18.588298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:35:18.588298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:35:18.588298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:18.588298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:35:18.588298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:18.588298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:35:18.588298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:35:18.588298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:18.588298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:35:18.589079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:18.589079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:35:18.589079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:18.589079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:35:18.589079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:35:18.589079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:35:18.589079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:35:18.589079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:35:18.589079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:35:18.589079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:35:18.589079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:35:18.589079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:35:18.589079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:35:18.589079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:35:18.589079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:35:18.589986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:35:18.589986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:35:18.589986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:35:18.589986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:35:18.589986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:35:18.589986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:18.589986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:35:18.589986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:18.589986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:35:18.589986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:18.589986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:35:18.589986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:18.589986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:35:18.589986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:18.589986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:35:18.589986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:35:18.589986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:18.589986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:35:18.589986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:18.591100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:35:18.591100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:35:18.591100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:18.591100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:35:18.591100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:18.591100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:35:18.688604 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:35:18.688984 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:35:18.689088 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:35:18.691997 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:35:18.692357 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:35:18.692639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:18.693553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:35:18.693553 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:18.693553 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:35:18.693553 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:35:18.693553 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:18.693553 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:35:18.693553 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:18.693553 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:35:18.694489 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:18.694489 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:35:18.694489 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:35:18.694489 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:35:18.694489 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:35:18.694489 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:35:18.694489 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:35:18.694489 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:18.694489 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:35:18.694489 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:18.694489 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:35:18.694489 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:18.695357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:35:18.695357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:18.695357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:35:18.695357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:18.695357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:35:18.695357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:35:18.695357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:18.695357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:35:18.695357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:18.695357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:35:18.695357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:35:18.695357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:18.695357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:35:18.696188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:18.696188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:35:18.696188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:18.696188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:35:18.696188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:35:18.696188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:35:18.696188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:35:18.696188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:35:18.696188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:35:18.696188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:35:18.696188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:35:18.696188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:35:18.696188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:35:18.696188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:35:18.696188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:35:18.697197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:35:18.697197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:35:18.697197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:35:18.697197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:35:18.697197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:35:18.697197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:18.697197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:35:18.697197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:18.697197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:35:18.697197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:18.697197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:35:18.697197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:18.697197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:35:18.697197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:18.697197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:35:18.697197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:35:18.697197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:18.697197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:35:18.697197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:18.698488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:35:18.698488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:35:18.698488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:18.698488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:35:18.698488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:18.698488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:35:18.698488 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:35:18.698837 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:35:18.698899 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:35:18.699045 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:35:18.699342 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:35:18.699569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:18.700096 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:35:18.700096 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:18.700096 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:35:18.700096 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:35:18.700096 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:18.700096 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:35:18.700096 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:18.700096 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:35:18.700615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:18.700615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:35:18.700615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:35:18.700615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:35:18.700615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:18.700615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:35:18.700615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:18.700615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:35:18.700615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:18.700615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:35:18.700615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:18.700615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:35:18.700615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:18.700615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:35:18.700615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:35:18.701279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:18.701279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:35:18.701279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:18.701279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:35:18.701279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:35:18.701279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:18.701279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:35:18.701279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:18.701279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:35:18.701653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:18.701653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:35:18.701653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:35:18.701653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:35:18.701653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:35:18.701653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:35:18.701653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:35:18.701653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:35:18.701653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:35:18.701653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:35:18.701653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:18.701653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:35:18.702201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:18.702201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:35:18.702201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:18.702201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:35:18.702201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:18.702201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:35:18.702201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:18.702201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:35:18.702201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:35:18.702201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:18.702201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:35:18.702201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:18.702201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:35:18.702201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:35:18.702691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:18.702691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:35:18.702691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:18.702691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:35:18.702691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:18.702691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:35:18.702691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:35:18.702691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:35:18.702691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:35:18.702691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:35:18.702691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:35:18.702691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:35:18.702691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:35:18.702691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:35:18.703227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:35:18.703227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:35:18.703227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:35:18.703227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:35:18.703227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:35:18.703227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:35:18.703227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:35:18.703227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:35:18.703227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:35:18.703227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:35:18.703227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:35:18.703227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:18.703227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:35:18.703227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:18.703227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:35:18.703227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:18.703227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:35:18.703227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:18.703227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:35:18.703926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:18.703926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:35:18.703926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:35:18.703926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:18.703926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:35:18.703926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:18.703926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:35:18.703926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:35:18.703926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:18.703926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:35:18.703926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:18.703926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:35:18.704474 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:35:18.704802 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:35:18.704923 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:35:18.705041 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:35:18.705368 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:35:18.705587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:18.706256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:35:18.706256 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:18.706256 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:35:18.706256 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:35:18.706256 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:18.706256 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:35:18.706256 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:18.706256 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:35:18.706954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:18.706954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:35:18.706954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:35:18.706954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:35:18.706954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:18.706954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:35:18.706954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:18.706954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:35:18.706954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:18.706954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:35:18.706954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:18.706954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:35:18.706954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:18.706954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:35:18.706954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:35:18.707309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:18.707309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:35:18.707309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:18.707309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:35:18.707309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:35:18.707309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:18.707309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:35:18.707309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:18.707309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:35:18.707885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:18.707885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:35:18.707885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:35:18.707885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:35:18.707885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:35:18.707885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:35:18.707885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:35:18.707885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:35:18.707885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:35:18.707885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:18.707885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:35:18.708210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:18.708210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:35:18.708210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:18.708210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:35:18.708210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:18.708210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:35:18.708210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:18.708210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:35:18.708210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:35:18.708210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:18.708210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:35:18.708210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:18.708210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:35:18.708210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:35:18.708548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:18.708548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:35:18.708548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:18.708548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:35:18.709043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:18.709043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:35:18.709043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:35:18.709043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:35:18.709043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:35:18.709043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:35:18.709043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:35:18.709043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:35:18.709043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:35:18.709043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:35:18.709043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:35:18.709043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:35:18.709043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:35:18.709043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:35:18.709043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:35:18.709043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:35:18.709043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:35:18.709528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:35:18.709528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:35:18.709528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:35:18.709528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:18.709528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:35:18.709528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:18.709528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:35:18.709528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:18.709528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:35:18.709528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:18.709528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:35:18.709528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:18.709528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:35:18.709528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:35:18.709528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:18.709528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:35:18.709528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:18.709528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:35:18.709528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:35:18.710041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:18.710041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:35:18.710041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:18.710041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:35:18.711377 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:35:18.711684 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:35:18.711826 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:35:18.711981 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:35:18.713568 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:35:18.713980 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:35:18.714157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:18.715039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:35:18.715039 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:18.715039 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:35:18.715039 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:35:18.715039 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:18.715039 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:35:18.715039 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:18.715039 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:35:18.716305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:18.716305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:35:18.716305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:35:18.716305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:35:18.716305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:35:18.716305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:35:18.716305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:35:18.716305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:18.716305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:35:18.716305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:18.716305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:35:18.716305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:18.716988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:35:18.716988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:18.716988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:35:18.716988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:18.716988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:35:18.716988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:35:18.716988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:18.716988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:35:18.716988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:18.716988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:35:18.716988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:35:18.716988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:18.716988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:35:18.717596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:18.717596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:35:18.717596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:18.717596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:35:18.717596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:35:18.717596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:35:18.717596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:35:18.717596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:35:18.717596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:35:18.717596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:35:18.717596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:35:18.717596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:35:18.717596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:35:18.717596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:35:18.717596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:35:18.718666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:35:18.718666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:35:18.718666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:35:18.718666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:35:18.718666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:35:18.718666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:18.718666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:35:18.718666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:18.718666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:35:18.718666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:18.718666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:35:18.718666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:18.718666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:35:18.718666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:18.718666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:35:18.718666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:35:18.718666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:18.718666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:35:18.718666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:18.719564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:35:18.719564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:35:18.719564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:18.719564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:35:18.719564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:18.719564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:35:23.551550 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:35:23.552474 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:35:23.552522 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:35:23.553888 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:35:23.591275 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:35:23.591686 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:23.592621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:35:23.592621 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:23.592621 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:35:23.592621 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:35:23.592621 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:23.592621 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:35:23.592621 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:23.592621 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:35:23.593535 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:23.593535 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:35:23.593535 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:35:23.593535 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:35:23.593535 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:35:23.593535 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:35:23.593535 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:35:23.593535 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:23.593535 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:35:23.593535 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:23.593535 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:35:23.593535 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:23.594266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:35:23.594266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:23.594266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:35:23.594266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:23.594266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:35:23.594266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:35:23.594266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:23.594266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:35:23.594266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:23.594266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:35:23.594266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:35:23.594266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:23.594266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:35:23.594945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:23.594945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:35:23.594945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:23.594945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:35:23.594945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:35:23.594945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:35:23.594945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:35:23.594945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:35:23.594945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:35:23.594945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:35:23.594945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:35:23.594945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:35:23.594945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:35:23.594945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:35:23.594945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:35:23.595873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:35:23.595873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:35:23.595873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:35:23.595873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:35:23.595873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:35:23.595873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:23.595873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:35:23.595873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:23.595873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:35:23.595873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:23.595873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:35:23.595873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:23.595873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:35:23.595873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:23.595873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:35:23.595873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:35:23.595873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:23.595873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:35:23.595873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:23.596853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:35:23.596853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:35:23.596853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:23.596853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:35:23.596853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:23.596853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:35:23.689479 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:35:23.690232 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:35:23.690370 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:35:23.691488 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:35:23.691872 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:35:23.692125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:23.693090 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:35:23.693090 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:23.693090 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:35:23.693090 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:35:23.693090 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:23.693090 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:35:23.693090 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:23.693090 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:35:23.694293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:23.694293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:35:23.694293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:35:23.694293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:35:23.694293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:35:23.694293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:35:23.694293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:35:23.694293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:23.694293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:35:23.694293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:23.694293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:35:23.694293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:23.695396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:35:23.695396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:23.695396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:35:23.695396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:23.695396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:35:23.695396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:35:23.695396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:23.695396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:35:23.695396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:23.695396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:35:23.695396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:35:23.695396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:23.695396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:35:23.696270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:23.696270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:35:23.696270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:23.696270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:35:23.696270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:35:23.696270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:35:23.696270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:35:23.696270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:35:23.696270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:35:23.696270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:35:23.696270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:35:23.696270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:35:23.696270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:35:23.696270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:35:23.696270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:35:23.696976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:35:23.696976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:35:23.696976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:35:23.696976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:35:23.696976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:35:23.696976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:23.696976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:35:23.696976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:23.696976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:35:23.696976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:23.696976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:35:23.696976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:23.696976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:35:23.696976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:23.696976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:35:23.696976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:35:23.696976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:23.696976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:35:23.696976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:23.698209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:35:23.698209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:35:23.698209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:23.698209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:35:23.698209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:23.698209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:35:23.699213 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:35:23.699490 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:35:23.699490 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:35:23.699632 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:35:23.700246 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:35:23.700589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:23.701550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:35:23.701550 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:23.701550 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:35:23.701550 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:35:23.701550 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:23.701550 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:35:23.701550 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:23.701550 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:35:23.702965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:23.702965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:35:23.702965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:35:23.702965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:35:23.702965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:23.702965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:35:23.702965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:23.702965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:35:23.702965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:23.702965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:35:23.702965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:23.702965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:35:23.702965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:23.702965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:35:23.702965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:35:23.703387 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:23.703387 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:35:23.703387 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:23.703387 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:35:23.703387 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:35:23.703387 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:23.703387 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:35:23.703387 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:23.703387 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:35:23.704555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:23.704555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:35:23.704555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:35:23.704555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:35:23.704555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:35:23.704555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:35:23.704555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:35:23.704555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:35:23.704555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:35:23.704555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:35:23.704555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:23.704555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:35:23.704877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:23.704877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:35:23.704877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:23.704877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:35:23.704877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:23.704877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:35:23.704877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:23.704877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:35:23.704877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:35:23.704877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:23.704877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:35:23.704877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:23.704877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:35:23.704877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:35:23.705143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:23.705143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:35:23.705143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:23.705143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:35:23.706435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:23.706435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:35:23.706435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:35:23.706435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:35:23.706435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:35:23.706435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:35:23.706435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:35:23.706435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:35:23.706435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:35:23.706435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:35:23.706435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:35:23.706435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:35:23.706435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:35:23.706435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:35:23.706435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:35:23.706435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:35:23.706770 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:35:23.706770 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:35:23.706770 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:35:23.706770 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:35:23.706770 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:35:23.706770 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:23.706770 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:35:23.706770 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:23.706770 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:35:23.706770 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:23.706770 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:35:23.706770 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:23.706770 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:35:23.706770 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:23.706770 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:35:23.706770 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:35:23.706770 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:23.706770 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:35:23.707120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:23.707120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:35:23.707120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:35:23.707120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:23.707120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:35:23.707120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:23.707120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:35:23.707120 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:35:23.707301 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:35:23.707493 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:35:23.707633 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:35:23.708225 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:35:23.708712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:23.709518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:35:23.709518 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:23.709518 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:35:23.709518 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:35:23.709518 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:23.709518 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:35:23.709518 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:23.709518 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:35:23.710302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:23.710302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:35:23.710302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:35:23.710302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:35:23.710302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:23.710302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:35:23.710302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:23.710302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:35:23.710302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:23.710302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:35:23.710302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:23.710302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:35:23.710302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:23.710302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:35:23.710302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:35:23.710768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:23.710768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:35:23.710768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:23.710768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:35:23.710768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:35:23.710768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:23.710768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:35:23.710768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:23.710768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:35:23.711228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:23.711228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:35:23.711228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:35:23.711228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:35:23.711228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:35:23.711228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:35:23.711228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:35:23.711228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:35:23.711228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:35:23.711228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:23.711228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:35:23.711646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:23.711646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:35:23.711646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:23.711646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:35:23.711646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:23.711646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:35:23.711646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:23.711646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:35:23.711646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:35:23.711646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:23.711646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:35:23.711646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:23.711646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:35:23.711646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:35:23.712017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:23.712017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:35:23.712017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:23.712017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:35:23.712527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:23.712527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:35:23.712527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:35:23.712527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:35:23.712527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:35:23.712527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:35:23.712527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:35:23.712527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:35:23.712527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:35:23.712527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:35:23.712527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:35:23.712527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:35:23.712527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:35:23.712527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:35:23.712527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:35:23.712527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:35:23.712527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:35:23.713081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:35:23.713081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:35:23.713081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:35:23.713081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:23.713081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:35:23.713081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:23.713081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:35:23.713081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:23.713081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:35:23.713081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:23.713081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:35:23.713081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:23.713081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:35:23.713081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:35:23.713081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:23.713081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:35:23.713081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:23.713081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:35:23.713081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:35:23.713629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:23.713629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:35:23.713629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:23.713629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:35:23.713629 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:35:23.713629 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:35:23.713629 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:35:23.713629 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:35:23.714070 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:35:23.714388 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:35:23.714764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:23.715548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:35:23.715548 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:23.715548 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:35:23.715548 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:35:23.715548 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:23.715548 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:35:23.715548 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:23.715548 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:35:23.716444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:23.716444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:35:23.716444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:35:23.716444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:35:23.716444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:35:23.716444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:35:23.716444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:35:23.716444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:23.716444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:35:23.716444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:23.716444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:35:23.716444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:23.716901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:35:23.716901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:23.716901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:35:23.716901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:23.716901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:35:23.716901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:35:23.716901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:23.716901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:35:23.716901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:23.716901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:35:23.716901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:35:23.716901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:23.716901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:35:23.717266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:23.717266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:35:23.717461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:23.717461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:35:23.717461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:35:23.717461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:35:23.717461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:35:23.717461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:35:23.717461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:35:23.717461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:35:23.717461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:35:23.717461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:35:23.717461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:35:23.717461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:35:23.717461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:35:23.717461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:35:23.717461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:35:23.717461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:35:23.717961 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:35:23.717961 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:35:23.717961 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:23.717961 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:35:23.717961 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:23.717961 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:35:23.717961 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:23.717961 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:35:23.717961 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:23.717961 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:35:23.717961 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:23.717961 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:35:23.717961 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:35:23.717961 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:23.717961 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:35:23.717961 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:23.717961 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:35:23.717961 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:35:23.717961 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:23.718464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:35:23.718464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:23.718464 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:35:28.552226 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:35:28.552873 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:35:28.552923 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:35:28.553715 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:35:28.587149 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:35:28.587446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:28.588325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:35:28.588325 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:28.588325 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:35:28.588325 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:35:28.588325 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:28.588325 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:35:28.588325 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:28.588325 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:35:28.589307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:28.589307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:35:28.589307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:35:28.589307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:35:28.589307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:35:28.589307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:35:28.589307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:35:28.589307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:28.589307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:35:28.589307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:28.589307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:35:28.589307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:28.590013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:35:28.590013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:28.590013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:35:28.590013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:28.590013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:35:28.590013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:35:28.590013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:28.590013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:35:28.590013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:28.590013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:35:28.590013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:35:28.590013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:28.590013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:35:28.590690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:28.590690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:35:28.590690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:28.590690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:35:28.590690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:35:28.590690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:35:28.590690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:35:28.590690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:35:28.590690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:35:28.590690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:35:28.590690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:35:28.590690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:35:28.590690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:35:28.590690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:35:28.590690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:35:28.591501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:35:28.591501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:35:28.591501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:35:28.591501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:35:28.591501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:35:28.591501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:28.591501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:35:28.591501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:28.591501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:35:28.591501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:28.591501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:35:28.591501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:28.591501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:35:28.591501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:28.591501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:35:28.591501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:35:28.591501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:28.591501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:35:28.591501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:28.592548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:35:28.592548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:35:28.592548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:28.592548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:35:28.592548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:28.592548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:35:28.690365 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:35:28.691779 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:35:28.691779 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:35:28.691779 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:35:28.691942 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:35:28.692055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:28.693039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:35:28.693039 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:28.693039 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:35:28.693039 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:35:28.693039 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:28.693039 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:35:28.693039 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:28.693039 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:35:28.693893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:28.693893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:35:28.693893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:35:28.693893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:35:28.693893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:35:28.693893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:35:28.693893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:35:28.693893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:28.693893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:35:28.693893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:28.693893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:35:28.693893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:28.694181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:35:28.694181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:28.694181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:35:28.694181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:28.694181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:35:28.694181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:35:28.694181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:28.694181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:35:28.694181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:28.694181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:35:28.694181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:35:28.694181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:28.694181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:35:28.694445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:28.694445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:35:28.694929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:28.694929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:35:28.694929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:35:28.694929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:35:28.694929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:35:28.694929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:35:28.694929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:35:28.694929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:35:28.694929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:35:28.694929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:35:28.694929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:35:28.694929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:35:28.694929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:35:28.694929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:35:28.694929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:35:28.694929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:35:28.695264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:35:28.695264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:35:28.695264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:28.695264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:35:28.695264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:28.695264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:35:28.695264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:28.695264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:35:28.695264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:28.695264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:35:28.695264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:28.695264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:35:28.695264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:35:28.695264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:28.695264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:35:28.695264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:28.695264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:35:28.695264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:35:28.695264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:28.695643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:35:28.695643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:28.695643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:35:28.699585 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:35:28.699966 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:35:28.700108 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:35:28.700231 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:35:28.700597 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:35:28.700872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:28.701417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:35:28.701417 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:28.701417 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:35:28.701417 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:35:28.701417 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:28.701417 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:35:28.701417 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:28.701417 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:35:28.702206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:28.702206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:35:28.702206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:35:28.702206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:35:28.702206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:28.702206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:35:28.702206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:28.702206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:35:28.702206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:28.702206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:35:28.702206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:28.702206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:35:28.702206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:28.702206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:35:28.702206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:35:28.702556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:28.702556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:35:28.702556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:28.702556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:35:28.702556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:35:28.702556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:28.702556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:35:28.702556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:28.702556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:35:28.703058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:28.703058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:35:28.703058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:35:28.703058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:35:28.703058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:35:28.703058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:35:28.703058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:35:28.703058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:35:28.703058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:35:28.703058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:35:28.703058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:28.703058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:35:28.703318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:28.703318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:35:28.703318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:28.703318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:35:28.703318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:28.703318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:35:28.703318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:28.703318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:35:28.703318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:35:28.703318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:28.703318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:35:28.703318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:28.703318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:35:28.703318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:35:28.703667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:28.703667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:35:28.703667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:28.703667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:35:28.704314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:28.704314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:35:28.704314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:35:28.704314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:35:28.704314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:35:28.704314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:35:28.704314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:35:28.704314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:35:28.704314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:35:28.704314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:35:28.704314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:35:28.704314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:35:28.704314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:35:28.704314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:35:28.704314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:35:28.704314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:35:28.704646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:35:28.704646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:35:28.704646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:35:28.704646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:35:28.704646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:35:28.704646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:28.704646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:35:28.704646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:28.704646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:35:28.704646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:28.704646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:35:28.704646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:28.704646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:35:28.704646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:28.704646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:35:28.704646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:35:28.704646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:28.704646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:35:28.705072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:28.705072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:35:28.705072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:35:28.705072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:28.705072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:35:28.705072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:28.705072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:35:28.707268 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:35:28.707701 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:35:28.708010 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:35:28.708298 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:35:28.708860 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:35:28.709378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:28.710375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:35:28.710375 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:28.710375 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:35:28.710375 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:35:28.710375 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:28.710375 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:35:28.710375 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:28.710375 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:35:28.711973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:28.711973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:35:28.711973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:35:28.711973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:35:28.711973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:28.711973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:35:28.711973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:28.711973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:35:28.711973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:28.711973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:35:28.711973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:28.711973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:35:28.711973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:28.711973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:35:28.711973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:35:28.712564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:28.712564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:35:28.712564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:28.712564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:35:28.712564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:35:28.712564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:28.712564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:35:28.712564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:28.712564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:35:28.713726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:28.713726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:35:28.713726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:35:28.713726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:35:28.713726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:35:28.713726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:35:28.713726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:35:28.713726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:35:28.713726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:35:28.713726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:28.713726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:35:28.714137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:28.714137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:35:28.714137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:28.714137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:35:28.714137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:28.714137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:35:28.714137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:28.714137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:35:28.714137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:35:28.714137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:28.714137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:35:28.714137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:28.714137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:35:28.714137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:35:28.714564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:28.714564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:35:28.714564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:28.714564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:35:28.715121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:28.715121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:35:28.715121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:35:28.715121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:35:28.715121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:35:28.715121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:35:28.715121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:35:28.715121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:35:28.715121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:35:28.715121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:35:28.715121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:35:28.715121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:35:28.715121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:35:28.715121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:35:28.715121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:35:28.715121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:35:28.715121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:35:28.715744 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:35:28.715744 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:35:28.715744 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:35:28.715744 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:28.715744 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:35:28.715744 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:28.715744 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:35:28.715744 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:28.715744 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:35:28.715744 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:28.715744 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:35:28.715744 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:28.715744 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:35:28.715744 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:35:28.715744 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:28.715744 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:35:28.715744 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:28.715744 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:35:28.715744 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:35:28.716472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:28.716472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:35:28.716472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:28.716472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:35:28.716472 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:35:28.716472 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:35:28.716472 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:35:28.716472 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:35:28.716818 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:35:28.716818 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:35:28.717182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:28.717877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:35:28.717877 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:28.717877 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:35:28.717877 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:35:28.717877 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:28.717877 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:35:28.717877 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:28.717877 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:35:28.718911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:28.718911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:35:28.718911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:35:28.718911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:35:28.718911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:35:28.718911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:35:28.718911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:35:28.718911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:28.718911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:35:28.718911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:28.718911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:35:28.718911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:28.719454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:35:28.719454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:28.719454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:35:28.719454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:28.719454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:35:28.719454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:35:28.719454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:28.719454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:35:28.719454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:28.719454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:35:28.719454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:35:28.719454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:28.719454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:35:28.719908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:28.719908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:35:28.719991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:28.719991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:35:28.719991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:35:28.719991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:35:28.719991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:35:28.719991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:35:28.719991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:35:28.719991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:35:28.719991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:35:28.719991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:35:28.719991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:35:28.719991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:35:28.719991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:35:28.719991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:35:28.719991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:35:28.719991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:35:28.720535 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:35:28.720535 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:35:28.720535 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:28.720535 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:35:28.720535 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:28.720535 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:35:28.720535 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:28.720535 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:35:28.720535 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:28.720535 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:35:28.720535 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:28.720535 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:35:28.720535 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:35:28.720535 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:28.720535 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:35:28.720535 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:28.720535 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:35:28.720535 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:35:28.720535 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:28.721951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:35:28.721951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:28.721951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:35:33.552903 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:35:33.553738 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:35:33.553888 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:35:33.554552 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:35:33.583855 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:35:33.584105 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:33.584584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:35:33.584584 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:33.584584 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:35:33.584584 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:35:33.584584 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:33.584584 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:35:33.584584 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:33.584584 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:35:33.586920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.031 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:33.586920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:35:33.586920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:35:33.586920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:35:33.586920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:35:33.586920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:35:33.586920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:35:33.586920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:33.586920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:35:33.586920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:33.586920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:35:33.586920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:33.587459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:35:33.587459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:33.587459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:35:33.587459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:33.587459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:35:33.587459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:35:33.587459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:33.587459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:35:33.587459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:33.587459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:35:33.587459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:35:33.587459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:33.587459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:35:33.587810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:33.587810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:35:33.587810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:33.587810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:35:33.587810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:35:33.587810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:35:33.587810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:35:33.587810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:35:33.587810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:35:33.587810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:35:33.587810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:35:33.587810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:35:33.587810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:35:33.587810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:35:33.587810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:35:33.588199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:35:33.588199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:35:33.588199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:35:33.588199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:35:33.588199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:35:33.588199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:33.588199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:35:33.588199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:33.588199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:35:33.588199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:33.588199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:35:33.588199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:33.588199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:35:33.588199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:33.588199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:35:33.588199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:35:33.588199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:33.588199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:35:33.588199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:33.588630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:35:33.588630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:35:33.588630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:33.588630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:35:33.588630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:33.588630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:35:33.690930 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:35:33.691466 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:35:33.691622 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:35:33.692648 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:35:33.693042 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:35:33.693281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:33.694117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:35:33.694117 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:33.694117 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:35:33.694117 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:35:33.694117 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:33.694117 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:35:33.694117 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:33.694117 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:35:33.695434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:33.695434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:35:33.695434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:35:33.695434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:35:33.695434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:35:33.695434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:35:33.695434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:35:33.695434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:33.695434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:35:33.695434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:33.695434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:35:33.695434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:33.695948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:35:33.695948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:33.695948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:35:33.695948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:33.695948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:35:33.695948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:35:33.695948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:33.695948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:35:33.695948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:33.695948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:35:33.695948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:35:33.695948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:33.695948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:35:33.696327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:33.696327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:35:33.697043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:33.697043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:35:33.697043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:35:33.697043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:35:33.697043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:35:33.697043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:35:33.697043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:35:33.697043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:35:33.697043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:35:33.697043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:35:33.697043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:35:33.697043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:35:33.697043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:35:33.697043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:35:33.697043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:35:33.697043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:35:33.697514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:35:33.697514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:35:33.697514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:33.697514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:35:33.697514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:33.697514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:35:33.697514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:33.697514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:35:33.697514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:33.697514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:35:33.697514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:33.697514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:35:33.697514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:35:33.697514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:33.697514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:35:33.697514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:33.697514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:35:33.697514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:35:33.697514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:33.698049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:35:33.698049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:33.698049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:35:33.700077 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:35:33.700532 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:35:33.700814 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:35:33.701079 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:35:33.701920 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:35:33.702062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:33.702778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:35:33.702778 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:33.702778 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:35:33.702778 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:35:33.702778 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:33.702778 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:35:33.702778 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:33.702778 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:35:33.703543 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:33.703543 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:35:33.703543 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:35:33.703543 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:35:33.703543 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:33.703543 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:35:33.703543 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:33.703543 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:35:33.703543 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:33.703543 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:35:33.703543 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:33.703543 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:35:33.703543 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:33.703543 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:35:33.703543 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:35:33.703947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:33.703947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:35:33.703947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:33.703947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:35:33.703947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:35:33.703947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:33.703947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:35:33.703947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:33.703947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:35:33.704561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:33.704561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:35:33.704561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:35:33.704561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:35:33.704561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:35:33.704561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:35:33.704561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:35:33.704561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:35:33.704561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:35:33.704561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:35:33.704561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:33.704561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:35:33.705022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:33.705022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:35:33.705022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:33.705022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:35:33.705022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:33.705022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:35:33.705022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:33.705022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:35:33.705022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:35:33.705022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:33.705022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:35:33.705022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:33.705022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:35:33.705022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:35:33.705471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:33.705471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:35:33.705471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:33.705471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:35:33.705670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:33.705670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:35:33.705670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:35:33.705670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:35:33.705670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:35:33.705670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:35:33.705670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:35:33.705670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:35:33.705670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:35:33.705670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:35:33.705670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:35:33.705670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:35:33.705670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:35:33.705670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:35:33.705670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:35:33.705670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:35:33.706926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:35:33.706926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:35:33.706926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:35:33.706926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:35:33.706926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:35:33.706926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:33.706926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:35:33.706926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:33.706926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:35:33.706926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:33.706926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:35:33.706926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:33.706926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:35:33.706926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:33.706926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:35:33.706926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:35:33.706926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:33.706926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:35:33.707384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:33.707384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:35:33.707384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:35:33.707384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:33.707384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:35:33.707384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:33.707384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:35:33.707825 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:35:33.708131 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:35:33.708247 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:35:33.708408 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:35:33.708753 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:35:33.709126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:33.709735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:35:33.709735 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:33.709735 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:35:33.709735 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:35:33.709735 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:33.709735 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:35:33.709735 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:33.709735 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:35:33.710766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:33.710766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:35:33.710766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:35:33.710766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:35:33.710766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:33.710766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:35:33.710766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:33.710766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:35:33.710766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:33.710766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:35:33.710766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:33.710766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:35:33.710766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:33.710766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:35:33.710766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:35:33.711434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:33.711434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:35:33.711434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:33.711434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:35:33.711434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:35:33.711434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:33.711434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:35:33.711434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:33.711434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:35:33.711790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:33.711790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:35:33.711790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:35:33.711790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:35:33.711790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:35:33.711790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:35:33.711790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:35:33.711790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:35:33.711790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:35:33.711790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:33.711790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:35:33.712456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:33.712456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:35:33.712456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:33.712456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:35:33.712456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:33.712456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:35:33.712456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:33.712456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:35:33.712456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:35:33.712456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:33.712456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:35:33.712456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:33.712456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:35:33.712456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:35:33.713169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:33.713169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:35:33.713169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:33.713169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:35:33.713169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:33.713169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:35:33.713169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:35:33.713169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:35:33.713169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:35:33.713169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:35:33.713169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:35:33.713169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:35:33.713169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:35:33.713169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:35:33.713726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:35:33.713726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:35:33.713726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:35:33.713726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:35:33.713726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:35:33.713726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:35:33.713726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:35:33.713726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:35:33.713726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:35:33.713726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:35:33.713726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:33.713726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:35:33.713726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:33.713726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:35:33.713726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:33.713726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:35:33.713726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:33.713726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:35:33.713726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:33.714297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:35:33.714297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:35:33.714297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:33.714297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:35:33.714297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:33.714297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:35:33.714297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:35:33.714297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:33.714297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:35:33.714297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:33.714297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:35:33.715722 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:35:33.716015 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:35:33.716122 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:35:33.716280 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:35:33.716638 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:35:33.716952 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:35:33.717103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:33.717625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:35:33.717625 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:33.717625 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:35:33.717625 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:35:33.717625 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:33.717625 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:35:33.717625 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:33.717625 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:35:33.718472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:33.718472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:35:33.718472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:35:33.718472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:35:33.718472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:35:33.718472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:35:33.718472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:35:33.718472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:33.718472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:35:33.718472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:33.718472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:35:33.718472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:33.718873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:35:33.718873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:33.718873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:35:33.718873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:33.718873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:35:33.718873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:35:33.718873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:33.718873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:35:33.718873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:33.718873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:35:33.718873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:35:33.718873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:33.718873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:35:33.719203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:33.719203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:35:33.719439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:33.719439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:35:33.719439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:35:33.719439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:35:33.719439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:35:33.719439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:35:33.719439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:35:33.719439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:35:33.719439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:35:33.719439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:35:33.719439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:35:33.719439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:35:33.719439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:35:33.719439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:35:33.719439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:35:33.719439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:35:33.719925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:35:33.719925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:35:33.719925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:33.719925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:35:33.719925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:33.719925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:35:33.719925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:33.719925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:35:33.719925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:33.719925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:35:33.719925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:33.719925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:35:33.719925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:35:33.719925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:33.719925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:35:33.719925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:33.719925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:35:33.719925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:35:33.719925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:33.720603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:35:33.720603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:33.720603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:35:38.554203 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:35:38.555141 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:35:38.555219 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:35:38.556722 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:35:38.616583 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:35:38.616946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:38.618028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:35:38.618028 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:38.618028 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:35:38.618028 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:35:38.618028 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:38.618028 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:35:38.618028 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:38.618028 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:35:38.618914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.063 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:38.618914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:35:38.618914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:35:38.618914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:35:38.618914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:35:38.618914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:35:38.618914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:35:38.618914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:38.618914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:35:38.618914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:38.618914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:35:38.618914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:38.619371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:35:38.619371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:38.619371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:35:38.619371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:38.619371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:35:38.619371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:35:38.619371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:38.619371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:35:38.619371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:38.619371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:35:38.619371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:35:38.619371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:38.619371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:35:38.619865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:38.619865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:35:38.619958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:38.619958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:35:38.619958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:35:38.619958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:35:38.619958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:35:38.619958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:35:38.619958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:35:38.619958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:35:38.619958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:35:38.619958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:35:38.619958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:35:38.619958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:35:38.619958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:35:38.619958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:35:38.619958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:35:38.619958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:35:38.620501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:35:38.620501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:35:38.620501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:38.620501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:35:38.620501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:38.620501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:35:38.620501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:38.620501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:35:38.620501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:38.620501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:35:38.620501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:38.620501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:35:38.620501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:35:38.620501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:38.620501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:35:38.620501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:38.620501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:35:38.620501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:35:38.620501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:38.621224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:35:38.621224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:38.621224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:35:38.691576 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:35:38.695778 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:35:38.695778 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:35:38.695778 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:35:38.695778 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:35:38.695778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:38.695778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:35:38.695778 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:38.695778 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:35:38.695778 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:35:38.696169 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:38.696169 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:35:38.696169 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:38.696169 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:35:38.696431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:38.696431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:35:38.696431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:35:38.696431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:35:38.696431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:35:38.696431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:35:38.696431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:35:38.696431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:38.696431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:35:38.696431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:38.696431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:35:38.696431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:38.696849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:35:38.696849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:38.696849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:35:38.696849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:38.696849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:35:38.696849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:35:38.696849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:38.696849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:35:38.696849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:38.696849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:35:38.696849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:35:38.696849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:38.696849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:35:38.697215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:38.697215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:35:38.697455 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:38.697455 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:35:38.697455 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:35:38.697455 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:35:38.697455 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:35:38.697455 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:35:38.697455 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:35:38.697455 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:35:38.697455 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:35:38.697455 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:35:38.697455 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:35:38.697455 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:35:38.697455 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:35:38.697455 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:35:38.697455 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:35:38.697455 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:35:38.697972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:35:38.697972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:35:38.697972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:38.697972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:35:38.697972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:38.697972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:35:38.697972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:38.697972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:35:38.697972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:38.697972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:35:38.697972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:38.697972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:35:38.697972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:35:38.697972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:38.697972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:35:38.697972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:38.697972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:35:38.697972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:35:38.697972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:38.698473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:35:38.698473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:38.698473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:35:38.700446 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:35:38.700751 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:35:38.700881 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:35:38.701391 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:35:38.701452 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:35:38.701663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:38.702264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:35:38.702264 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:38.702264 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:35:38.702264 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:35:38.702264 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:38.702264 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:35:38.702264 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:38.702264 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:35:38.703046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:38.703046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:35:38.703046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:35:38.703046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:35:38.703046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:38.703046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:35:38.703046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:38.703046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:35:38.703046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:38.703046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:35:38.703046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:38.703046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:35:38.703046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:38.703046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:35:38.703046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:35:38.703430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:38.703430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:35:38.703430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:38.703430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:35:38.703430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:35:38.703430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:38.703430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:35:38.703430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:38.703430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:35:38.703857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:38.703857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:35:38.703857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:35:38.703857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:35:38.703857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:35:38.703857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:35:38.703857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:35:38.703857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:35:38.703857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:35:38.703857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:35:38.703857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:38.703857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:35:38.704162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:38.704162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:35:38.704162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:38.704162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:35:38.704162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:38.704162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:35:38.704162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:38.704162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:35:38.704162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:35:38.704162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:38.704162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:35:38.704162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:38.704162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:35:38.704162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:35:38.704418 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:38.704418 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:35:38.704418 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:38.704418 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:35:38.704825 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:38.704825 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:35:38.704825 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:35:38.704825 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:35:38.704825 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:35:38.704825 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:35:38.704825 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:35:38.704825 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:35:38.704825 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:35:38.704825 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:35:38.704825 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:35:38.704825 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:35:38.704825 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:35:38.704825 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:35:38.704825 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:35:38.704825 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:35:38.705168 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:35:38.705168 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:35:38.705168 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:35:38.705168 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:35:38.705168 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:35:38.705168 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:38.705168 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:35:38.705168 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:38.705168 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:35:38.705168 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:38.705168 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:35:38.705168 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:38.705168 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:35:38.705168 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:38.705168 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:35:38.705168 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:35:38.705168 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:38.705168 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:35:38.705526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:38.705526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:35:38.705526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:35:38.705526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:38.705526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:35:38.705526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:38.705526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:35:38.708190 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:35:38.708488 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:35:38.708602 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:35:38.708738 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:35:38.709228 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:35:38.709585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:38.710408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:35:38.710408 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:38.710408 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:35:38.710408 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:35:38.710408 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:38.710408 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:35:38.710408 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:38.710408 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:35:38.711142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:38.711142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:35:38.711142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:35:38.711142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:35:38.711142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:38.711142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:35:38.711142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:38.711142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:35:38.711142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:38.711142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:35:38.711142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:38.711142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:35:38.711142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:38.711142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:35:38.711142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:35:38.711715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:38.711715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:35:38.711715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:38.711715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:35:38.711715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:35:38.711715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:38.711715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:35:38.711715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:38.711715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:35:38.712037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:38.712037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:35:38.712037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:35:38.712037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:35:38.712037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:35:38.712037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:35:38.712037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:35:38.712037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:35:38.712037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:35:38.712037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:38.712037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:35:38.712391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:38.712391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:35:38.712391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:38.712391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:35:38.712391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:38.712391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:35:38.712391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:38.712391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:35:38.712391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:35:38.712391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:38.712391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:35:38.712391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:38.712391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:35:38.712391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:35:38.712795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:38.712795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:35:38.712795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:38.712795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:35:38.713151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:38.713151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:35:38.713151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:35:38.713151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:35:38.713151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:35:38.713151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:35:38.713151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:35:38.713151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:35:38.713151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:35:38.713151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:35:38.713151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:35:38.713151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:35:38.713151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:35:38.713151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:35:38.713151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:35:38.713151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:35:38.713151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:35:38.714129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:35:38.714129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:35:38.714129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:35:38.714129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:38.714129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:35:38.714129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:38.714129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:35:38.714129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:38.714129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:35:38.714129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:38.714129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:35:38.714129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:38.714129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:35:38.714129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:35:38.714129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:38.714129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:35:38.714129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:38.714129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:35:38.714129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:35:38.714818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:38.714818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:35:38.714818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:38.714818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:35:38.716049 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:35:38.716314 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:35:38.719799 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:35:38.719799 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:35:38.719799 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:35:38.719799 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:35:38.719799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:38.719799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:35:38.719799 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:38.719799 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:35:38.719799 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:35:38.720175 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:38.720175 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:35:38.720175 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:38.720175 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:35:38.720175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:38.720175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:35:38.720175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:35:38.720175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:35:38.720175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:35:38.720175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:35:38.720175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:35:38.720175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:38.720175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:35:38.720175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:38.720589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:35:38.720589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:38.720589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:35:38.720589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:38.720589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:35:38.720589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:38.720589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:35:38.720589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:35:38.720589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:38.720589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:35:38.720589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:38.720589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:35:38.720589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:35:38.720589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:38.721002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:35:38.721002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:38.721002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:35:38.721089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:38.721089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:35:38.721089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:35:38.721089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:35:38.721089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:35:38.721089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:35:38.721089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:35:38.721089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:35:38.721089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:35:38.721089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:35:38.721089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:35:38.721089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:35:38.721089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:35:38.721089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:35:38.721089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:35:38.721089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:35:38.721603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:35:38.721603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:35:38.721603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:38.721603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:35:38.721603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:38.721603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:35:38.721603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:38.721603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:35:38.721603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:38.721603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:35:38.721603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:38.721603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:35:38.721603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:35:38.721603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:38.721603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:35:38.721603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:38.721603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:35:38.721603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:35:38.721603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:38.722250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:35:38.722250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:38.722250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:35:43.555052 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:35:43.555860 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:35:43.556058 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:35:43.557311 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:35:43.593775 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:35:43.594535 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:43.595371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:35:43.595371 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:43.595371 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:35:43.595371 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:35:43.595371 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:43.595371 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:35:43.595371 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:43.595371 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:35:43.596362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:43.596362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:35:43.596362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:35:43.596362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:35:43.596362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:35:43.596362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:35:43.596362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:35:43.596362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:43.596362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:35:43.596362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:43.596362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:35:43.596362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:43.596999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:35:43.596999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:43.596999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:35:43.596999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:43.596999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:35:43.596999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:35:43.596999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:43.596999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:35:43.596999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:43.596999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:35:43.596999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:35:43.596999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:43.596999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:35:43.597582 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:43.597582 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:35:43.597582 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:43.597582 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:35:43.597582 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:35:43.597582 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:35:43.597582 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:35:43.597582 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:35:43.597582 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:35:43.597582 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:35:43.597582 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:35:43.597582 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:35:43.597582 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:35:43.597582 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:35:43.597582 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:35:43.598457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:35:43.598457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:35:43.598457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:35:43.598457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:35:43.598457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:35:43.598457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:43.598457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:35:43.598457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:43.598457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:35:43.598457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:43.598457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:35:43.598457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:43.598457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:35:43.598457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:43.598457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:35:43.598457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:35:43.598457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:43.598457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:35:43.598457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:43.599231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:35:43.599231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:35:43.599231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:43.599231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:35:43.599231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:43.599231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:35:43.692364 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:35:43.692797 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:35:43.692871 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:35:43.693940 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:35:43.694368 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:35:43.694615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:43.695601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:35:43.695601 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:43.695601 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:35:43.695601 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:35:43.695601 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:43.695601 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:35:43.695601 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:43.695601 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:35:43.696696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:43.696696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:35:43.696696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:35:43.696696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:35:43.696696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:35:43.696696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:35:43.696696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:35:43.696696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:43.696696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:35:43.696696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:43.696696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:35:43.696696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:43.697101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:35:43.697101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:43.697101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:35:43.697101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:43.697101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:35:43.697101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:35:43.697101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:43.697101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:35:43.697101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:43.697101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:35:43.697101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:35:43.697101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:43.697101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:35:43.697418 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:43.697418 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:35:43.697812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:43.697812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:35:43.697812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:35:43.697812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:35:43.697812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:35:43.697812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:35:43.697812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:35:43.697812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:35:43.697812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:35:43.697812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:35:43.697812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:35:43.697812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:35:43.697812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:35:43.697812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:35:43.697812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:35:43.697812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:35:43.698298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:35:43.698298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:35:43.698298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:43.698298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:35:43.698298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:43.698298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:35:43.698298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:43.698298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:35:43.698298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:43.698298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:35:43.698298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:43.698298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:35:43.698298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:35:43.698298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:43.698298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:35:43.698298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:43.698298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:35:43.698298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:35:43.698298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:43.699002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:35:43.699002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:43.699002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:35:43.701024 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:35:43.702159 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:35:43.702159 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:35:43.702159 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:35:43.702397 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:35:43.702629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:43.703277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:35:43.703277 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:43.703277 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:35:43.703277 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:35:43.703277 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:43.703277 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:35:43.703277 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:43.703277 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:35:43.704565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:43.704565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:35:43.704565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:35:43.704565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:35:43.704565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:43.704565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:35:43.704565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:43.704565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:35:43.704565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:43.704565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:35:43.704565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:43.704565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:35:43.704565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:43.704565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:35:43.704565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:35:43.704963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:43.704963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:35:43.704963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:43.704963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:35:43.704963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:35:43.704963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:43.704963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:35:43.704963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:43.704963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:35:43.706190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:43.706190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:35:43.706190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:35:43.706190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:35:43.706190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:35:43.706190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:35:43.706190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:35:43.706190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:35:43.706190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:35:43.706190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:35:43.706190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:43.706190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:35:43.706586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:43.706586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:35:43.706586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:43.706586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:35:43.706586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:43.706586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:35:43.706586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:43.706586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:35:43.706586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:35:43.706586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:43.706586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:35:43.706586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:43.706586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:35:43.706586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:35:43.707054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:43.707054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:35:43.707054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:43.707054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:35:43.707831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:43.707831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:35:43.707831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:35:43.707831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:35:43.707831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:35:43.707831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:35:43.707831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:35:43.707831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:35:43.707831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:35:43.707831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:35:43.707831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:35:43.707831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:35:43.707831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:35:43.707831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:35:43.707831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:35:43.707831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:35:43.708422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:35:43.708422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:35:43.708422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:35:43.708422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:35:43.708422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:35:43.708422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:43.708422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:35:43.708422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:43.708422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:35:43.708422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:43.708422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:35:43.708422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:43.708422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:35:43.708422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:43.708422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:35:43.708422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:35:43.708422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:43.708422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:35:43.709587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:43.709587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:35:43.709587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:35:43.709587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:43.709587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:35:43.709587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:43.709587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:35:43.709587 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:35:43.709587 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:35:43.709587 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:35:43.710330 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:35:43.710574 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:35:43.710899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:43.711740 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:35:43.711740 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:43.711740 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:35:43.711740 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:35:43.711740 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:43.711740 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:35:43.711740 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:43.711740 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:35:43.712601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:43.712601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:35:43.712601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:35:43.712601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:35:43.712601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:43.712601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:35:43.712601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:43.712601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:35:43.712601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:43.712601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:35:43.712601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:43.712601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:35:43.712601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:43.712601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:35:43.712601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:35:43.713516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:43.713516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:35:43.713516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:43.713516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:35:43.713516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:35:43.713516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:43.713516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:35:43.713516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:43.713516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:35:43.713516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:43.713516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:35:43.713516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:35:43.713516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:35:43.713516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:35:43.714395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:35:43.714395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:35:43.714395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:35:43.714395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:35:43.714395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:43.714395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:35:43.714395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:43.714395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:35:43.714395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:43.714395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:35:43.714395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:43.714395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:35:43.714395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:43.715096 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:35:43.715096 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:35:43.715096 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:43.715096 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:35:43.715096 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:43.715096 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:35:43.715096 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:35:43.715096 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:43.715096 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:35:43.715096 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:43.715096 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:35:43.715096 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:43.715577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:35:43.715577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:35:43.715577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:35:43.715577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:35:43.715577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:35:43.715577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:35:43.715577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:35:43.715577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:35:43.715577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:35:43.715577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:35:43.715577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:35:43.715577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:35:43.715577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:35:43.715577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:35:43.715577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:35:43.715577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:35:43.715577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:35:43.715577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:35:43.715577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:35:43.716197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:43.716197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:35:43.716197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:43.716197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:35:43.716197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:43.716197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:35:43.716197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:43.716197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:35:43.716197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:43.716197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:35:43.716197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:35:43.716197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:43.716197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:35:43.716197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:43.716197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:35:43.716197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:35:43.716197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:43.716197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:35:43.716197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:43.716197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:35:43.716939 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:35:43.716939 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:35:43.717045 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:35:43.717283 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:35:43.717903 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:35:43.718442 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:35:43.718686 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:43.719405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:35:43.719405 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:43.719405 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:35:43.719405 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:35:43.719405 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:43.719405 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:35:43.719405 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:43.719405 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:35:43.720318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:43.720318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:35:43.720318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:35:43.720318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:35:43.720318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:35:43.720318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:35:43.720318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:35:43.720318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:43.720318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:35:43.720318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:43.720318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:35:43.720318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:43.720892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:35:43.720892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:43.720892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:35:43.720892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:43.720892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:35:43.720892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:35:43.720892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:43.720892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:35:43.720892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:43.720892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:35:43.720892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:35:43.720892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:43.720892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:35:43.721246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:43.721246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:35:43.721332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:43.721332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:35:43.721332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:35:43.721332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:35:43.721332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:35:43.721332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:35:43.721332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:35:43.721332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:35:43.721332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:35:43.721332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:35:43.721332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:35:43.721332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:35:43.721332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:35:43.721332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:35:43.721332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:35:43.721332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:35:43.721942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:35:43.721942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:35:43.721942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:43.721942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:35:43.721942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:43.721942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:35:43.721942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:43.721942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:35:43.721942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:43.721942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:35:43.721942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:43.721942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:35:43.721942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:35:43.721942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:43.721942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:35:43.721942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:43.721942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:35:43.721942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:35:43.721942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:43.722612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:35:43.722612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:43.722612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:35:48.555867 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:35:48.556907 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:35:48.557062 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:35:48.557639 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:35:48.594614 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:35:48.595071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:48.595968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:35:48.595968 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:48.595968 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:35:48.595968 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:35:48.595968 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:48.595968 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:35:48.595968 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:48.595968 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:35:48.596734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:48.596734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:35:48.596734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:35:48.596734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:35:48.596734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:35:48.596734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:35:48.596734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:35:48.596734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:48.596734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:35:48.596734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:48.596734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:35:48.596734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:48.597380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:35:48.597380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:48.597380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:35:48.597380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:48.597380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:35:48.597380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:35:48.597380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:48.597380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:35:48.597380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:48.597380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:35:48.597380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:35:48.597380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:48.597380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:35:48.598037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:48.598037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:35:48.598037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:48.598037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:35:48.598037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:35:48.598037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:35:48.598037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:35:48.598037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:35:48.598037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:35:48.598037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:35:48.598037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:35:48.598037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:35:48.598037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:35:48.598037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:35:48.598037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:35:48.598805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:35:48.598805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:35:48.598805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:35:48.598805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:35:48.598805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:35:48.598805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:48.598805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:35:48.598805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:48.598805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:35:48.598805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:48.598805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:35:48.598805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:48.598805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:35:48.598805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:48.598805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:35:48.598805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:35:48.598805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:48.598805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:35:48.598805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:48.601021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:35:48.601021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:35:48.601021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:48.601021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:35:48.601021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:48.601021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:35:48.692991 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:35:48.693540 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:35:48.693763 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:35:48.694898 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:35:48.695302 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:35:48.695582 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:48.696435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:35:48.696435 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:48.696435 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:35:48.696435 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:35:48.696435 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:48.696435 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:35:48.696435 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:48.696435 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:35:48.698186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:48.698186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:35:48.698186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:35:48.698186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:35:48.698186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:35:48.698186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:35:48.698186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:35:48.698186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:48.698186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:35:48.698186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:48.698186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:35:48.698186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:48.698851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:35:48.698851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:48.698851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:35:48.698851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:48.698851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:35:48.698851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:35:48.698851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:48.698851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:35:48.698851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:48.698851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:35:48.698851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:35:48.698851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:48.698851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:35:48.699513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:48.699513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:35:48.700150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:48.700150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:35:48.700150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:35:48.700150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:35:48.700150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:35:48.700150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:35:48.700150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:35:48.700150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:35:48.700150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:35:48.700150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:35:48.700150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:35:48.700150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:35:48.700150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:35:48.700150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:35:48.700150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:35:48.700150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:35:48.700991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:35:48.700991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:35:48.700991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:48.700991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:35:48.700991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:48.700991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:35:48.700991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:48.700991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:35:48.700991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:48.700991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:35:48.700991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:48.700991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:35:48.700991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:35:48.700991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:48.700991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:35:48.700991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:48.700991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:35:48.700991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:35:48.700991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:48.702006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:35:48.702006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:48.702006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:35:48.702006 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:35:48.702006 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:35:48.702322 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:35:48.702322 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:35:48.702931 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:35:48.703274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:48.704381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:35:48.704381 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:48.704381 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:35:48.704381 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:35:48.704381 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:48.704381 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:35:48.704381 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:48.704381 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:35:48.705722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:48.705722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:35:48.705722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:35:48.705722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:35:48.705722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:48.705722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:35:48.705722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:48.705722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:35:48.705722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:48.705722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:35:48.705722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:48.705722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:35:48.705722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:48.705722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:35:48.705722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:35:48.706415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:48.706415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:35:48.706415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:48.706415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:35:48.706415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:35:48.706415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:48.706415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:35:48.706415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:48.706415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:35:48.706870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:48.706870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:35:48.706870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:35:48.706870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:35:48.706870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:35:48.706870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:35:48.706870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:35:48.706870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:35:48.706870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:35:48.706870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:35:48.706870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:48.706870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:35:48.707394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:48.707394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:35:48.707394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:48.707394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:35:48.707394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:48.707394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:35:48.707394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:48.707394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:35:48.707394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:35:48.707394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:48.707394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:35:48.707394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:48.707394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:35:48.707394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:35:48.707955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:48.707955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:35:48.707955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:48.707955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:35:48.707955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:48.707955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:35:48.707955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:35:48.707955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:35:48.707955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:35:48.707955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:35:48.707955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:35:48.707955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:35:48.707955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:35:48.707955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:35:48.708514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:35:48.708514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:35:48.708514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:35:48.708514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:35:48.708514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:35:48.708514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:35:48.708514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:35:48.708514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:35:48.708514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:35:48.708514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:35:48.708514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:35:48.708514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:48.708514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:35:48.708514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:48.708514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:35:48.708514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:48.708514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:35:48.708514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:48.708514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:35:48.709462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:48.709462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:35:48.709462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:35:48.709462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:48.709462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:35:48.709462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:48.709462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:35:48.709462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:35:48.709462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:48.709462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:35:48.709462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:48.709462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:35:48.709462 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:35:48.710008 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:35:48.710008 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:35:48.710008 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:35:48.710493 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:35:48.710718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:48.711325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:35:48.711325 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:48.711325 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:35:48.711325 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:35:48.711325 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:48.711325 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:35:48.711325 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:48.711325 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:35:48.712358 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:48.712358 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:35:48.712358 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:35:48.712358 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:35:48.712358 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:48.712358 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:35:48.712358 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:48.712358 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:35:48.712358 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:48.712358 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:35:48.712358 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:48.712358 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:35:48.712358 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:48.712358 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:35:48.712358 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:35:48.712853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:48.712853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:35:48.712853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:48.712853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:35:48.712853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:35:48.712853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:48.712853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:35:48.712853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:48.712853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:35:48.713982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:48.713982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:35:48.713982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:35:48.713982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:35:48.713982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:35:48.713982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:35:48.713982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:35:48.713982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:35:48.713982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:35:48.713982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:48.713982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:35:48.714434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:48.714434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:35:48.714434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:48.714434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:35:48.714434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:48.714434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:35:48.714434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:48.714434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:35:48.714434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:35:48.714434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:48.714434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:35:48.714434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:48.714434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:35:48.714434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:35:48.714829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:48.714829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:35:48.714829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:48.714829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:35:48.715621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:48.715621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:35:48.715621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:35:48.715621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:35:48.715621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:35:48.715621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:35:48.715621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:35:48.715621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:35:48.715621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:35:48.715621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:35:48.715621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:35:48.715621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:35:48.715621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:35:48.715621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:35:48.715621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:35:48.715621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:35:48.715621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:35:48.716191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:35:48.716191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:35:48.716191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:35:48.716191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:48.716191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:35:48.716191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:48.716191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:35:48.716191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:48.716191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:35:48.716191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:48.716191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:35:48.716191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:48.716191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:35:48.716191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:35:48.716191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:48.716191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:35:48.716191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:48.716191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:35:48.716191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:35:48.716764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:48.716764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:35:48.716764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:48.716764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:35:48.716994 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:35:48.717342 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:35:48.717484 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:35:48.717648 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:35:48.718108 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:35:48.718361 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:35:48.718588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:48.719409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:35:48.719409 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:48.719409 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:35:48.719409 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:35:48.719409 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:48.719409 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:35:48.719409 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:48.719409 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:35:48.720513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:48.720513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:35:48.720513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:35:48.720513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:35:48.720513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:35:48.720513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:35:48.720513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:35:48.720513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:48.720513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:35:48.720513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:48.720513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:35:48.720513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:48.721065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:35:48.721065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:48.721065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:35:48.721065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:48.721065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:35:48.721065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:35:48.721065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:48.721065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:35:48.721065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:48.721065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:35:48.721065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:35:48.721065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:48.721065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:35:48.721526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:48.721526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:35:48.721606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:48.721606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:35:48.721606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:35:48.721606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:35:48.721606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:35:48.721606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:35:48.721606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:35:48.721606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:35:48.721606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:35:48.721606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:35:48.721606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:35:48.721606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:35:48.721606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:35:48.721606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:35:48.721606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:35:48.721606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:35:48.722219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:35:48.722219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:35:48.722219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:48.722219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:35:48.722219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:48.722219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:35:48.722219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:48.722219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:35:48.722219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:48.722219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:35:48.722219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:48.722219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:35:48.722219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:35:48.722219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:48.722219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:35:48.722219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:48.722219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:35:48.722219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:35:48.722219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:48.722799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:35:48.722799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:48.722799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:35:53.556929 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:35:53.557803 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:35:53.557945 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:35:53.559518 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:35:53.596730 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:35:53.597149 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:53.598066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:35:53.598066 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:53.598066 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:35:53.598066 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:35:53.598066 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:53.598066 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:35:53.598066 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:53.598066 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:35:53.599375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:53.599375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:35:53.599375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:35:53.599375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:35:53.599375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:35:53.599375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:35:53.599375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:35:53.599375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:53.599375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:35:53.599375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:53.599375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:35:53.599375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:53.600073 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:35:53.600073 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:53.600073 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:35:53.600073 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:53.600073 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:35:53.600073 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:35:53.600073 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:53.600073 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:35:53.600073 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:53.600073 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:35:53.600073 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:35:53.600073 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:53.600073 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:35:53.600676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:53.600676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:35:53.600676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:53.600676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:35:53.600676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:35:53.600676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:35:53.600676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:35:53.600676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:35:53.600676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:35:53.600676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:35:53.600676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:35:53.600676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:35:53.600676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:35:53.600676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:35:53.600676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:35:53.601619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:35:53.601619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:35:53.601619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:35:53.601619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:35:53.601619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:35:53.601619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:53.601619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:35:53.601619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:53.601619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:35:53.601619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:53.601619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:35:53.601619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:53.601619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:35:53.601619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:53.601619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:35:53.601619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:35:53.601619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:53.601619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:35:53.601619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:53.602488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:35:53.602488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:35:53.602488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:53.602488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:35:53.602488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:53.602488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:35:53.693548 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:35:53.694370 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:35:53.694570 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:35:53.696185 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:35:53.696485 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:35:53.696667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:53.697502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:35:53.697502 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:53.697502 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:35:53.697502 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:35:53.697502 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:53.697502 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:35:53.697502 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:53.697502 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:35:53.699276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:53.699276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:35:53.699276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:35:53.699276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:35:53.699276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:35:53.699276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:35:53.699276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:35:53.699276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:53.699276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:35:53.699276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:53.699276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:35:53.699276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:53.700067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:35:53.700067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:53.700067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:35:53.700067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:53.700067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:35:53.700067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:35:53.700067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:53.700067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:35:53.700067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:53.700067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:35:53.700067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:35:53.700067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:53.700067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:35:53.700979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:53.700979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:35:53.701617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:53.701617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:35:53.701617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:35:53.701617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:35:53.701617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:35:53.701617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:35:53.701617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:35:53.701617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:35:53.701617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:35:53.701617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:35:53.701617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:35:53.701617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:35:53.701617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:35:53.701617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:35:53.701617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:35:53.701617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:35:53.702365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:35:53.702365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:35:53.702365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:53.702365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:35:53.702365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:53.702365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:35:53.702365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:53.702365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:35:53.702365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:53.702365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:35:53.702365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:53.702365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:35:53.702365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:35:53.702365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:53.702365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:35:53.702365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:53.702365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:35:53.702365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:35:53.702365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:53.703307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:35:53.703307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:53.703307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:35:53.703307 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:35:53.703307 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:35:53.703307 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:35:53.703307 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:35:53.703766 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:35:53.703998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:53.704712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:35:53.704712 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:53.704712 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:35:53.704712 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:35:53.704712 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:53.704712 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:35:53.704712 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:53.704712 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:35:53.706151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:53.706151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:35:53.706151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:35:53.706151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:35:53.706151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:53.706151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:35:53.706151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:53.706151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:35:53.706151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:53.706151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:35:53.706151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:53.706151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:35:53.706151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:53.706151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:35:53.706151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:35:53.706579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:53.706579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:35:53.706579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:53.706579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:35:53.706579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:35:53.706579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:53.706579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:35:53.706579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:53.706579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:35:53.708232 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:53.708232 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:35:53.708232 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:35:53.708232 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:35:53.708232 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:35:53.708232 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:35:53.708232 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:35:53.708232 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:35:53.708232 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:35:53.708232 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:35:53.708232 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:53.708232 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:35:53.708586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:53.708586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:35:53.708586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:53.708586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:35:53.708586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:53.708586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:35:53.708586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:53.708586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:35:53.708586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:35:53.708586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:53.708586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:35:53.708586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:53.708586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:35:53.708586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:35:53.708954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:53.708954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:35:53.708954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:53.708954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:35:53.709474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:53.709474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:35:53.709474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:35:53.709474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:35:53.709474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:35:53.709474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:35:53.709474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:35:53.709474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:35:53.709474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:35:53.709474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:35:53.709474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:35:53.709474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:35:53.709474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:35:53.709474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:35:53.709474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:35:53.709474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:35:53.709935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:35:53.709935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:35:53.709935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:35:53.709935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:35:53.709935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:35:53.709935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:53.709935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:35:53.709935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:53.709935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:35:53.709935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:53.709935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:35:53.709935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:53.709935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:35:53.709935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:53.709935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:35:53.709935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:35:53.709935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:53.709935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:35:53.710410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:53.710410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:35:53.710410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:35:53.710410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:53.710410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:35:53.710410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:53.710410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:35:53.710410 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:35:53.710410 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:35:53.710410 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:35:53.710697 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:35:53.710829 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:35:53.711116 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:53.711703 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:35:53.711703 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:53.711703 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:35:53.711703 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:35:53.711703 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:53.711703 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:35:53.711703 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:53.711703 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:35:53.712300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:53.712300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:35:53.712300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:35:53.712300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:35:53.712300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:53.712300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:35:53.712300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:53.712300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:35:53.712300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:53.712300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:35:53.712300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:53.712300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:35:53.712300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:53.712300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:35:53.712300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:35:53.712661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:53.712661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:35:53.712661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:53.712661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:35:53.712661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:35:53.712661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:53.712661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:35:53.712661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:53.712661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:35:53.712989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:53.712989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:35:53.712989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:35:53.712989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:35:53.712989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:35:53.712989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:35:53.712989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:35:53.712989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:35:53.712989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:35:53.712989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:53.712989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:35:53.713319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:53.713319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:35:53.713319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:53.713319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:35:53.713319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:53.713319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:35:53.713319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:53.713319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:35:53.713319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:35:53.713319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:53.713319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:35:53.713319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:53.713319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:35:53.713319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:35:53.713661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:53.713661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:35:53.713661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:53.713661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:35:53.713971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:53.713971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:35:53.713971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:35:53.713971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:35:53.713971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:35:53.713971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:35:53.713971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:35:53.713971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:35:53.713971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:35:53.713971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:35:53.713971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:35:53.713971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:35:53.713971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:35:53.713971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:35:53.713971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:35:53.713971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:35:53.713971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:35:53.714400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:35:53.714400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:35:53.714400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:35:53.714400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:53.714400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:35:53.714400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:53.714400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:35:53.714400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:53.714400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:35:53.714400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:53.714400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:35:53.714400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:53.714400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:35:53.714400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:35:53.714400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:53.714400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:35:53.714400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:53.714400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:35:53.714400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:35:53.714895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:53.714895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:35:53.714895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:53.714895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:35:53.717502 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:35:53.718026 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:35:53.718209 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:35:53.718468 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:35:53.718947 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:35:53.719211 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:35:53.719368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:53.720144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:35:53.720144 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:53.720144 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:35:53.720144 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:35:53.720144 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:53.720144 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:35:53.720144 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:53.720144 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:35:53.721290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:53.721290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:35:53.721290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:35:53.721290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:35:53.721290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:35:53.721290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:35:53.721290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:35:53.721290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:53.721290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:35:53.721290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:53.721290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:35:53.721290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:53.721614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:35:53.721614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:53.721614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:35:53.721614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:53.721614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:35:53.721614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:35:53.721614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:53.721614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:35:53.721614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:53.721614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:35:53.721614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:35:53.721614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:53.721614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:35:53.721979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:53.721979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:35:53.722774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:53.722774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:35:53.722774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:35:53.722774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:35:53.722774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:35:53.722774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:35:53.722774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:35:53.722774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:35:53.722774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:35:53.722774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:35:53.722774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:35:53.722774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:35:53.722774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:35:53.722774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:35:53.722774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:35:53.722774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:35:53.723313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:35:53.723313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:35:53.723313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:53.723313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:35:53.723313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:53.723313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:35:53.723313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:53.723313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:35:53.723313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:53.723313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:35:53.723313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:53.723313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:35:53.723313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:35:53.723313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:53.723313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:35:53.723313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:53.723313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:35:53.723313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:35:53.723313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:53.723929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:35:53.723929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:53.723929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:35:58.557709 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:35:58.558939 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:35:58.558939 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:35:58.560388 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:35:58.562877 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG dbcounter [-] [106314] Writing DB stats neutron:SELECT=240 {{(pid=106314) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 09 09:35:58.596496 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:35:58.596852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:58.597824 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:35:58.597824 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:58.597824 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:35:58.597824 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:35:58.597824 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:58.597824 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:35:58.597824 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:58.597824 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:35:58.598718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:58.598718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:35:58.598718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:35:58.598718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:35:58.598718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:35:58.598718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:35:58.598718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:35:58.598718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:58.598718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:35:58.598718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:58.598718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:35:58.598718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:58.599347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:35:58.599347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:58.599347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:35:58.599347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:58.599347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:35:58.599347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:35:58.599347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:58.599347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:35:58.599347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:58.599347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:35:58.599347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:35:58.599347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:58.599347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:35:58.599980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:58.599980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:35:58.599980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:58.599980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:35:58.599980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:35:58.599980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:35:58.599980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:35:58.599980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:35:58.599980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:35:58.599980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:35:58.599980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:35:58.599980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:35:58.599980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:35:58.599980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:35:58.599980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:35:58.600681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:35:58.600681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:35:58.600681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:35:58.600681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:35:58.600681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:35:58.600681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:58.600681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:35:58.600681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:58.600681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:35:58.600681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:58.600681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:35:58.600681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:58.600681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:35:58.600681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:58.600681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:35:58.600681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:35:58.600681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:58.600681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:35:58.600681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:58.601619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:35:58.601619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:35:58.601619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:58.601619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:35:58.601619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:58.601619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:35:58.694416 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:35:58.695027 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:35:58.695230 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:35:58.696921 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:35:58.697097 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:35:58.697373 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:58.698527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:35:58.698527 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:58.698527 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:35:58.698527 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:35:58.698527 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:58.698527 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:35:58.698527 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:58.698527 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:35:58.699910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:58.699910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:35:58.699910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:35:58.699910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:35:58.699910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:35:58.699910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:35:58.699910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:35:58.699910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:58.699910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:35:58.699910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:58.699910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:35:58.699910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:58.700343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:35:58.700343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:58.700343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:35:58.700343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:58.700343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:35:58.700343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:35:58.700343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:58.700343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:35:58.700343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:58.700343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:35:58.700343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:35:58.700343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:58.700343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:35:58.700734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:58.700734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:35:58.701083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:58.701083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:35:58.701083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:35:58.701083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:35:58.701083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:35:58.701083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:35:58.701083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:35:58.701083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:35:58.701083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:35:58.701083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:35:58.701083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:35:58.701083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:35:58.701083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:35:58.701083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:35:58.701083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:35:58.701083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:35:58.701601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:35:58.701601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:35:58.701601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:58.701601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:35:58.701601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:58.701601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:35:58.701601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:58.701601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:35:58.701601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:58.701601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:35:58.701601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:58.701601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:35:58.701601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:35:58.701601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:58.701601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:35:58.701601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:58.701601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:35:58.701601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:35:58.701601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:58.702196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:35:58.702196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:58.702196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:35:58.702490 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:35:58.702760 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:35:58.702883 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:35:58.703018 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:35:58.703472 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:35:58.703705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:58.704288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:35:58.704288 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:58.704288 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:35:58.704288 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:35:58.704288 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:58.704288 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:35:58.704288 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:58.704288 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:35:58.705108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:58.705108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:35:58.705108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:35:58.705108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:35:58.705108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:58.705108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:35:58.705108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:58.705108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:35:58.705108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:58.705108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:35:58.705108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:58.705108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:35:58.705108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:58.705108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:35:58.705108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:35:58.705875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:58.705875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:35:58.705875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:58.705875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:35:58.705875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:35:58.705875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:58.705875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:35:58.705875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:58.705875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:35:58.706299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:58.706299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:35:58.706299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:35:58.706299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:35:58.706299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:35:58.706299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:35:58.706299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:35:58.706299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:35:58.706299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:35:58.706299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:35:58.706299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:58.706299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:35:58.706821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:58.706821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:35:58.706821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:58.706821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:35:58.706821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:58.706821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:35:58.706821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:58.706821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:35:58.706821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:35:58.706821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:58.706821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:35:58.706821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:58.706821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:35:58.706821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:35:58.707312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:58.707312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:35:58.707312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:58.707312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:35:58.707312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:58.707312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:35:58.707312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:35:58.707312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:35:58.707312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:35:58.707312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:35:58.707312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:35:58.707312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:35:58.707312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:35:58.707312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:35:58.707856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:35:58.707856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:35:58.707856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:35:58.707856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:35:58.707856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:35:58.707856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:35:58.707856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:35:58.707856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:35:58.707856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:35:58.707856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:35:58.707856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:35:58.707856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:58.707856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:35:58.707856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:58.707856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:35:58.707856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:58.707856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:35:58.707856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:58.707856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:35:58.708526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:58.708526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:35:58.708526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:35:58.708526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:58.708526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:35:58.708526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:58.708526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:35:58.708526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:35:58.708526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:58.708526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:35:58.708526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:58.708526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:35:58.710209 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:35:58.710428 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:35:58.710521 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:35:58.710700 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:35:58.711232 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:35:58.711479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:58.712104 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:35:58.712104 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:58.712104 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:35:58.712104 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:35:58.712104 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:58.712104 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:35:58.712104 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:58.712104 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:35:58.712937 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:58.712937 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:35:58.712937 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:35:58.712937 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:35:58.712937 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:58.712937 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:35:58.712937 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:58.712937 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:35:58.712937 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:58.712937 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:35:58.712937 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:58.712937 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:35:58.712937 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:58.712937 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:35:58.712937 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:35:58.713545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:58.713545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:35:58.713545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:58.713545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:35:58.713545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:35:58.713545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:58.713545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:35:58.713545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:58.713545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:35:58.713961 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:58.713961 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:35:58.713961 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:35:58.713961 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:35:58.713961 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:35:58.713961 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:35:58.713961 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:35:58.713961 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:35:58.713961 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:35:58.713961 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:58.713961 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:35:58.714468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:58.714468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:35:58.714468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:58.714468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:35:58.714468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:58.714468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:35:58.714468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:58.714468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:35:58.714468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:35:58.714468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:58.714468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:35:58.714468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:58.714468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:35:58.714468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:35:58.715099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:58.715099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:35:58.715099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:58.715099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:35:58.715099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:58.715099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:35:58.715099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:35:58.715099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:35:58.715099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:35:58.715099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:35:58.715099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:35:58.715099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:35:58.715099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:35:58.715099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:35:58.715736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:35:58.715736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:35:58.715736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:35:58.715736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:35:58.715736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:35:58.715736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:35:58.715736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:35:58.715736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:35:58.715736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:35:58.715736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:35:58.715736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:58.715736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:35:58.715736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:58.715736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:35:58.715736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:58.715736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:35:58.715736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:58.715736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:35:58.715736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:58.716603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:35:58.716603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:35:58.716603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:58.716603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:35:58.716603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:58.716603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:35:58.716603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:35:58.716603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:58.716603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:35:58.716603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:58.716603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:35:58.718417 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:35:58.718417 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:35:58.718417 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:35:58.718695 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:35:58.719139 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:35:58.719659 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:35:58.720063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:58.720950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:35:58.720950 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:58.720950 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:35:58.720950 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:35:58.720950 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:58.720950 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:35:58.720950 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:58.720950 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:35:58.721951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:58.721951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:35:58.721951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:35:58.721951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:35:58.721951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:35:58.721951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:35:58.721951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:35:58.721951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:58.721951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:35:58.721951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:58.721951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:35:58.721951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:58.722542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:35:58.722542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:58.722542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:35:58.722542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:58.722542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:35:58.722542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:35:58.722542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:58.722542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:35:58.722542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:58.722542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:35:58.722542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:35:58.722542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:58.722542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:35:58.723091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:58.723091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:35:58.723091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:58.723091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:35:58.723091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:35:58.723091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:35:58.723091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:35:58.723091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:35:58.723091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:35:58.723091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:35:58.723091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:35:58.723091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:35:58.723091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:35:58.723091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:35:58.723091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:35:58.723442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:35:58.723442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:35:58.723442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:35:58.723442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:35:58.723442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:35:58.723442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:58.723442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:35:58.723442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:35:58.723442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:35:58.723442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:35:58.723442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:35:58.723442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:35:58.723442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:35:58.723442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:35:58.723442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:35:58.723442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:35:58.723442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:35:58.723442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:35:58.723442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:35:58.723859 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:35:58.723859 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:35:58.723859 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:35:58.723859 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:35:58.723859 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:35:58.723859 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:36:03.558507 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:36:03.561888 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:36:03.561888 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:36:03.561888 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:36:03.602225 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:36:03.602787 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:03.604128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:36:03.604128 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:03.604128 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:36:03.604128 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:36:03.604128 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:03.604128 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:36:03.604128 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:03.604128 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:36:03.605346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.045 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:03.605346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:36:03.605346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:36:03.605346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:36:03.605346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:36:03.605346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:36:03.605346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:36:03.605346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:03.605346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:36:03.605346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:03.605346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:36:03.605346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:03.606067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:36:03.606067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:03.606067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:36:03.606067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:03.606067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:36:03.606067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:36:03.606067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:03.606067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:36:03.606067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:03.606067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:36:03.606067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:36:03.606067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:03.606067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:36:03.606695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:03.606695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:36:03.606695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:03.606695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:36:03.606695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:36:03.606695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:36:03.606695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:36:03.606695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:36:03.606695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:36:03.606695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:36:03.606695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:36:03.606695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:36:03.606695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:36:03.606695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:36:03.606695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:36:03.607430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:36:03.607430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:36:03.607430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:36:03.607430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:36:03.607430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:36:03.607430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:03.607430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:36:03.607430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:03.607430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:36:03.607430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:03.607430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:36:03.607430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:03.607430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:36:03.607430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:03.607430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:36:03.607430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:36:03.607430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:03.607430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:36:03.607430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:03.608288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:36:03.608288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:36:03.608288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:03.608288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:36:03.608288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:03.608288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:36:03.695277 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:36:03.695657 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:36:03.695780 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:36:03.696779 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:36:03.697093 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:36:03.697286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:03.698080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:36:03.698080 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:03.698080 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:36:03.698080 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:36:03.698080 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:03.698080 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:36:03.698080 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:03.698080 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:36:03.699156 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:03.699156 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:36:03.699156 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:36:03.699156 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:36:03.699156 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:36:03.699156 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:36:03.699156 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:36:03.699156 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:03.699156 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:36:03.699156 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:03.699156 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:36:03.699156 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:03.699780 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:36:03.699780 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:03.699780 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:36:03.699780 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:03.699780 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:36:03.699780 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:36:03.699780 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:03.699780 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:36:03.699780 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:03.699780 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:36:03.699780 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:36:03.699780 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:03.699780 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:36:03.700326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:03.700326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:36:03.700326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:03.700326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:36:03.700326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:36:03.700326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:36:03.700326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:36:03.700326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:36:03.700326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:36:03.700326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:36:03.700326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:36:03.700326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:36:03.700326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:36:03.700326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:36:03.700326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:36:03.701072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:36:03.701072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:36:03.701072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:36:03.701072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:36:03.701072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:36:03.701072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:03.701072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:36:03.701072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:03.701072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:36:03.701072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:03.701072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:36:03.701072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:03.701072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:36:03.701072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:03.701072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:36:03.701072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:36:03.701072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:03.701072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:36:03.701072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:03.702157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:36:03.702157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:36:03.702157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:03.702157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:36:03.702157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:03.702157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:36:03.702864 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:36:03.703066 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:36:03.703168 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:36:03.703319 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:36:03.703814 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:36:03.704100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:03.704788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:36:03.704788 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:03.704788 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:36:03.704788 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:36:03.704788 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:03.704788 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:36:03.704788 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:03.704788 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:36:03.705529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:03.705529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:36:03.705529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:36:03.705529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:36:03.705529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:03.705529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:36:03.705529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:03.705529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:36:03.705529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:03.705529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:36:03.705529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:03.705529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:36:03.705529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:03.705529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:36:03.705529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:36:03.706122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:03.706122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:36:03.706122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:03.706122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:36:03.706122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:36:03.706122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:03.706122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:36:03.706122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:03.706122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:36:03.706397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:03.706397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:36:03.706397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:36:03.706397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:36:03.706397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:36:03.706397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:36:03.706397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:36:03.706397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:36:03.706397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:36:03.706397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:36:03.706397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:03.706397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:36:03.706733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:03.706733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:36:03.706733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:03.706733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:36:03.706733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:03.706733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:36:03.706733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:03.706733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:36:03.706733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:36:03.706733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:03.706733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:36:03.706733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:03.706733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:36:03.706733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:36:03.707144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:03.707144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:36:03.707144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:03.707144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:36:03.707338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:03.707338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:36:03.707338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:36:03.707338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:36:03.707338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:36:03.707338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:36:03.707338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:36:03.707338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:36:03.707338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:36:03.707338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:36:03.707338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:36:03.707338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:36:03.707338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:36:03.707338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:36:03.707338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:36:03.707338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:36:03.707831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:36:03.707831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:36:03.707831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:36:03.707831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:36:03.707831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:36:03.707831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:03.707831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:36:03.707831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:03.707831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:36:03.707831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:03.707831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:36:03.707831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:03.707831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:36:03.707831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:03.707831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:36:03.707831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:36:03.707831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:03.707831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:36:03.708330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:03.708330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:36:03.708330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:36:03.708330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:03.708330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:36:03.708330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:03.708330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:36:03.712993 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:36:03.713256 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:36:03.713353 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:36:03.713513 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:36:03.713959 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:36:03.714261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:03.714929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:36:03.714929 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:03.714929 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:36:03.714929 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:36:03.714929 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:03.714929 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:36:03.714929 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:03.714929 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:36:03.715672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:03.715672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:36:03.715672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:36:03.715672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:36:03.715672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:03.715672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:36:03.715672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:03.715672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:36:03.715672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:03.715672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:36:03.715672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:03.715672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:36:03.715672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:03.715672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:36:03.715672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:36:03.716221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:03.716221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:36:03.716221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:03.716221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:36:03.716221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:36:03.716221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:03.716221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:36:03.716221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:03.716221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:36:03.716559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:03.716559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:36:03.716559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:36:03.716559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:36:03.716559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:36:03.716559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:36:03.716559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:36:03.716559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:36:03.716559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:36:03.716559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:03.716559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:36:03.716990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:03.716990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:36:03.716990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:03.716990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:36:03.716990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:03.716990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:36:03.716990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:03.716990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:36:03.716990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:36:03.716990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:03.716990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:36:03.716990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:03.716990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:36:03.716990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:36:03.717549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:03.717549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:36:03.717549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:03.717549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:36:03.717711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:03.717711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:36:03.717711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:36:03.717711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:36:03.717711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:36:03.717711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:36:03.717711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:36:03.717711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:36:03.717711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:36:03.717711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:36:03.717711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:36:03.717711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:36:03.717711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:36:03.717711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:36:03.717711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:36:03.717711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:36:03.717711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:36:03.718379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:36:03.718379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:36:03.718379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:36:03.718379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:03.718379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:36:03.718379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:03.718379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:36:03.718379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:03.718379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:36:03.718379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:03.718379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:36:03.718379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:03.718379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:36:03.718379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:36:03.718379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:03.718379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:36:03.718379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:03.718379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:36:03.718379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:36:03.719142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:03.719142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:36:03.719142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:03.719142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:36:03.719142 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:36:03.719142 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:36:03.719142 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:36:03.719142 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:36:03.719514 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:36:03.719514 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:36:03.719514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:03.719943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:36:03.719943 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:03.719943 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:36:03.719943 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:36:03.719943 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:03.719943 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:36:03.719943 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:03.719943 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:36:03.720558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:03.720558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:36:03.720558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:36:03.720558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:36:03.720558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:36:03.720558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:36:03.720558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:36:03.720558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:03.720558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:36:03.720558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:03.720558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:36:03.720558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:03.721168 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:36:03.721168 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:03.721168 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:36:03.721168 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:03.721168 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:36:03.721168 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:36:03.721168 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:03.721168 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:36:03.721168 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:03.721168 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:36:03.721168 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:36:03.721168 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:03.721168 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:36:03.721718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:03.721718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:36:03.721718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:03.721718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:36:03.721718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:36:03.721718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:36:03.721718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:36:03.721718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:36:03.721718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:36:03.721718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:36:03.721718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:36:03.721718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:36:03.721718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:36:03.721718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:36:03.721718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:36:03.722286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:36:03.722286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:36:03.722286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:36:03.722286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:36:03.722286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:36:03.722286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:03.722286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:36:03.722286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:03.722286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:36:03.722286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:03.722286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:36:03.722286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:03.722286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:36:03.722286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:03.722286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:36:03.722286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:36:03.722286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:03.722286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:36:03.722286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:03.722760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:36:03.722760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:36:03.722760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:03.722760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:36:03.722760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:03.722760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:36:08.559232 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:36:08.560200 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:36:08.560335 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:36:08.561263 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:36:08.598699 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:36:08.598897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:08.599715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:36:08.599715 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:08.599715 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:36:08.599715 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:36:08.599715 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:08.599715 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:36:08.599715 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:08.599715 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:36:08.600816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:08.600816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:36:08.600816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:36:08.600816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:36:08.600816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:36:08.600816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:36:08.600816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:36:08.600816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:08.600816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:36:08.600816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:08.600816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:36:08.600816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:08.601566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:36:08.601566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:08.601566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:36:08.601566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:08.601566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:36:08.601566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:36:08.601566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:08.601566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:36:08.601566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:08.601566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:36:08.601566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:36:08.601566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:08.601566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:36:08.602378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:08.602378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:36:08.602378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:08.602378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:36:08.602378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:36:08.602378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:36:08.602378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:36:08.602378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:36:08.602378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:36:08.602378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:36:08.602378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:36:08.602378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:36:08.602378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:36:08.602378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:36:08.602378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:36:08.603363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:36:08.603363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:36:08.603363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:36:08.603363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:36:08.603363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:36:08.603363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:08.603363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:36:08.603363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:08.603363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:36:08.603363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:08.603363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:36:08.603363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:08.603363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:36:08.603363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:08.603363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:36:08.603363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:36:08.603363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:08.603363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:36:08.603363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:08.605679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:36:08.605679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:36:08.605679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:08.605679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:36:08.605679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:08.605679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:36:08.696242 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:36:08.696973 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:36:08.697202 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:36:08.698278 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:36:08.698528 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:36:08.698678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:08.699411 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:36:08.699411 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:08.699411 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:36:08.699411 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:36:08.699411 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:08.699411 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:36:08.699411 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:08.699411 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:36:08.701137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:08.701137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:36:08.701137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:36:08.701137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:36:08.701137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:36:08.701137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:36:08.701137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:36:08.701137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:08.701137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:36:08.701137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:08.701137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:36:08.701137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:08.701548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:36:08.701548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:08.701548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:36:08.701548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:08.701548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:36:08.701548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:36:08.701548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:08.701548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:36:08.701548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:08.701548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:36:08.701548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:36:08.701548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:08.701548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:36:08.701951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:08.701951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:36:08.703197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:08.703197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:36:08.703197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:36:08.703197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:36:08.703197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:36:08.703197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:36:08.703197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:36:08.703197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:36:08.703197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:36:08.703197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:36:08.703197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:36:08.703197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:36:08.703197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:36:08.703197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:36:08.703197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:36:08.703197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:36:08.704016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:36:08.704016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:36:08.704016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:08.704016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:36:08.704016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:08.704016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:36:08.704016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:08.704016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:36:08.704016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:08.704016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:36:08.704016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:08.704016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:36:08.704016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:36:08.704016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:08.704016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:36:08.704016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:08.704016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:36:08.704016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:36:08.704016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:08.704883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:36:08.704883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:08.704883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:36:08.704883 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:36:08.704883 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:36:08.704883 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:36:08.704883 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:36:08.704883 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:36:08.705454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:08.706084 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:36:08.706084 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:08.706084 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:36:08.706084 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:36:08.706084 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:08.706084 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:36:08.706084 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:08.706084 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:36:08.706930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:08.706930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:36:08.706930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:36:08.706930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:36:08.706930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:08.706930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:36:08.706930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:08.706930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:36:08.706930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:08.706930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:36:08.706930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:08.706930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:36:08.706930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:08.706930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:36:08.706930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:36:08.707781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:08.707781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:36:08.707781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:08.707781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:36:08.707781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:36:08.707781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:08.707781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:36:08.707781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:08.707781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:36:08.707781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:08.707781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:36:08.707781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:36:08.707781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:36:08.707781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:36:08.708501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:36:08.708501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:36:08.708501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:36:08.708501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:36:08.708501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:36:08.708501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:08.708501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:36:08.708501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:08.708501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:36:08.708501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:08.708501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:36:08.708501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:08.708501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:36:08.709086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:08.709086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:36:08.709086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:36:08.709086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:08.709086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:36:08.709086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:08.709086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:36:08.709086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:36:08.709086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:08.709086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:36:08.709086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:08.709086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:36:08.709696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:08.709696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:36:08.709696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:36:08.709696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:36:08.709696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:36:08.709696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:36:08.709696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:36:08.709696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:36:08.709696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:36:08.709696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:36:08.709696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:36:08.709696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:36:08.709696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:36:08.709696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:36:08.709696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:36:08.709696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:36:08.710637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:36:08.710637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:36:08.710637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:36:08.710637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:36:08.710637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:36:08.710637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:08.710637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:36:08.710637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:08.710637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:36:08.710637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:08.710637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:36:08.710637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:08.710637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:36:08.710637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:08.710637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:36:08.710637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:36:08.710637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:08.710637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:36:08.711499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:08.711499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:36:08.711499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:36:08.711499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:08.711499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:36:08.711499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:08.711499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:36:08.713379 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:36:08.713735 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:36:08.713868 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:36:08.713996 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:36:08.714471 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:36:08.714852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:08.715577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:36:08.715577 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:08.715577 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:36:08.715577 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:36:08.715577 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:08.715577 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:36:08.715577 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:08.715577 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:36:08.716354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:08.716354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:36:08.716354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:36:08.716354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:36:08.716354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:08.716354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:36:08.716354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:08.716354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:36:08.716354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:08.716354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:36:08.716354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:08.716354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:36:08.716354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:08.716354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:36:08.716354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:36:08.716765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:08.716765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:36:08.716765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:08.716765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:36:08.716765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:36:08.716765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:08.716765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:36:08.716765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:08.716765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:36:08.717424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:08.717424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:36:08.717424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:36:08.717424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:36:08.717424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:36:08.717424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:36:08.717424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:36:08.717424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:36:08.717424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:36:08.717424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:08.717424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:36:08.717809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:08.717809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:36:08.717809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:08.717809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:36:08.717809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:08.717809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:36:08.717809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:08.717809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:36:08.717809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:36:08.717809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:08.717809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:36:08.717809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:08.717809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:36:08.717809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:36:08.718170 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:08.718170 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:36:08.718170 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:08.718170 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:36:08.719237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:08.719237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:36:08.719237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:36:08.719237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:36:08.719237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:36:08.719237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:36:08.719237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:36:08.719237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:36:08.719237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:36:08.719237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:36:08.719237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:36:08.719237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:36:08.719237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:36:08.719237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:36:08.719237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:36:08.719237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:36:08.719237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:36:08.719754 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:36:08.719754 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:36:08.719754 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:36:08.719754 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:08.719754 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:36:08.719754 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:08.719754 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:36:08.719754 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:08.719754 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:36:08.719754 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:08.719754 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:36:08.719754 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:08.719754 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:36:08.719754 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:36:08.719754 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:08.719754 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:36:08.719754 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:08.719754 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:36:08.719754 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:36:08.720426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:08.720426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:36:08.720426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:08.720426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:36:08.720426 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:36:08.720586 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:36:08.720635 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:36:08.720878 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:36:08.721668 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:36:08.721851 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:36:08.721973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:08.722662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:36:08.722662 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:08.722662 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:36:08.722662 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:36:08.722662 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:08.722662 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:36:08.722662 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:08.722662 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:36:08.723696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:08.723696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:36:08.723696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:36:08.723696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:36:08.723696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:36:08.723696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:36:08.723696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:36:08.723696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:08.723696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:36:08.723696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:08.723696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:36:08.723696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:08.724307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:36:08.724307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:08.724307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:36:08.724307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:08.724307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:36:08.724307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:36:08.724307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:08.724307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:36:08.724307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:08.724307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:36:08.724307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:36:08.724307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:08.724307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:36:08.724876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:08.724876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:36:08.725000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:08.725000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:36:08.725000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:36:08.725000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:36:08.725000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:36:08.725000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:36:08.725000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:36:08.725000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:36:08.725000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:36:08.725000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:36:08.725000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:36:08.725000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:36:08.725000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:36:08.725000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:36:08.725000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:36:08.725000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:36:08.725645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:36:08.725645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:36:08.725645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:08.725645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:36:08.725645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:08.725645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:36:08.725645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:08.725645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:36:08.725645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:08.725645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:36:08.725645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:08.725645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:36:08.725645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:36:08.725645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:08.725645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:36:08.725645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:08.725645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:36:08.725645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:36:08.725645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:08.726456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:36:08.726456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:08.726456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:36:13.559887 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:36:13.560953 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:36:13.560953 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:36:13.561181 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:36:13.596303 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:36:13.596840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:13.597764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:36:13.597764 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:13.597764 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:36:13.597764 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:36:13.597764 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:13.597764 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:36:13.597764 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:13.597764 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:36:13.598893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:13.598893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:36:13.598893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:36:13.598893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:36:13.598893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:36:13.598893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:36:13.598893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:36:13.598893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:13.598893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:36:13.598893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:13.598893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:36:13.598893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:13.599609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:36:13.599609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:13.599609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:36:13.599609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:13.599609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:36:13.599609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:36:13.599609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:13.599609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:36:13.599609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:13.599609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:36:13.599609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:36:13.599609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:13.599609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:36:13.600507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:13.600507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:36:13.600507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:13.600507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:36:13.600507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:36:13.600507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:36:13.600507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:36:13.600507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:36:13.600507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:36:13.600507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:36:13.600507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:36:13.600507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:36:13.600507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:36:13.600507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:36:13.600507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:36:13.602672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:36:13.602672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:36:13.602672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:36:13.602672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:36:13.602672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:36:13.602672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:13.602672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:36:13.602672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:13.602672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:36:13.602672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:13.602672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:36:13.602672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:13.602672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:36:13.602672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:13.602672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:36:13.602672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:36:13.602672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:13.602672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:36:13.602672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:13.603505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:36:13.603505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:36:13.603505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:13.603505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:36:13.603505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:13.603505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:36:13.697265 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:36:13.697995 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:36:13.698227 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:36:13.699460 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:36:13.699925 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:36:13.700193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:13.701269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:36:13.701269 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:13.701269 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:36:13.701269 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:36:13.701269 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:13.701269 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:36:13.701269 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:13.701269 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:36:13.702846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:13.702846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:36:13.702846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:36:13.702846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:36:13.702846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:36:13.702846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:36:13.702846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:36:13.702846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:13.702846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:36:13.702846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:13.702846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:36:13.702846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:13.703446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:36:13.703446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:13.703446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:36:13.703446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:13.703446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:36:13.703446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:36:13.703446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:13.703446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:36:13.703446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:13.703446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:36:13.703446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:36:13.703446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:13.703446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:36:13.704042 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:13.704042 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:36:13.705066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:13.705066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:36:13.705066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:36:13.705066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:36:13.705066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:36:13.705066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:36:13.705066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:36:13.705066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:36:13.705066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:36:13.705066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:36:13.705066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:36:13.705066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:36:13.705066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:36:13.705066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:36:13.705066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:36:13.705066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:36:13.707009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:36:13.707009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:36:13.707009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:13.707009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:36:13.707009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:13.707009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:36:13.707009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:13.707009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:36:13.707009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:13.707009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:36:13.707009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:13.707009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:36:13.707009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:36:13.707009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:13.707009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:36:13.707009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:13.707009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:36:13.707009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:36:13.707009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:13.708566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:36:13.708566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:13.708566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:36:13.708566 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:36:13.708566 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:36:13.708566 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:36:13.708566 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:36:13.708566 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:36:13.709017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:13.709017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:36:13.709017 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:13.709017 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:36:13.709017 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:36:13.709017 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:13.709017 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:36:13.709017 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:13.709017 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:36:13.709017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:13.709017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:36:13.709017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:36:13.709017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:36:13.709017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:13.709017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:36:13.709017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:13.709017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:36:13.710294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:13.710294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:36:13.710294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:13.710294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:36:13.710294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:13.710294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:36:13.710294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:36:13.710294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:13.710294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:36:13.710294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:13.710294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:36:13.710294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:36:13.710294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:13.710294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:36:13.710294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:13.710294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:36:13.711524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:13.711524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:36:13.711524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:36:13.711524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:36:13.711524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:36:13.711524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:36:13.711524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:36:13.711524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:36:13.711524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:36:13.711524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:36:13.711524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:13.711524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:36:13.712211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:13.712211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:36:13.712211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:13.712211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:36:13.712211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:13.712211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:36:13.712211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:13.712211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:36:13.712211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:36:13.712211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:13.712211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:36:13.712211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:13.712211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:36:13.712211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:36:13.712960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:13.712960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:36:13.712960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:13.712960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:36:13.712960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:13.712960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:36:13.712960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:36:13.712960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:36:13.712960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:36:13.712960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:36:13.712960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:36:13.712960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:36:13.712960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:36:13.712960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:36:13.713306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:36:13.713306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:36:13.713306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:36:13.713306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:36:13.713306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:36:13.713306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:36:13.713306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:36:13.713306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:36:13.713306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:36:13.713306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:36:13.713306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:36:13.713306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:13.713306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:36:13.713306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:13.713306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:36:13.713306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:13.713306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:36:13.713306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:13.713306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:36:13.713810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:13.713810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:36:13.713810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:36:13.713810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:13.713810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:36:13.713810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:13.713810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:36:13.713810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:36:13.713810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:13.713810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:36:13.713810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:13.713810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:36:13.714093 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:36:13.714229 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:36:13.714343 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:36:13.714529 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:36:13.714940 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:36:13.715177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:13.715711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:36:13.715711 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:13.715711 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:36:13.715711 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:36:13.715711 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:13.715711 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:36:13.715711 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:13.715711 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:36:13.716491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:13.716491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:36:13.716491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:36:13.716491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:36:13.716491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:13.716491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:36:13.716491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:13.716491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:36:13.716491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:13.716491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:36:13.716491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:13.716491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:36:13.716491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:13.716491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:36:13.716491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:36:13.716771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:13.716771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:36:13.716771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:13.716771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:36:13.716771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:36:13.716771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:13.716771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:36:13.716771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:13.716771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:36:13.717411 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:13.717411 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:36:13.717411 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:36:13.717411 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:36:13.717411 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:36:13.717411 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:36:13.717411 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:36:13.717411 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:36:13.717411 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:36:13.717411 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:13.717411 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:36:13.717640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:13.717640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:36:13.717640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:13.717640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:36:13.717640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:13.717640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:36:13.717640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:13.717640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:36:13.717640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:36:13.717640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:13.717640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:36:13.717640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:13.717640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:36:13.717640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:36:13.717925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:13.717925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:36:13.717925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:13.717925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:36:13.718577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:13.718577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:36:13.718577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:36:13.718577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:36:13.718577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:36:13.718577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:36:13.718577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:36:13.718577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:36:13.718577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:36:13.718577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:36:13.718577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:36:13.718577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:36:13.718577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:36:13.718577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:36:13.718577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:36:13.718577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:36:13.718577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:36:13.718922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:36:13.718922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:36:13.718922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:36:13.718922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:13.718922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:36:13.718922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:13.718922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:36:13.718922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:13.718922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:36:13.718922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:13.718922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:36:13.718922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:13.718922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:36:13.718922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:36:13.718922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:13.718922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:36:13.718922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:13.718922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:36:13.718922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:36:13.719258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:13.719258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:36:13.719258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:13.719258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:36:13.720622 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:36:13.720980 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:36:13.721033 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:36:13.721205 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:36:13.722138 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:36:13.722451 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:36:13.722650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:13.723629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:36:13.723629 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:13.723629 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:36:13.723629 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:36:13.723629 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:13.723629 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:36:13.723629 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:13.723629 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:36:13.724536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:13.724536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:36:13.724536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:36:13.724536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:36:13.724536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:36:13.724536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:36:13.724536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:36:13.724536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:13.724536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:36:13.724536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:13.724536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:36:13.724536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:13.724928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:36:13.724928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:13.724928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:36:13.724928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:13.724928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:36:13.724928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:36:13.724928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:13.724928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:36:13.724928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:13.724928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:36:13.724928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:36:13.724928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:13.724928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:36:13.725372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:13.725372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:36:13.725563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:13.725563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:36:13.725563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:36:13.725563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:36:13.725563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:36:13.725563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:36:13.725563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:36:13.725563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:36:13.725563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:36:13.725563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:36:13.725563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:36:13.725563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:36:13.725563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:36:13.725563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:36:13.725563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:36:13.725563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:36:13.725984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:36:13.725984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:36:13.725984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:13.725984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:36:13.725984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:13.725984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:36:13.725984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:13.725984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:36:13.725984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:13.725984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:36:13.725984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:13.725984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:36:13.725984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:36:13.725984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:13.725984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:36:13.725984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:13.725984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:36:13.725984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:36:13.725984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:13.726471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:36:13.726471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:13.726471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:36:18.560530 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:36:18.561407 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:36:18.561482 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:36:18.563585 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:36:18.603103 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:36:18.603365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:18.604115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:36:18.604115 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:18.604115 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:36:18.604115 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:36:18.604115 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:18.604115 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:36:18.604115 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:18.604115 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:36:18.605206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.043 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:18.605206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:36:18.605206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:36:18.605206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:36:18.605206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:36:18.605206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:36:18.605206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:36:18.605206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:18.605206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:36:18.605206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:18.605206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:36:18.605206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:18.605915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:36:18.605915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:18.605915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:36:18.605915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:18.605915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:36:18.605915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:36:18.605915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:18.605915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:36:18.605915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:18.605915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:36:18.605915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:36:18.605915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:18.605915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:36:18.606587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:18.606587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:36:18.606587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:18.606587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:36:18.606587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:36:18.606587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:36:18.606587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:36:18.606587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:36:18.606587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:36:18.606587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:36:18.606587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:36:18.606587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:36:18.606587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:36:18.606587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:36:18.606587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:36:18.607437 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:36:18.607437 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:36:18.607437 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:36:18.607437 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:36:18.607437 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:36:18.607437 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:18.607437 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:36:18.607437 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:18.607437 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:36:18.607437 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:18.607437 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:36:18.607437 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:18.607437 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:36:18.607437 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:18.607437 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:36:18.607437 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:36:18.607437 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:18.607437 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:36:18.607437 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:18.609082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:36:18.609082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:36:18.609082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:18.609082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:36:18.609082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:18.609082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:36:18.698163 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:36:18.698775 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:36:18.699013 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:36:18.700416 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:36:18.700854 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:36:18.701183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:18.702077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:36:18.702077 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:18.702077 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:36:18.702077 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:36:18.702077 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:18.702077 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:36:18.702077 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:18.702077 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:36:18.703712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:18.703712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:36:18.703712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:36:18.703712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:36:18.703712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:36:18.703712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:36:18.703712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:36:18.703712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:18.703712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:36:18.703712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:18.703712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:36:18.703712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:18.704360 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:36:18.704360 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:18.704360 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:36:18.704360 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:18.704360 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:36:18.704360 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:36:18.704360 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:18.704360 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:36:18.704360 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:18.704360 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:36:18.704360 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:36:18.704360 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:18.704360 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:36:18.705163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:18.705163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:36:18.705726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:18.705726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:36:18.705726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:36:18.705726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:36:18.705726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:36:18.705726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:36:18.705726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:36:18.705726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:36:18.705726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:36:18.705726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:36:18.705726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:36:18.705726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:36:18.705726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:36:18.705726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:36:18.705726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:36:18.705726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:36:18.706212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:36:18.706212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:36:18.706212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:18.706212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:36:18.706212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:18.706212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:36:18.706212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:18.706212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:36:18.706212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:18.706212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:36:18.706212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:18.706212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:36:18.706212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:36:18.706212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:18.706212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:36:18.706212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:18.706212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:36:18.706212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:36:18.706212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:18.706705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:36:18.706705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:18.706705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:36:18.706705 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:36:18.706705 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:36:18.706705 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:36:18.706705 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:36:18.706930 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:36:18.707048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:18.707546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:36:18.707546 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:18.707546 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:36:18.707546 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:36:18.707546 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:18.707546 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:36:18.707546 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:18.707546 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:36:18.708589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:18.708589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:36:18.708589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:36:18.708589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:36:18.708589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:18.708589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:36:18.708589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:18.708589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:36:18.708589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:18.708589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:36:18.708589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:18.708589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:36:18.708589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:18.708589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:36:18.708589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:36:18.709016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:18.709016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:36:18.709016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:18.709016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:36:18.709016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:36:18.709016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:18.709016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:36:18.709016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:18.709016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:36:18.709584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:18.709584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:36:18.709584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:36:18.709584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:36:18.709584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:36:18.709584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:36:18.709584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:36:18.709584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:36:18.709584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:36:18.709584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:36:18.709584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:18.709584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:36:18.709962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:18.709962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:36:18.709962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:18.709962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:36:18.709962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:18.709962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:36:18.709962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:18.709962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:36:18.709962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:36:18.709962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:18.709962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:36:18.709962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:18.709962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:36:18.709962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:36:18.710317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:18.710317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:36:18.710317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:18.710317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:36:18.710695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:18.710695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:36:18.710695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:36:18.710695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:36:18.710695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:36:18.710695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:36:18.710695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:36:18.710695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:36:18.710695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:36:18.710695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:36:18.710695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:36:18.710695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:36:18.710695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:36:18.710695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:36:18.710695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:36:18.710695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:36:18.711160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:36:18.711160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:36:18.711160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:36:18.711160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:36:18.711160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:36:18.711160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:18.711160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:36:18.711160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:18.711160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:36:18.711160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:18.711160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:36:18.711160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:18.711160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:36:18.711160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:18.711160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:36:18.711160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:36:18.711160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:18.711160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:36:18.711624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:18.711624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:36:18.711624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:36:18.711624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:18.711624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:36:18.711624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:18.711624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:36:18.714300 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:36:18.714623 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:36:18.714802 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:36:18.714977 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:36:18.715579 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:36:18.716153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:18.717257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:36:18.717257 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:18.717257 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:36:18.717257 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:36:18.717257 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:18.717257 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:36:18.717257 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:18.717257 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:36:18.718101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:18.718101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:36:18.718101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:36:18.718101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:36:18.718101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:18.718101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:36:18.718101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:18.718101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:36:18.718101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:18.718101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:36:18.718101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:18.718101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:36:18.718101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:18.718101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:36:18.718101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:36:18.718513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:18.718513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:36:18.718513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:18.718513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:36:18.718513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:36:18.718513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:18.718513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:36:18.718513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:18.718513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:36:18.719195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:18.719195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:36:18.719195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:36:18.719195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:36:18.719195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:36:18.719195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:36:18.719195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:36:18.719195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:36:18.719195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:36:18.719195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:18.719195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:36:18.719540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:18.719540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:36:18.719540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:18.719540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:36:18.719540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:18.719540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:36:18.719540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:18.719540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:36:18.719540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:36:18.719540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:18.719540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:36:18.719540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:18.719540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:36:18.719540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:36:18.719925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:18.719925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:36:18.719925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:18.719925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:36:18.720438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:18.720438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:36:18.720438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:36:18.720438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:36:18.720438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:36:18.720438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:36:18.720438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:36:18.720438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:36:18.720438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:36:18.720438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:36:18.720438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:36:18.720438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:36:18.720438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:36:18.720438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:36:18.720438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:36:18.720438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:36:18.720438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:36:18.720914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:36:18.720914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:36:18.720914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:36:18.720914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:18.720914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:36:18.720914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:18.720914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:36:18.720914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:18.720914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:36:18.720914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:18.720914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:36:18.720914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:18.720914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:36:18.720914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:36:18.720914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:18.720914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:36:18.720914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:18.720914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:36:18.720914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:36:18.721455 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:18.721455 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:36:18.721455 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:18.721455 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:36:18.721455 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:36:18.721455 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:36:18.721455 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:36:18.721661 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:36:18.721979 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:36:18.722323 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:36:18.722607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:18.723472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:36:18.723472 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:18.723472 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:36:18.723472 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:36:18.723472 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:18.723472 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:36:18.723472 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:18.723472 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:36:18.724658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:18.724658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:36:18.724658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:36:18.724658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:36:18.724658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:36:18.724658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:36:18.724658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:36:18.724658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:18.724658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:36:18.724658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:18.724658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:36:18.724658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:18.725344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:36:18.725344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:18.725344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:36:18.725344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:18.725344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:36:18.725344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:36:18.725344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:18.725344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:36:18.725344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:18.725344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:36:18.725344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:36:18.725344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:18.725344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:36:18.725865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:18.725865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:36:18.726290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:18.726290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:36:18.726290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:36:18.726290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:36:18.726290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:36:18.726290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:36:18.726290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:36:18.726290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:36:18.726290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:36:18.726290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:36:18.726290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:36:18.726290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:36:18.726290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:36:18.726290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:36:18.726290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:36:18.726290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:36:18.727125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:36:18.727125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:36:18.727125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:18.727125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:36:18.727125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:18.727125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:36:18.727125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:18.727125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:36:18.727125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:18.727125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:36:18.727125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:18.727125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:36:18.727125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:36:18.727125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:18.727125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:36:18.727125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:18.727125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:36:18.727125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:36:18.727125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:18.727637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:36:18.727637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:18.727637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:36:23.561095 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:36:23.561860 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:36:23.562996 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:36:23.564584 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:36:23.599696 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:36:23.599696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:23.599696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:36:23.599696 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:23.599696 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:36:23.599696 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:36:23.599696 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:23.599696 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:36:23.599696 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:23.599696 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:36:23.600585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:23.600585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:36:23.600585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:36:23.600585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:36:23.600585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:36:23.600585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:36:23.600585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:36:23.600585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:23.600585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:36:23.600585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:23.600585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:36:23.600585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:23.601018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:36:23.601018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:23.601018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:36:23.601018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:23.601018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:36:23.601018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:36:23.601018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:23.601018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:36:23.601018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:23.601018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:36:23.601018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:36:23.601018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:23.601018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:36:23.601387 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:23.601387 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:36:23.601872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:23.601872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:36:23.601872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:36:23.601872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:36:23.601872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:36:23.601872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:36:23.601872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:36:23.601872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:36:23.601872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:36:23.601872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:36:23.601872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:36:23.601872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:36:23.601872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:36:23.601872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:36:23.601872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:36:23.601872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:36:23.603509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:36:23.603509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:36:23.603509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:23.603509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:36:23.603509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:23.603509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:36:23.603509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:23.603509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:36:23.603509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:23.603509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:36:23.603509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:23.603509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:36:23.603509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:36:23.603509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:23.603509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:36:23.603509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:23.603509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:36:23.603509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:36:23.603509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:23.604432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:36:23.604432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:23.604432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:36:23.698949 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:36:23.699521 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:36:23.699709 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:36:23.701771 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:36:23.701771 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:36:23.701771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:23.701934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:36:23.701934 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:23.701934 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:36:23.701934 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:36:23.701934 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:23.701934 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:36:23.701934 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:23.701934 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:36:23.703217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:23.703217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:36:23.703217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:36:23.703217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:36:23.703217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:36:23.703217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:36:23.703217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:36:23.703217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:23.703217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:36:23.703217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:23.703217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:36:23.703217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:23.703640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:36:23.703640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:23.703640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:36:23.703640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:23.703640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:36:23.703640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:36:23.703640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:23.703640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:36:23.703640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:23.703640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:36:23.703640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:36:23.703640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:23.703640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:36:23.704066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:23.704066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:36:23.707339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:23.707339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:36:23.707339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:36:23.707339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:36:23.707339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:36:23.707339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:36:23.707339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:36:23.707339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:36:23.707339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:36:23.707339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:36:23.707339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:36:23.707339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:36:23.707339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:36:23.707339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:36:23.707339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:36:23.707339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:36:23.708251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:36:23.708251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:36:23.708251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:23.708251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:36:23.708251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:23.708251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:36:23.708251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:23.708251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:36:23.708251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:23.708251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:36:23.708251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:23.708251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:36:23.708251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:36:23.708251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:23.708251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:36:23.708251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:23.708251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:36:23.708251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:36:23.708251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:23.708794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:36:23.708794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:23.708794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:36:23.708794 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:36:23.708794 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:36:23.708794 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:36:23.708794 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:36:23.710069 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:36:23.710069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:23.710275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:36:23.710275 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:23.710275 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:36:23.710275 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:36:23.710275 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:23.710275 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:36:23.710275 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:23.710275 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:36:23.710946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:23.710946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:36:23.710946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:36:23.710946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:36:23.710946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:23.710946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:36:23.710946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:23.710946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:36:23.710946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:23.710946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:36:23.710946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:23.710946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:36:23.710946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:23.710946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:36:23.710946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:36:23.711649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:23.711649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:36:23.711649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:23.711649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:36:23.711649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:36:23.711649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:23.711649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:36:23.711649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:23.711649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:36:23.711649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:23.711649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:36:23.711649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:36:23.711649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:36:23.711649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:36:23.712154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:36:23.712154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:36:23.712154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:36:23.712154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:36:23.712154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:36:23.712154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:23.712154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:36:23.712154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:23.712154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:36:23.712154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:23.712154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:36:23.712154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:23.712154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:36:23.712512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:23.712512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:36:23.712512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:36:23.712512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:23.712512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:36:23.712512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:23.712512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:36:23.712512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:36:23.712512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:23.712512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:36:23.712512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:23.712512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:36:23.715554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:23.715554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:36:23.715554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:36:23.715554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:36:23.715554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:36:23.715554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:36:23.715554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:36:23.715554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:36:23.715554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:36:23.715554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:36:23.715554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:36:23.715554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:36:23.715554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:36:23.715554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:36:23.715554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:36:23.715554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:36:23.716301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:36:23.716301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:36:23.716301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:36:23.716301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:36:23.716301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:36:23.716301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:23.716301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:36:23.716301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:23.716301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:36:23.716301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:23.716301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:36:23.716301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:23.716301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:36:23.716301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:23.716301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:36:23.716301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:36:23.716301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:23.716301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:36:23.717180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:23.717180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:36:23.717180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:36:23.717180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:23.717180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:36:23.717180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:23.717180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:36:23.717180 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:36:23.717180 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:36:23.717180 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:36:23.717626 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:36:23.718583 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:36:23.718583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:23.718727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:36:23.718727 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:23.718727 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:36:23.718727 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:36:23.718727 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:23.718727 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:36:23.718727 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:23.718727 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:36:23.719339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:23.719339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:36:23.719339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:36:23.719339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:36:23.719339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:23.719339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:36:23.719339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:23.719339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:36:23.719339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:23.719339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:36:23.719339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:23.719339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:36:23.719339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:23.719339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:36:23.719339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:36:23.719958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:23.719958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:36:23.719958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:23.719958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:36:23.719958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:36:23.719958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:23.719958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:36:23.719958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:23.719958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:36:23.720223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:23.720223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:36:23.720223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:36:23.720223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:36:23.720223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:36:23.720223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:36:23.720223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:36:23.720223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:36:23.720223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:36:23.720223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:23.720223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:36:23.720548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:23.720548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:36:23.720548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:23.720548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:36:23.720548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:23.720548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:36:23.720548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:23.720548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:36:23.720548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:36:23.720548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:23.720548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:36:23.720548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:23.720548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:36:23.720548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:36:23.720918 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:23.720918 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:36:23.720918 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:23.720918 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:36:23.720918 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:23.720918 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:36:23.720918 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:36:23.720918 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:36:23.720918 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:36:23.720918 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:36:23.720918 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:36:23.720918 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:36:23.720918 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:36:23.720918 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:36:23.721299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:36:23.721299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:36:23.721299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:36:23.721299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:36:23.721299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:36:23.721299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:36:23.721299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:36:23.721299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:36:23.721299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:36:23.721299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:36:23.721299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:23.721299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:36:23.721299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:23.721299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:36:23.721299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:23.721299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:36:23.721299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:23.721299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:36:23.721299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:23.722862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:36:23.722862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:36:23.722862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:23.722862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:36:23.722862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:23.722862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:36:23.722862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:36:23.722862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:23.722862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:36:23.722862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:23.722862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:36:23.722862 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:36:23.722862 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:36:23.723237 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:36:23.723237 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:36:23.723237 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:36:23.723237 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:36:23.723237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:23.723237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:36:23.723237 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:23.723237 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:36:23.723237 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:36:23.723504 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:23.723504 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:36:23.723504 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:23.723504 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:36:23.723504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:23.723504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:36:23.723504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:36:23.723504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:36:23.723504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:36:23.723504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:36:23.723504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:36:23.723504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:23.723504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:36:23.723504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:23.723937 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:36:23.723937 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:23.723937 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:36:23.723937 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:23.723937 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:36:23.723937 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:23.723937 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:36:23.723937 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:36:23.723937 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:23.723937 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:36:23.723937 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:23.723937 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:36:23.723937 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:36:23.723937 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:23.724284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:36:23.724284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:23.724284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:36:23.724284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:23.724284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:36:23.724284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:36:23.724284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:36:23.724284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:36:23.724284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:36:23.724284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:36:23.724284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:36:23.724284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:36:23.724284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:36:23.724284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:36:23.724284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:36:23.724284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:36:23.724698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:36:23.724698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:36:23.724698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:36:23.724698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:36:23.724698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:36:23.724698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:23.724698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:36:23.724698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:23.724698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:36:23.724698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:23.724698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:36:23.724698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:23.724698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:36:23.724698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:23.724698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:36:23.724698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:36:23.724698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:23.724698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:36:23.724698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:23.725228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:36:23.725228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:36:23.725228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:23.725228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:36:23.725228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:23.725228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:36:28.561809 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:36:28.562886 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:36:28.562886 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:36:28.564108 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:36:28.595409 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:36:28.595883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:28.597010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:36:28.597010 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:28.597010 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:36:28.597010 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:36:28.597010 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:28.597010 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:36:28.597010 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:28.597010 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:36:28.597737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:28.597737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:36:28.597737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:36:28.597737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:36:28.597737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:36:28.597737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:36:28.597737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:36:28.597737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:28.597737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:36:28.597737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:28.597737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:36:28.597737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:28.598553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:36:28.598553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:28.598553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:36:28.598553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:28.598553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:36:28.598553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:36:28.598553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:28.598553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:36:28.598553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:28.598553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:36:28.598553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:36:28.598553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:28.598553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:36:28.600200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:28.600200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:36:28.600200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:28.600200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:36:28.600200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:36:28.600200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:36:28.600200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:36:28.600200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:36:28.600200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:36:28.600200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:36:28.600200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:36:28.600200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:36:28.600200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:36:28.600200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:36:28.600200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:36:28.602005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:36:28.602005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:36:28.602005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:36:28.602005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:36:28.602005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:36:28.602005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:28.602005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:36:28.602005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:28.602005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:36:28.602005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:28.602005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:36:28.602005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:28.602005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:36:28.602005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:28.602005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:36:28.602005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:36:28.602005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:28.602005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:36:28.602005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:28.602583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:36:28.602583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:36:28.602583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:28.602583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:36:28.602583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:28.602583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:36:28.699678 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:36:28.700356 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:36:28.700550 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:36:28.701659 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:36:28.702029 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:36:28.702316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:28.703344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:36:28.703344 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:28.703344 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:36:28.703344 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:36:28.703344 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:28.703344 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:36:28.703344 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:28.703344 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:36:28.705119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:28.705119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:36:28.705119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:36:28.705119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:36:28.705119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:36:28.705119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:36:28.705119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:36:28.705119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:28.705119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:36:28.705119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:28.705119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:36:28.705119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:28.706107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:36:28.706107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:28.706107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:36:28.706107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:28.706107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:36:28.706107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:36:28.706107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:28.706107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:36:28.706107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:28.706107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:36:28.706107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:36:28.706107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:28.706107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:36:28.706733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:28.706733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:36:28.708773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:28.708773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:36:28.708773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:36:28.708773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:36:28.708773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:36:28.708773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:36:28.708773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:36:28.708773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:36:28.708773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:36:28.708773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:36:28.708773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:36:28.708773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:36:28.708773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:36:28.708773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:36:28.708773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:36:28.708773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:36:28.710048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:36:28.710048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:36:28.710048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:28.710048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:36:28.710048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:28.710048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:36:28.710048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:28.710048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:36:28.710048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:28.710048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:36:28.710048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:28.710048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:36:28.710048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:36:28.710048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:28.710048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:36:28.710048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:28.710048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:36:28.710048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:36:28.710048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:28.710926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:36:28.710926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:28.710926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:36:28.710926 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:36:28.710926 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:36:28.710926 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:36:28.710926 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:36:28.710926 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:36:28.711368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:28.712228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:36:28.712228 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:28.712228 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:36:28.712228 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:36:28.712228 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:28.712228 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:36:28.712228 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:28.712228 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:36:28.713693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:28.713693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:36:28.713693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:36:28.713693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:36:28.713693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:28.713693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:36:28.713693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:28.713693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:36:28.713693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:28.713693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:36:28.713693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:28.713693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:36:28.713693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:28.713693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:36:28.713693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:36:28.714178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:28.714178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:36:28.714178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:28.714178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:36:28.714178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:36:28.714178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:28.714178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:36:28.714178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:28.714178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:36:28.715444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:28.715444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:36:28.715444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:36:28.715444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:36:28.715444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:36:28.715444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:36:28.715444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:36:28.715444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:36:28.715444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:36:28.715444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:36:28.715444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:28.715444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:36:28.716153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:28.716153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:36:28.716153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:28.716153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:36:28.716153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:28.716153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:36:28.716153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:28.716153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:36:28.716153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:36:28.716153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:28.716153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:36:28.716153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:28.716153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:36:28.716153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:36:28.716501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:28.716501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:36:28.716501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:28.716501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:36:28.717063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:28.717063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:36:28.717063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:36:28.717063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:36:28.717063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:36:28.717063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:36:28.717063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:36:28.717063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:36:28.717063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:36:28.717063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:36:28.717063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:36:28.717063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:36:28.717063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:36:28.717063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:36:28.717063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:36:28.717063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:36:28.718031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:36:28.718031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:36:28.718031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:36:28.718031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:36:28.718031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:36:28.718031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:28.718031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:36:28.718031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:28.718031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:36:28.718031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:28.718031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:36:28.718031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:28.718031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:36:28.718031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:28.718031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:36:28.718031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:36:28.718031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:28.718031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:36:28.718539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:28.718539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:36:28.718539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:36:28.718539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:28.718539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:36:28.718539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:28.718539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:36:28.718539 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:36:28.718539 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:36:28.718539 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:36:28.718873 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:36:28.719222 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:36:28.719772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:28.720923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:36:28.720923 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:28.720923 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:36:28.720923 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:36:28.720923 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:28.720923 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:36:28.720923 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:28.720923 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:36:28.722059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:28.722059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:36:28.722059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:36:28.722059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:36:28.722059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:28.722059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:36:28.722059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:28.722059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:36:28.722059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:28.722059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:36:28.722059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:28.722059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:36:28.722059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:28.722059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:36:28.722059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:36:28.722605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:28.722605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:36:28.722605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:28.722605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:36:28.722605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:36:28.722605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:28.722605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:36:28.722605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:28.722605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:36:28.723977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:28.723977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:36:28.723977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:36:28.723977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:36:28.723977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:36:28.723977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:36:28.723977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:36:28.723977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:36:28.723977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:36:28.723977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:28.723977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:36:28.724345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:28.724345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:36:28.724345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:28.724345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:36:28.724345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:28.724345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:36:28.724345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:28.724345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:36:28.724345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:36:28.724345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:28.724345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:36:28.724345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:28.724345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:36:28.724345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:36:28.724727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:28.724727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:36:28.724727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:28.724727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:36:28.724727 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:36:28.726562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:28.726562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:36:28.726562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:36:28.726562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:36:28.726562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:36:28.726562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:36:28.726562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:36:28.726562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:36:28.726562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:36:28.726562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:36:28.726562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:36:28.726562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:36:28.726562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:36:28.726562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:36:28.726562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:36:28.726562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:36:28.726562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:36:28.727124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:36:28.727124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:36:28.727124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:36:28.727124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:28.727124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:36:28.727124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:28.727124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:36:28.727124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:28.727124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:36:28.727124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:28.727124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:36:28.727124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:28.727124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:36:28.727124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:36:28.727124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:28.727124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:36:28.727124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:28.727124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:36:28.727124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:36:28.727672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:28.727672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:36:28.727672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:28.727672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:36:28.727672 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:36:28.727672 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:36:28.727672 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:36:28.727672 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:36:28.727977 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:36:28.728139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:28.728801 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:36:28.728801 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:28.728801 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:36:28.728801 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:36:28.728801 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:28.728801 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:36:28.728801 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:28.728801 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:36:28.729687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:28.729687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:36:28.729687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:36:28.729687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:36:28.729687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:36:28.729687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:36:28.729687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:36:28.729687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:28.729687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:36:28.729687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:28.729687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:36:28.729687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:28.730068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:36:28.730068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:28.730068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:36:28.730068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:28.730068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:36:28.730068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:36:28.730068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:28.730068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:36:28.730068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:28.730068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:36:28.730068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:36:28.730068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:28.730068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:36:28.730461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:28.730461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:36:28.730890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:28.730890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:36:28.730890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:36:28.730890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:36:28.730890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:36:28.730890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:36:28.730890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:36:28.730890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:36:28.730890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:36:28.730890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:36:28.730890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:36:28.730890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:36:28.730890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:36:28.730890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:36:28.730890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:36:28.730890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:36:28.731334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:36:28.731334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:36:28.731334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:28.731334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:36:28.731334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:28.731334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:36:28.731334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:28.731334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:36:28.731334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:28.731334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:36:28.731334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:28.731334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:36:28.731334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:36:28.731334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:28.731334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:36:28.731334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:28.731334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:36:28.731334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:36:28.731334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:28.731858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:36:28.731858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:28.731858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:36:33.562586 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:36:33.563285 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:36:33.563351 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:36:33.564714 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:36:33.607973 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:36:33.608412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:33.609422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:36:33.609422 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:33.609422 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:36:33.609422 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:36:33.609422 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:33.609422 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:36:33.609422 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:33.609422 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:36:33.610462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.046 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:33.610462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:36:33.610462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:36:33.610462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:36:33.610462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:36:33.610462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:36:33.610462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:36:33.610462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:33.610462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:36:33.610462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:33.610462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:36:33.610462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:33.612005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:36:33.612005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:33.612005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:36:33.612005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:33.612005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:36:33.612005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:36:33.612005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:33.612005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:36:33.612005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:33.612005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:36:33.612005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:36:33.612005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:33.612005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:36:33.612547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:33.612547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:36:33.612547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:33.612547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:36:33.612547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:36:33.612547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:36:33.612547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:36:33.612547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:36:33.612547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:36:33.612547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:36:33.612547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:36:33.612547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:36:33.612547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:36:33.612547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:36:33.612547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:36:33.613572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:36:33.613572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:36:33.613572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:36:33.613572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:36:33.613572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:36:33.613572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:33.613572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:36:33.613572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:33.613572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:36:33.613572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:33.613572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:36:33.613572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:33.613572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:36:33.613572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:33.613572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:36:33.613572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:36:33.613572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:33.613572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:36:33.613572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:33.614462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:36:33.614462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:36:33.614462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:33.614462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:36:33.614462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:33.614462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:36:33.700650 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:36:33.701196 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:36:33.701358 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:36:33.702637 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:36:33.703027 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:36:33.703394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:33.704564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:36:33.704564 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:33.704564 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:36:33.704564 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:36:33.704564 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:33.704564 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:36:33.704564 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:33.704564 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:36:33.706111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:33.706111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:36:33.706111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:36:33.706111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:36:33.706111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:36:33.706111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:36:33.706111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:36:33.706111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:33.706111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:36:33.706111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:33.706111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:36:33.706111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:33.706852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:36:33.706852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:33.706852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:36:33.706852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:33.706852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:36:33.706852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:36:33.706852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:33.706852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:36:33.706852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:33.706852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:36:33.706852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:36:33.706852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:33.706852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:36:33.707666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:33.707666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:36:33.708180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:33.708180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:36:33.708180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:36:33.708180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:36:33.708180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:36:33.708180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:36:33.708180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:36:33.708180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:36:33.708180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:36:33.708180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:36:33.708180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:36:33.708180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:36:33.708180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:36:33.708180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:36:33.708180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:36:33.708180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:36:33.708888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:36:33.708888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:36:33.708888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:33.708888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:36:33.708888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:33.708888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:36:33.708888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:33.708888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:36:33.708888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:33.708888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:36:33.708888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:33.708888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:36:33.708888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:36:33.708888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:33.708888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:36:33.708888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:33.708888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:36:33.708888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:36:33.708888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:33.710282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:36:33.710282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:33.710282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:36:33.710282 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:36:33.710282 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:36:33.710555 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:36:33.710662 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:36:33.711173 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:36:33.711463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:33.712238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:36:33.712238 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:33.712238 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:36:33.712238 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:36:33.712238 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:33.712238 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:36:33.712238 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:33.712238 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:36:33.713416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:33.713416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:36:33.713416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:36:33.713416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:36:33.713416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:33.713416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:36:33.713416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:33.713416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:36:33.713416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:33.713416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:36:33.713416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:33.713416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:36:33.713416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:33.713416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:36:33.713416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:36:33.714039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:33.714039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:36:33.714039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:33.714039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:36:33.714039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:36:33.714039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:33.714039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:36:33.714039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:33.714039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:36:33.714952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:33.714952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:36:33.714952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:36:33.714952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:36:33.714952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:36:33.714952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:36:33.714952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:36:33.714952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:36:33.714952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:36:33.714952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:36:33.714952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:33.714952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:36:33.715844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:33.715844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:36:33.715844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:33.715844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:36:33.715844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:33.715844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:36:33.715844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:33.715844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:36:33.715844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:36:33.715844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:33.715844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:36:33.715844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:33.715844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:36:33.715844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:36:33.716216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:33.716216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:36:33.716216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:33.716216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:36:33.716503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:33.716503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:36:33.716503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:36:33.716503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:36:33.716503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:36:33.716503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:36:33.716503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:36:33.716503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:36:33.716503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:36:33.716503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:36:33.716503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:36:33.716503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:36:33.716503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:36:33.716503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:36:33.716503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:36:33.716503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:36:33.716848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:36:33.716848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:36:33.716848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:36:33.716848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:36:33.716848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:36:33.716848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:33.716848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:36:33.716848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:33.716848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:36:33.716848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:33.716848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:36:33.716848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:33.716848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:36:33.716848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:33.716848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:36:33.716848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:36:33.716848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:33.716848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:36:33.717174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:33.717174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:36:33.717174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:36:33.717174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:33.717174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:36:33.717174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:33.717174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:36:33.717995 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:36:33.718205 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:36:33.718327 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:36:33.718456 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:36:33.718941 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:36:33.719178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:33.719756 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:36:33.719756 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:33.719756 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:36:33.719756 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:36:33.719756 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:33.719756 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:36:33.719756 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:33.719756 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:36:33.720538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:33.720538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:36:33.720538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:36:33.720538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:36:33.720538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:33.720538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:36:33.720538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:33.720538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:36:33.720538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:33.720538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:36:33.720538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:33.720538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:36:33.720538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:33.720538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:36:33.720538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:36:33.720852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:33.720852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:36:33.720852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:33.720852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:36:33.720852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:36:33.720852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:33.720852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:36:33.720852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:33.720852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:36:33.721421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:33.721421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:36:33.721421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:36:33.721421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:36:33.721421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:36:33.721421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:36:33.721421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:36:33.721421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:36:33.721421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:36:33.721421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:33.721421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:36:33.721984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:33.721984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:36:33.721984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:33.721984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:36:33.721984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:33.721984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:36:33.721984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:33.721984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:36:33.721984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:36:33.721984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:33.721984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:36:33.721984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:33.721984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:36:33.721984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:36:33.722238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:33.722238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:36:33.722238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:33.722238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:36:33.722595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:33.722595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:36:33.722595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:36:33.722595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:36:33.722595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:36:33.722595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:36:33.722595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:36:33.722595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:36:33.722595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:36:33.722595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:36:33.722595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:36:33.722595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:36:33.722595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:36:33.722595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:36:33.722595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:36:33.722595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:36:33.722595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:36:33.723210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:36:33.723210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:36:33.723210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:36:33.723210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:33.723210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:36:33.723210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:33.723210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:36:33.723210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:33.723210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:36:33.723210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:33.723210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:36:33.723210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:33.723210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:36:33.723210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:36:33.723210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:33.723210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:36:33.723210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:33.723210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:36:33.723210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:36:33.723550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:33.723550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:36:33.723550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:33.723550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:36:33.727024 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:36:33.727256 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:36:33.727402 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:36:33.727570 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:36:33.728056 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:36:33.728401 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:36:33.728621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:33.729377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:36:33.729377 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:33.729377 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:36:33.729377 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:36:33.729377 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:33.729377 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:36:33.729377 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:33.729377 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:36:33.730204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:33.730204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:36:33.730204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:36:33.730204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:36:33.730204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:36:33.730204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:36:33.730204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:36:33.730204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:33.730204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:36:33.730204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:33.730204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:36:33.730204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:33.730531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:36:33.730531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:33.730531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:36:33.730531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:33.730531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:36:33.730531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:36:33.730531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:33.730531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:36:33.730531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:33.730531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:36:33.730531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:36:33.730531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:33.730531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:36:33.730868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:33.730868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:36:33.731180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:33.731180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:36:33.731180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:36:33.731180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:36:33.731180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:36:33.731180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:36:33.731180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:36:33.731180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:36:33.731180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:36:33.731180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:36:33.731180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:36:33.731180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:36:33.731180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:36:33.731180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:36:33.731180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:36:33.731180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:36:33.731573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:36:33.731573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:36:33.731573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:33.731573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:36:33.731573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:33.731573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:36:33.731573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:33.731573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:36:33.731573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:33.731573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:36:33.731573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:33.731573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:36:33.731573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:36:33.731573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:33.731573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:36:33.731573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:33.731573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:36:33.731573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:36:33.731573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:33.732052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:36:33.732052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:33.732052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:36:38.563230 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:36:38.564053 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:36:38.564053 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:36:38.564862 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:36:38.604557 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:36:38.604557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:38.604763 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:36:38.604763 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:38.604763 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:36:38.604763 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:36:38.604763 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:38.604763 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:36:38.604763 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:38.604763 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:36:38.605579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:38.605579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:36:38.605579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:36:38.605579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:36:38.605579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:36:38.605579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:36:38.605579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:36:38.605579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:38.605579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:36:38.605579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:38.605579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:36:38.605579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:38.606566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:36:38.606566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:38.606566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:36:38.606566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:38.606566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:36:38.606566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:36:38.606566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:38.606566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:36:38.606566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:38.606566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:36:38.606566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:36:38.606566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:38.606566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:36:38.607303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:38.607303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:36:38.607303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:38.607303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:36:38.607303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:36:38.607303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:36:38.607303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:36:38.607303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:36:38.607303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:36:38.607303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:36:38.607303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:36:38.607303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:36:38.607303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:36:38.607303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:36:38.607303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:36:38.608132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:36:38.608132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:36:38.608132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:36:38.608132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:36:38.608132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:36:38.608132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:38.608132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:36:38.608132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:38.608132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:36:38.608132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:38.608132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:36:38.608132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:38.608132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:36:38.608132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:38.608132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:36:38.608132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:36:38.608132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:38.608132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:36:38.608132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:38.609694 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:36:38.609694 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:36:38.609694 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:38.609694 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:36:38.609694 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:38.609694 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:36:38.701177 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:36:38.702501 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:36:38.702607 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:36:38.704901 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:36:38.704901 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:36:38.704901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:38.705322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:36:38.705322 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:38.705322 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:36:38.705322 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:36:38.705322 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:38.705322 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:36:38.705322 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:38.705322 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:36:38.706336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:38.706336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:36:38.706336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:36:38.706336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:36:38.706336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:36:38.706336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:36:38.706336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:36:38.706336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:38.706336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:36:38.706336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:38.706336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:36:38.706336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:38.707144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:36:38.707144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:38.707144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:36:38.707144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:38.707144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:36:38.707144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:36:38.707144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:38.707144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:36:38.707144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:38.707144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:36:38.707144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:36:38.707144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:38.707144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:36:38.707793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:38.707793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:36:38.707793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:38.707793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:36:38.707793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:36:38.707793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:36:38.707793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:36:38.707793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:36:38.707793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:36:38.707793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:36:38.707793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:36:38.707793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:36:38.707793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:36:38.707793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:36:38.707793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:36:38.708468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:36:38.708468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:36:38.708468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:36:38.708468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:36:38.708468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:36:38.708468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:38.708468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:36:38.708468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:38.708468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:36:38.708468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:38.708468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:36:38.708468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:38.708468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:36:38.708468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:38.708468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:36:38.708468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:36:38.708468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:38.708468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:36:38.708468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:38.709296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:36:38.709296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:36:38.709296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:38.709296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:36:38.709296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:38.709296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:36:38.710772 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:36:38.711195 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:36:38.711329 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:36:38.711448 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:36:38.711843 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:36:38.712032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:38.712639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:36:38.712639 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:38.712639 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:36:38.712639 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:36:38.712639 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:38.712639 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:36:38.712639 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:38.712639 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:36:38.715151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:38.715151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:36:38.715151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:36:38.715151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:36:38.715151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:38.715151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:36:38.715151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:38.715151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:36:38.715151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:38.715151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:36:38.715151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:38.715151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:36:38.715151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:38.715151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:36:38.715151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:36:38.715649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:38.715649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:36:38.715649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:38.715649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:36:38.715649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:36:38.715649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:38.715649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:36:38.715649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:38.715649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:36:38.716903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:38.716903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:36:38.716903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:36:38.716903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:36:38.716903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:36:38.716903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:36:38.716903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:36:38.716903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:36:38.716903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:36:38.716903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:36:38.716903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:38.716903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:36:38.717443 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:38.717443 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:36:38.717443 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:38.717443 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:36:38.717443 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:38.717443 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:36:38.717443 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:38.717443 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:36:38.717443 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:36:38.717443 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:38.717443 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:36:38.717443 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:38.717443 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:36:38.717443 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:36:38.717860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:38.717860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:36:38.717860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:38.717860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:36:38.719004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:38.719004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:36:38.719004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:36:38.719004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:36:38.719004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:36:38.719004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:36:38.719004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:36:38.719004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:36:38.719004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:36:38.719004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:36:38.719004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:36:38.719004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:36:38.719004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:36:38.719004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:36:38.719004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:36:38.719004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:36:38.719492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:36:38.719492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:36:38.719492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:36:38.719492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:36:38.719492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:36:38.719492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:38.719492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:36:38.719492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:38.719492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:36:38.719492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:38.719492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:36:38.719492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:38.719492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:36:38.719492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:38.719492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:36:38.719492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:36:38.719492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:38.719492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:36:38.720069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:38.720069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:36:38.720069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:36:38.720069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:38.720069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:36:38.720069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:38.720069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:36:38.720069 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:36:38.721690 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:36:38.721690 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:36:38.721690 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:36:38.722200 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:36:38.722592 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:38.724900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:36:38.724900 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:38.724900 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:36:38.724900 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:36:38.724900 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:38.724900 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:36:38.724900 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:38.724900 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:36:38.724900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:38.724900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:36:38.724900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:36:38.724900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:36:38.724900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:38.724900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:36:38.724900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:38.724900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:36:38.724900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:38.724900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:36:38.725655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:38.725655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:36:38.725655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:38.725655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:36:38.725655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:36:38.725655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:38.725655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:36:38.725655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:38.725655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:36:38.725655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:36:38.725655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:38.725655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:36:38.725655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:38.725655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:36:38.725655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:38.727587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:36:38.727587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:36:38.727587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:36:38.727587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:36:38.727587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:36:38.727587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:36:38.727587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:36:38.727587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:36:38.727587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:38.727587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:36:38.727587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:38.727587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:36:38.727587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:38.728434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:36:38.728434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:38.728434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:36:38.728434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:38.728434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:36:38.728434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:36:38.728434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:38.728434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:36:38.728434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:38.728434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:36:38.728434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:36:38.728434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:38.728434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:36:38.729109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:38.729109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:36:38.729109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:38.729109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:36:38.729109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:36:38.729109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:36:38.729109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:36:38.729109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:36:38.729109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:36:38.729109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:36:38.729109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:36:38.729109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:36:38.729109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:36:38.729109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:36:38.729109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:36:38.730211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:36:38.730211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:36:38.730211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:36:38.730211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:36:38.730211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:36:38.730211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:36:38.730211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:36:38.730211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:38.730211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:36:38.730211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:38.730211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:36:38.730211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:38.730211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:36:38.730211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:38.730211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:36:38.730211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:38.730211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:36:38.730211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:36:38.730211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:38.730800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:36:38.730800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:38.730800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:36:38.730800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:36:38.730800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:38.730800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:36:38.730800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:38.730800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:36:38.730800 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:36:38.730800 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:36:38.730800 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:36:38.731199 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:36:38.731199 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:36:38.731199 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:36:38.731199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:38.731199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:36:38.731199 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:38.731199 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:36:38.731199 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:36:38.731199 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:38.731199 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:36:38.731199 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:38.731199 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:36:38.731594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:38.731594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:36:38.731594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:36:38.731594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:36:38.731594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:36:38.731594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:36:38.731594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:36:38.731594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:38.731594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:36:38.731594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:38.731594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:36:38.731594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:38.732030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:36:38.732030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:38.732030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:36:38.732030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:38.732030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:36:38.732030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:36:38.732030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:38.732030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:36:38.732030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:38.732030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:36:38.732030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:36:38.732030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:38.732030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:36:38.732389 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:38.732389 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:36:38.732389 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:38.732389 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:36:38.732389 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:36:38.732389 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:36:38.732389 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:36:38.732389 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:36:38.732389 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:36:38.732389 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:36:38.732389 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:36:38.732389 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:36:38.732389 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:36:38.732389 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:36:38.732389 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:36:38.732971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:36:38.732971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:36:38.732971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:36:38.732971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:36:38.732971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:36:38.732971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:38.732971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:36:38.732971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:38.732971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:36:38.732971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:38.732971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:36:38.732971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:38.732971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:36:38.732971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:38.732971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:36:38.732971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:36:38.732971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:38.732971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:36:38.732971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:38.733522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:36:38.733522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:36:38.733522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:38.733522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:36:38.733522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:38.733522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:36:43.563777 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:36:43.564595 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:36:43.564595 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:36:43.566397 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:36:43.601070 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:36:43.601463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:43.602209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:36:43.602209 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:43.602209 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:36:43.602209 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:36:43.602209 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:43.602209 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:36:43.602209 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:43.602209 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:36:43.603307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:43.603307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:36:43.603307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:36:43.603307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:36:43.603307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:36:43.603307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:36:43.603307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:36:43.603307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:43.603307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:36:43.603307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:43.603307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:36:43.603307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:43.604140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:36:43.604140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:43.604140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:36:43.604140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:43.604140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:36:43.604140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:36:43.604140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:43.604140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:36:43.604140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:43.604140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:36:43.604140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:36:43.604140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:43.604140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:36:43.604871 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:43.604871 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:36:43.604871 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:43.604871 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:36:43.604871 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:36:43.604871 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:36:43.604871 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:36:43.604871 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:36:43.604871 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:36:43.604871 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:36:43.604871 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:36:43.604871 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:36:43.604871 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:36:43.604871 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:36:43.604871 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:36:43.605659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:36:43.605659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:36:43.605659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:36:43.605659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:36:43.605659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:36:43.605659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:43.605659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:36:43.605659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:43.605659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:36:43.605659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:43.605659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:36:43.605659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:43.605659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:36:43.605659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:43.605659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:36:43.605659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:36:43.605659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:43.605659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:36:43.605659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:43.608263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:36:43.608263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:36:43.608263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:43.608263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:36:43.608263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:43.608263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:36:43.703269 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:36:43.703888 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:36:43.704064 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:36:43.705177 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:36:43.705642 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:36:43.706114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:43.707289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:36:43.707289 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:43.707289 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:36:43.707289 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:36:43.707289 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:43.707289 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:36:43.707289 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:43.707289 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:36:43.708869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:43.708869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:36:43.708869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:36:43.708869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:36:43.708869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:36:43.708869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:36:43.708869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:36:43.708869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:43.708869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:36:43.708869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:43.708869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:36:43.708869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:43.710596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:36:43.710596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:43.710596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:36:43.710596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:43.710596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:36:43.710596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:36:43.710596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:43.710596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:36:43.710596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:43.710596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:36:43.710596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:36:43.710596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:43.710596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:36:43.711154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:43.711154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:36:43.711154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:43.711154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:36:43.711154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:36:43.711154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:36:43.711154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:36:43.711154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:36:43.711154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:36:43.711154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:36:43.711154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:36:43.711154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:36:43.711154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:36:43.711154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:36:43.711154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:36:43.712637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:36:43.712637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:36:43.712637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:36:43.712637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:36:43.712637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:36:43.712637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:43.712637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:36:43.712637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:43.712637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:36:43.712637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:43.712637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:36:43.712637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:43.712637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:36:43.712637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:43.712637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:36:43.712637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:36:43.712637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:43.712637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:36:43.712637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:43.713913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:36:43.713913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:36:43.713913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:43.713913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:36:43.713913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:43.713913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:36:43.713913 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:36:43.713913 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:36:43.713913 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:36:43.713913 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:36:43.714364 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:36:43.714364 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:43.714364 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:36:43.714364 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:43.714364 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:36:43.714364 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:36:43.714364 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:43.714364 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:36:43.714364 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:43.714364 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:36:43.714364 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:43.714364 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:36:43.714364 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:36:43.714364 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:36:43.714818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:43.714818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:36:43.714818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:43.714818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:36:43.714818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:43.714818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:36:43.714818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:43.714818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:36:43.714818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:43.714818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:36:43.714818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:36:43.714818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:43.714818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:36:43.714818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:43.714818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:36:43.714818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:36:43.714818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:43.714818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:36:43.715311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:43.715311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:36:43.715311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:43.715311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:36:43.715311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:36:43.715311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:36:43.715311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:36:43.715311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:36:43.715311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:36:43.715311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:36:43.715311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:36:43.715311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:36:43.715651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:43.715651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:36:43.715651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:43.715651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:36:43.715651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:43.715651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:36:43.715651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:43.715651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:36:43.715651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:43.715651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:36:43.715651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:36:43.715651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:43.715651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:36:43.715651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:43.717257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:36:43.717257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:36:43.717257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:43.717257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:36:43.717257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:43.717257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:36:43.717257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:43.717257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:36:43.717257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:36:43.717257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:36:43.717257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:36:43.717257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:36:43.717257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:36:43.717257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:36:43.718298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:36:43.718298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:36:43.718298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:36:43.718298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:36:43.718298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:36:43.718298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:36:43.718298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:36:43.718298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:36:43.718298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:36:43.718298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:36:43.718298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:36:43.718298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:36:43.718298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:36:43.718298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:43.718298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:36:43.718298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:43.718298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:36:43.718298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:43.718298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:36:43.719929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:43.719929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:36:43.719929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:43.719929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:36:43.719929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:36:43.719929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:43.719929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:36:43.719929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:43.719929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:36:43.719929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:36:43.719929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:43.719929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:36:43.719929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:43.719929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:36:43.721230 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:36:43.721469 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:36:43.721721 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:36:43.721950 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:36:43.722416 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:36:43.722717 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:43.723426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:36:43.723426 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:43.723426 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:36:43.723426 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:36:43.723426 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:43.723426 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:36:43.723426 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:43.723426 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:36:43.724772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:43.724772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:36:43.724772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:36:43.724772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:36:43.724772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:43.724772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:36:43.724772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:43.724772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:36:43.724772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:43.724772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:36:43.724772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:43.724772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:36:43.724772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:43.724772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:36:43.724772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:36:43.725168 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:43.725168 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:36:43.725168 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:43.725168 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:36:43.725168 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:36:43.725168 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:43.725168 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:36:43.725168 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:43.725168 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:36:43.726386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:43.726386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:36:43.726386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:36:43.726386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:36:43.726386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:36:43.726386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:36:43.726386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:36:43.726386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:36:43.726386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:36:43.726386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:43.726386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:36:43.726788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:43.726788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:36:43.726788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:43.726788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:36:43.726788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:43.726788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:36:43.726788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:43.726788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:36:43.726788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:36:43.726788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:43.726788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:36:43.726788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:43.726788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:36:43.726788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:36:43.727040 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:43.727040 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:36:43.727040 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:43.727040 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:36:43.728353 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:43.728353 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:36:43.728353 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:36:43.728353 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:36:43.728353 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:36:43.728353 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:36:43.728353 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:36:43.728353 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:36:43.728353 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:36:43.728353 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:36:43.728353 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:36:43.728353 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:36:43.728353 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:36:43.728353 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:36:43.728353 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:36:43.728353 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:36:43.728353 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:36:43.728806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:36:43.728806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:36:43.728806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:36:43.728806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:43.728806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:36:43.728806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:43.728806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:36:43.728806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:43.728806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:36:43.728806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:43.728806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:36:43.728806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:43.728806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:36:43.728806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:36:43.728806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:43.728806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:36:43.728806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:43.728806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:36:43.728806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:36:43.729318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:43.729318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:36:43.729318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:43.729318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:36:43.729318 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:36:43.729318 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:36:43.729318 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:36:43.729527 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:36:43.729842 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:36:43.730032 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:36:43.730239 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:43.730825 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:36:43.730825 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:43.730825 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:36:43.730825 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:36:43.730825 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:43.730825 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:36:43.730825 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:43.730825 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:36:43.731614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:43.731614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:36:43.731614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:36:43.731614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:36:43.731614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:36:43.731614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:36:43.731614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:36:43.731614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:43.731614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:36:43.731614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:43.731614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:36:43.731614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:43.732005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:36:43.732005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:43.732005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:36:43.732005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:43.732005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:36:43.732005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:36:43.732005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:43.732005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:36:43.732005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:43.732005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:36:43.732005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:36:43.732005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:43.732005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:36:43.732343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:43.732343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:36:43.732588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:43.732588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:36:43.732588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:36:43.732588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:36:43.732588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:36:43.732588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:36:43.732588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:36:43.732588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:36:43.732588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:36:43.732588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:36:43.732588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:36:43.732588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:36:43.732588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:36:43.732588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:36:43.732588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:36:43.732588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:36:43.733155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:36:43.733155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:36:43.733155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:43.733155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:36:43.733155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:43.733155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:36:43.733155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:43.733155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:36:43.733155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:43.733155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:36:43.733155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:43.733155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:36:43.733155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:36:43.733155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:43.733155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:36:43.733155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:43.733155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:36:43.733155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:36:43.733155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:43.733680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:36:43.733680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:43.733680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:36:48.564430 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:36:48.565389 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:36:48.565389 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:36:48.566644 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:36:48.602776 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:36:48.602983 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:48.603709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:36:48.603709 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:48.603709 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:36:48.603709 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:36:48.603709 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:48.603709 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:36:48.603709 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:48.603709 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:36:48.604678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:48.604678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:36:48.604678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:36:48.604678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:36:48.604678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:36:48.604678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:36:48.604678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:36:48.604678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:48.604678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:36:48.604678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:48.604678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:36:48.604678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:48.605512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:36:48.605512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:48.605512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:36:48.605512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:48.605512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:36:48.605512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:36:48.605512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:48.605512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:36:48.605512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:48.605512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:36:48.605512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:36:48.605512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:48.605512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:36:48.606061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:48.606061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:36:48.606451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:48.606451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:36:48.606451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:36:48.606451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:36:48.606451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:36:48.606451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:36:48.606451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:36:48.606451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:36:48.606451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:36:48.606451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:36:48.606451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:36:48.606451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:36:48.606451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:36:48.606451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:36:48.606451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:36:48.606451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:36:48.608015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:36:48.608015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:36:48.608015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:48.608015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:36:48.608015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:48.608015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:36:48.608015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:48.608015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:36:48.608015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:48.608015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:36:48.608015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:48.608015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:36:48.608015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:36:48.608015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:48.608015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:36:48.608015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:48.608015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:36:48.608015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:36:48.608015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:48.608899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:36:48.608899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:48.608899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:36:48.703932 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:36:48.704404 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:36:48.704659 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:36:48.705764 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:36:48.706059 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:36:48.706253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:48.707115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:36:48.707115 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:48.707115 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:36:48.707115 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:36:48.707115 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:48.707115 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:36:48.707115 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:48.707115 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:36:48.709056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:48.709056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:36:48.709056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:36:48.709056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:36:48.709056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:36:48.709056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:36:48.709056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:36:48.709056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:48.709056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:36:48.709056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:48.709056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:36:48.709056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:48.709392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:36:48.709392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:48.709392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:36:48.709392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:48.709392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:36:48.709392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:36:48.709392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:48.709392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:36:48.709392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:48.709392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:36:48.709392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:36:48.709392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:48.709392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:36:48.709722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:48.709722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:36:48.710873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:48.710873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:36:48.710873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:36:48.710873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:36:48.710873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:36:48.710873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:36:48.710873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:36:48.710873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:36:48.710873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:36:48.710873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:36:48.710873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:36:48.710873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:36:48.710873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:36:48.710873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:36:48.710873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:36:48.710873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:36:48.711275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:36:48.711275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:36:48.711275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:48.711275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:36:48.711275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:48.711275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:36:48.711275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:48.711275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:36:48.711275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:48.711275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:36:48.711275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:48.711275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:36:48.711275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:36:48.711275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:48.711275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:36:48.711275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:48.711275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:36:48.711275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:36:48.711275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:48.711866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:36:48.711866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:48.711866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:36:48.711866 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:36:48.712261 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:36:48.712261 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:36:48.712816 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:36:48.713251 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:36:48.713452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:48.714034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:36:48.714034 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:48.714034 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:36:48.714034 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:36:48.714034 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:48.714034 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:36:48.714034 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:48.714034 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:36:48.714829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:48.714829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:36:48.714829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:36:48.714829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:36:48.714829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:48.714829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:36:48.714829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:48.714829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:36:48.714829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:48.714829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:36:48.714829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:48.714829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:36:48.714829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:48.714829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:36:48.714829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:36:48.715587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:48.715587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:36:48.715587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:48.715587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:36:48.715587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:36:48.715587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:48.715587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:36:48.715587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:48.715587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:36:48.716041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:48.716041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:36:48.716041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:36:48.716041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:36:48.716041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:36:48.716041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:36:48.716041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:36:48.716041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:36:48.716041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:36:48.716041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:36:48.716041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:48.716041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:36:48.716657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:48.716657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:36:48.716657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:48.716657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:36:48.716657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:48.716657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:36:48.716657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:48.716657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:36:48.716657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:36:48.716657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:48.716657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:36:48.716657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:48.716657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:36:48.716657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:36:48.717006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:48.717006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:36:48.717006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:48.717006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:36:48.720616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:48.720616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:36:48.720616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:36:48.720616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:36:48.720616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:36:48.720616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:36:48.720616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:36:48.720616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:36:48.720616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:36:48.720616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:36:48.720616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:36:48.720616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:36:48.720616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:36:48.720616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:36:48.720616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:36:48.720616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:36:48.721028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:36:48.721028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:36:48.721028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:36:48.721028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:36:48.721028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:36:48.721028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:48.721028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:36:48.721028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:48.721028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:36:48.721028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:48.721028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:36:48.721028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:48.721028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:36:48.721028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:48.721028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:36:48.721028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:36:48.721028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:48.721028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:36:48.721460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:48.721460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:36:48.721460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:36:48.721460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:48.721460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:36:48.721460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:48.721460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:36:48.721627 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:36:48.722009 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:36:48.722213 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:36:48.722517 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:36:48.723000 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:36:48.723369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:48.724828 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:36:48.724828 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:48.724828 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:36:48.724828 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:36:48.724828 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:48.724828 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:36:48.724828 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:48.724828 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:36:48.726646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:48.726646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:36:48.726646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:36:48.726646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:36:48.726646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:48.726646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:36:48.726646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:48.726646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:36:48.726646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:48.726646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:36:48.726646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:48.726646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:36:48.726646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:48.726646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:36:48.726646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:36:48.727030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:48.727030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:36:48.727030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:48.727030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:36:48.727030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:36:48.727030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:48.727030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:36:48.727030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:48.727030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:36:48.728981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:48.728981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:36:48.728981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:36:48.728981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:36:48.728981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:36:48.728981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:36:48.728981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:36:48.728981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:36:48.728981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:36:48.728981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:48.728981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:36:48.729467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:48.729467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:36:48.729467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:48.729467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:36:48.729467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:48.729467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:36:48.729467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:48.729467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:36:48.729467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:36:48.729467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:48.729467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:36:48.729467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:48.729467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:36:48.729467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:36:48.729809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:48.729809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:36:48.729809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:48.729809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:36:48.729809 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:36:48.730912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:48.730912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:36:48.730912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:36:48.730912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:36:48.730912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:36:48.730912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:36:48.730912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:36:48.730912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:36:48.730912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:36:48.730912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:36:48.730912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:36:48.730912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:36:48.730912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:36:48.730912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:36:48.730912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:36:48.730912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:36:48.730912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:36:48.731492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:36:48.731492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:36:48.731492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:36:48.731492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:48.731492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:36:48.731492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:48.731492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:36:48.731492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:48.731492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:36:48.731492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:48.731492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:36:48.731492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:48.731492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:36:48.731492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:36:48.731492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:48.731492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:36:48.731492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:48.731492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:36:48.731492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:36:48.732057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:48.732057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:36:48.732057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:48.732057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:36:48.732057 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:36:48.732057 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:36:48.732057 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:36:48.732240 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:36:48.732371 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:36:48.732596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:48.733079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:36:48.733079 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:48.733079 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:36:48.733079 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:36:48.733079 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:48.733079 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:36:48.733079 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:48.733079 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:36:48.734225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:48.734225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:36:48.734225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:36:48.734225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:36:48.734225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:36:48.734225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:36:48.734225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:36:48.734225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:48.734225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:36:48.734225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:48.734225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:36:48.734225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:48.734533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:36:48.734533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:48.734533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:36:48.734533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:48.734533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:36:48.734533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:36:48.734533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:48.734533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:36:48.734533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:48.734533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:36:48.734533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:36:48.734533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:48.734533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:36:48.734890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:48.734890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:36:48.735238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:48.735238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:36:48.735238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:36:48.735238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:36:48.735238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:36:48.735238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:36:48.735238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:36:48.735238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:36:48.735238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:36:48.735238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:36:48.735238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:36:48.735238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:36:48.735238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:36:48.735238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:36:48.735238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:36:48.735238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:36:48.735627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:36:48.735627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:36:48.735627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:48.735627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:36:48.735627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:48.735627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:36:48.735627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:48.735627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:36:48.735627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:48.735627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:36:48.735627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:48.735627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:36:48.735627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:36:48.735627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:48.735627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:36:48.735627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:48.735627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:36:48.735627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:36:48.735627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:48.736109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:36:48.736109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:48.736109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:36:53.565132 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:36:53.566109 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:36:53.566109 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:36:53.566862 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:36:53.606471 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:36:53.607027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:53.608020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:36:53.608020 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:53.608020 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:36:53.608020 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:36:53.608020 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:53.608020 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:36:53.608020 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:53.608020 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:36:53.609851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:53.609851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:36:53.609851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:36:53.609851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:36:53.609851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:36:53.609851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:36:53.609851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:36:53.609851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:53.609851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:36:53.609851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:53.609851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:36:53.609851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:53.610660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:36:53.610660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:53.610660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:36:53.610660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:53.610660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:36:53.610660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:36:53.610660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:53.610660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:36:53.610660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:53.610660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:36:53.610660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:36:53.610660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:53.610660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:36:53.611400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:53.611400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:36:53.611400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:53.611400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:36:53.611400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:36:53.611400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:36:53.611400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:36:53.611400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:36:53.611400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:36:53.611400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:36:53.611400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:36:53.611400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:36:53.611400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:36:53.611400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:36:53.611400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:36:53.612301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:36:53.612301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:36:53.612301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:36:53.612301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:36:53.612301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:36:53.612301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:53.612301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:36:53.612301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:53.612301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:36:53.612301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:53.612301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:36:53.612301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:53.612301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:36:53.612301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:53.612301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:36:53.612301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:36:53.612301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:53.612301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:36:53.612301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:53.613524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:36:53.613524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:36:53.613524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:53.613524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:36:53.613524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:53.613524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:36:53.704381 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:36:53.704863 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:36:53.704981 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:36:53.705496 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:36:53.705632 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:36:53.705723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:53.706409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:36:53.706409 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:53.706409 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:36:53.706409 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:36:53.706409 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:53.706409 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:36:53.706409 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:53.706409 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:36:53.707872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:53.707872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:36:53.707872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:36:53.707872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:36:53.707872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:36:53.707872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:36:53.707872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:36:53.707872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:53.707872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:36:53.707872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:53.707872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:36:53.707872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:53.708652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:36:53.708652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:53.708652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:36:53.708652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:53.708652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:36:53.708652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:36:53.708652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:53.708652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:36:53.708652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:53.708652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:36:53.708652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:36:53.708652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:53.708652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:36:53.709332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:53.709332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:36:53.709771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:53.709771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:36:53.709771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:36:53.709771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:36:53.709771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:36:53.709771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:36:53.709771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:36:53.709771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:36:53.709771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:36:53.709771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:36:53.709771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:36:53.709771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:36:53.709771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:36:53.709771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:36:53.709771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:36:53.709771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:36:53.710965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:36:53.710965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:36:53.710965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:53.710965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:36:53.710965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:53.710965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:36:53.710965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:53.710965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:36:53.710965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:53.710965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:36:53.710965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:53.710965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:36:53.710965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:36:53.710965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:53.710965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:36:53.710965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:53.710965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:36:53.710965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:36:53.710965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:53.712148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:36:53.712148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:53.712148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:36:53.712148 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:36:53.712511 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:36:53.712698 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:36:53.712698 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:36:53.713231 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:36:53.713588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:53.714772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:36:53.714772 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:53.714772 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:36:53.714772 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:36:53.714772 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:53.714772 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:36:53.714772 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:53.714772 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:36:53.715501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:53.715501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:36:53.715501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:36:53.715501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:36:53.715501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:53.715501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:36:53.715501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:53.715501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:36:53.715501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:53.715501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:36:53.715501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:53.715501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:36:53.715501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:53.715501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:36:53.715501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:36:53.716856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:53.716856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:36:53.716856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:53.716856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:36:53.716856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:36:53.716856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:53.716856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:36:53.716856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:53.716856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:36:53.716856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:53.716856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:36:53.716856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:36:53.716856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:36:53.716856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:36:53.717814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:36:53.717814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:36:53.717814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:36:53.717814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:36:53.717814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:36:53.717814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:53.717814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:36:53.717814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:53.717814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:36:53.717814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:53.717814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:36:53.717814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:53.717814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:36:53.718325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:53.718325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:36:53.718325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:36:53.718325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:53.718325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:36:53.718325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:53.718325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:36:53.718325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:36:53.718325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:53.718325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:36:53.718325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:53.718325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:36:53.718629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:53.718629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:36:53.718629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:36:53.718629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:36:53.718629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:36:53.718629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:36:53.718629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:36:53.718629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:36:53.718629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:36:53.718629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:36:53.718629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:36:53.718629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:36:53.718629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:36:53.718629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:36:53.718629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:36:53.718629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:36:53.719040 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:36:53.719040 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:36:53.719040 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:36:53.719040 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:36:53.719040 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:36:53.719040 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:53.719040 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:36:53.719040 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:53.719040 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:36:53.719040 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:53.719040 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:36:53.719040 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:53.719040 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:36:53.719040 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:53.719040 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:36:53.719040 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:36:53.719040 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:53.719040 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:36:53.719536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:53.719536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:36:53.719536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:36:53.719536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:53.719536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:36:53.719536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:53.719536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:36:53.722096 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:36:53.722419 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:36:53.722533 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:36:53.722657 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:36:53.723110 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:36:53.723481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:53.724222 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:36:53.724222 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:53.724222 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:36:53.724222 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:36:53.724222 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:53.724222 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:36:53.724222 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:53.724222 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:36:53.724952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:53.724952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:36:53.724952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:36:53.724952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:36:53.724952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:53.724952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:36:53.724952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:53.724952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:36:53.724952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:53.724952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:36:53.724952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:53.724952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:36:53.724952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:53.724952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:36:53.724952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:36:53.725420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:53.725420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:36:53.725420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:53.725420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:36:53.725420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:36:53.725420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:53.725420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:36:53.725420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:53.725420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:36:53.725864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:53.725864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:36:53.725864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:36:53.725864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:36:53.725864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:36:53.725864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:36:53.725864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:36:53.725864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:36:53.725864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:36:53.725864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:53.725864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:36:53.726266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:53.726266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:36:53.726266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:53.726266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:36:53.726266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:53.726266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:36:53.726266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:53.726266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:36:53.726266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:36:53.726266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:53.726266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:36:53.726266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:53.726266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:36:53.726266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:36:53.726794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:53.726794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:36:53.726794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:53.726794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:36:53.726933 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:53.726933 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:36:53.726933 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:36:53.726933 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:36:53.726933 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:36:53.726933 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:36:53.726933 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:36:53.726933 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:36:53.726933 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:36:53.726933 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:36:53.726933 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:36:53.726933 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:36:53.726933 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:36:53.726933 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:36:53.726933 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:36:53.726933 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:36:53.726933 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:36:53.727301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:36:53.727301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:36:53.727301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:36:53.727301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:53.727301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:36:53.727301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:53.727301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:36:53.727301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:53.727301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:36:53.727301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:53.727301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:36:53.727301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:53.727301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:36:53.727301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:36:53.727301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:53.727301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:36:53.727301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:53.727301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:36:53.727301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:36:53.727670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:53.727670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:36:53.727670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:53.727670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:36:53.731708 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:36:53.732115 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:36:53.732195 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:36:53.732373 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:36:53.733601 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:36:53.734073 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:36:53.734320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:53.735191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:36:53.735191 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:53.735191 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:36:53.735191 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:36:53.735191 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:53.735191 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:36:53.735191 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:53.735191 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:36:53.736040 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:53.736040 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:36:53.736040 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:36:53.736040 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:36:53.736040 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:36:53.736040 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:36:53.736040 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:36:53.736040 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:53.736040 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:36:53.736040 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:53.736040 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:36:53.736040 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:53.736410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:36:53.736410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:53.736410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:36:53.736410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:53.736410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:36:53.736410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:36:53.736410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:53.736410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:36:53.736410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:53.736410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:36:53.736410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:36:53.736410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:53.736410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:36:53.737037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:53.737037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:36:53.737037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:53.737037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:36:53.737037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:36:53.737037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:36:53.737037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:36:53.737037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:36:53.737037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:36:53.737037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:36:53.737037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:36:53.737037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:36:53.737037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:36:53.737037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:36:53.737037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:36:53.737793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:36:53.737793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:36:53.737793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:36:53.737793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:36:53.737793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:36:53.737793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:53.737793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:36:53.737793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:53.737793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:36:53.737793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:53.737793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:36:53.737793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:53.737793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:36:53.737793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:53.737793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:36:53.737793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:36:53.737793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:53.737793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:36:53.737793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:53.738627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:36:53.738627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:36:53.738627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:53.738627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:36:53.738627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:53.738627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:36:58.565983 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:36:58.567531 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:36:58.567807 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:36:58.569245 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:36:58.571910 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG dbcounter [-] [106314] Writing DB stats neutron:SELECT=240 {{(pid=106314) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 09 09:36:58.606877 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:36:58.607047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:58.607526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:36:58.607526 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:58.607526 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:36:58.607526 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:36:58.607526 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:58.607526 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:36:58.607526 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:58.607526 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:36:58.608211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:58.608211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:36:58.608211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:36:58.608211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:36:58.608211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:36:58.608211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:36:58.608211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:36:58.608211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:58.608211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:36:58.608211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:58.608211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:36:58.608211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:58.609255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:36:58.609255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:58.609255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:36:58.609255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:58.609255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:36:58.609255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:36:58.609255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:58.609255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:36:58.609255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:58.609255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:36:58.609255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:36:58.609255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:58.609255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:36:58.609934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:58.609934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:36:58.609934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:58.609934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:36:58.609934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:36:58.609934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:36:58.609934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:36:58.609934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:36:58.609934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:36:58.609934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:36:58.609934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:36:58.609934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:36:58.609934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:36:58.609934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:36:58.609934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:36:58.610821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:36:58.610821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:36:58.610821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:36:58.610821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:36:58.610821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:36:58.610821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:58.610821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:36:58.610821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:58.610821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:36:58.610821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:58.610821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:36:58.610821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:58.610821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:36:58.610821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:58.610821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:36:58.610821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:36:58.610821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:58.610821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:36:58.610821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:58.611905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:36:58.611905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:36:58.611905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:58.611905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:36:58.611905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:58.611905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:36:58.705064 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:36:58.705602 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:36:58.705841 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:36:58.707070 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:36:58.707352 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:36:58.707591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:58.708591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:36:58.708591 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:58.708591 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:36:58.708591 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:36:58.708591 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:58.708591 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:36:58.708591 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:58.708591 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:36:58.710421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:58.710421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:36:58.710421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:36:58.710421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:36:58.710421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:36:58.710421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:36:58.710421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:36:58.710421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:58.710421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:36:58.710421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:58.710421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:36:58.710421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:58.711082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:36:58.711082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:58.711082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:36:58.711082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:58.711082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:36:58.711082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:36:58.711082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:58.711082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:36:58.711082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:58.711082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:36:58.711082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:36:58.711082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:58.711082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:36:58.711606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:58.711606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:36:58.712145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:58.712145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:36:58.712145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:36:58.712145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:36:58.712145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:36:58.712145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:36:58.712145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:36:58.712145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:36:58.712145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:36:58.712145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:36:58.712145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:36:58.712145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:36:58.712145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:36:58.712145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:36:58.712145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:36:58.712145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:36:58.712838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:36:58.712838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:36:58.712838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:58.712838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:36:58.712838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:58.712838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:36:58.712838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:58.712838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:36:58.712838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:58.712838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:36:58.712838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:58.712838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:36:58.712838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:36:58.712838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:58.712838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:36:58.712838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:58.712838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:36:58.712838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:36:58.712838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:58.713706 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:36:58.713706 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:58.713706 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:36:58.713706 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:36:58.713706 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:36:58.713706 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:36:58.713706 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:36:58.713706 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:36:58.714163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:58.714163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:36:58.714163 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:58.714163 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:36:58.714163 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:36:58.714163 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:58.714163 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:36:58.714163 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:58.714163 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:36:58.714975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:58.714975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:36:58.714975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:36:58.714975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:36:58.714975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:58.714975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:36:58.714975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:58.714975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:36:58.714975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:58.714975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:36:58.714975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:58.714975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:36:58.714975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:58.714975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:36:58.714975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:36:58.715711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:58.715711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:36:58.715711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:58.715711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:36:58.715711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:36:58.715711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:58.715711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:36:58.715711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:58.715711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:36:58.715994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:58.715994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:36:58.715994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:36:58.715994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:36:58.715994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:36:58.715994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:36:58.715994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:36:58.715994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:36:58.715994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:36:58.715994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:36:58.715994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:58.715994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:36:58.716307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:58.716307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:36:58.716307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:58.716307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:36:58.716307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:58.716307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:36:58.716307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:58.716307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:36:58.716307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:36:58.716307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:58.716307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:36:58.716307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:58.716307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:36:58.716307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:36:58.716642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:58.716642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:36:58.716642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:58.716642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:36:58.717139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:58.717139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:36:58.717139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:36:58.717139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:36:58.717139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:36:58.717139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:36:58.717139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:36:58.717139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:36:58.717139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:36:58.717139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:36:58.717139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:36:58.717139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:36:58.717139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:36:58.717139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:36:58.717139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:36:58.717139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:36:58.717545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:36:58.717545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:36:58.717545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:36:58.717545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:36:58.717545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:36:58.717545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:58.717545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:36:58.717545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:58.717545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:36:58.717545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:58.717545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:36:58.717545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:58.717545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:36:58.717545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:58.717545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:36:58.717545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:36:58.717545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:58.717545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:36:58.718027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:58.718027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:36:58.718027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:36:58.718027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:58.718027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:36:58.718027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:58.718027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:36:58.722654 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:36:58.723050 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:36:58.723216 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:36:58.723409 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:36:58.723807 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:36:58.724124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:58.724764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:36:58.724764 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:58.724764 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:36:58.724764 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:36:58.724764 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:58.724764 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:36:58.724764 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:58.724764 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:36:58.725624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:58.725624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:36:58.725624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:36:58.725624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:36:58.725624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:58.725624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:36:58.725624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:58.725624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:36:58.725624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:58.725624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:36:58.725624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:58.725624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:36:58.725624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:58.725624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:36:58.725624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:36:58.726039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:58.726039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:36:58.726039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:58.726039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:36:58.726039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:36:58.726039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:58.726039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:36:58.726039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:58.726039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:36:58.726825 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:58.726825 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:36:58.726825 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:36:58.726825 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:36:58.726825 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:36:58.726825 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:36:58.726825 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:36:58.726825 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:36:58.726825 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:36:58.726825 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:58.726825 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:36:58.727142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:58.727142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:36:58.727142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:58.727142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:36:58.727142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:58.727142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:36:58.727142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:58.727142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:36:58.727142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:36:58.727142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:58.727142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:36:58.727142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:58.727142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:36:58.727142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:36:58.727476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:58.727476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:36:58.727476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:58.727476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:36:58.727999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:58.727999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:36:58.727999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:36:58.727999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:36:58.727999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:36:58.727999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:36:58.727999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:36:58.727999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:36:58.727999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:36:58.727999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:36:58.727999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:36:58.727999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:36:58.727999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:36:58.727999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:36:58.727999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:36:58.727999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:36:58.727999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:36:58.728461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:36:58.728461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:36:58.728461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:36:58.728461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:58.728461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:36:58.728461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:58.728461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:36:58.728461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:58.728461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:36:58.728461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:58.728461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:36:58.728461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:58.728461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:36:58.728461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:36:58.728461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:58.728461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:36:58.728461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:58.728461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:36:58.728461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:36:58.729016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:58.729016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:36:58.729016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:58.729016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:36:58.733529 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:36:58.733529 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:36:58.733529 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:36:58.733716 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:36:58.734160 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:36:58.734459 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:36:58.734713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:58.735369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:36:58.735369 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:58.735369 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:36:58.735369 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:36:58.735369 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:58.735369 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:36:58.735369 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:58.735369 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:36:58.736497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:58.736497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:36:58.736497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:36:58.736497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:36:58.736497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:36:58.736497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:36:58.736497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:36:58.736497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:58.736497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:36:58.736497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:58.736497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:36:58.736497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:58.736876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:36:58.736876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:58.736876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:36:58.736876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:58.736876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:36:58.736876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:36:58.736876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:58.736876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:36:58.736876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:58.736876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:36:58.736876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:36:58.736876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:58.736876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:36:58.737232 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:58.737232 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:36:58.738309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:58.738309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:36:58.738309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:36:58.738309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:36:58.738309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:36:58.738309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:36:58.738309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:36:58.738309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:36:58.738309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:36:58.738309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:36:58.738309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:36:58.738309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:36:58.738309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:36:58.738309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:36:58.738309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:36:58.738309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:36:58.738911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:36:58.738911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:36:58.738911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:58.738911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:36:58.738911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:36:58.738911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:36:58.738911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:36:58.738911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:36:58.738911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:36:58.738911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:36:58.738911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:36:58.738911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:36:58.738911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:36:58.738911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:36:58.738911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:36:58.738911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:36:58.738911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:36:58.738911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:36:58.738911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:36:58.739668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:36:58.739668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:36:58.739668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:37:03.567548 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:37:03.568214 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:37:03.568262 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:37:03.569238 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:37:03.603233 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:37:03.603412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:03.603957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:37:03.603957 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:03.603957 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:37:03.603957 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:37:03.603957 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:03.603957 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:37:03.603957 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:03.603957 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:37:03.604734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:03.604734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:37:03.604734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:37:03.604734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:37:03.604734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:37:03.604734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:37:03.604734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:37:03.604734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:03.604734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:37:03.604734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:03.604734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:37:03.604734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:03.605461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:37:03.605461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:03.605461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:37:03.605461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:03.605461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:37:03.605461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:37:03.605461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:03.605461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:37:03.605461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:03.605461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:37:03.605461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:37:03.605461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:03.605461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:37:03.606142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:03.606142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:37:03.606142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:03.606142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:37:03.606142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:37:03.606142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:37:03.606142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:37:03.606142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:37:03.606142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:37:03.606142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:37:03.606142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:37:03.606142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:37:03.606142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:37:03.606142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:37:03.606142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:37:03.607067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:37:03.607067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:37:03.607067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:37:03.607067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:37:03.607067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:37:03.607067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:03.607067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:37:03.607067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:03.607067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:37:03.607067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:03.607067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:37:03.607067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:03.607067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:37:03.607067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:03.607067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:37:03.607067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:37:03.607067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:03.607067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:37:03.607067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:03.608047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:37:03.608047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:37:03.608047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:03.608047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:37:03.608047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:03.608047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:37:03.705602 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:37:03.706350 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:37:03.706496 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:37:03.707700 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:37:03.708070 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:37:03.708315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:03.709201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:37:03.709201 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:03.709201 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:37:03.709201 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:37:03.709201 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:03.709201 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:37:03.709201 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:03.709201 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:37:03.710892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:03.710892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:37:03.710892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:37:03.710892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:37:03.710892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:37:03.710892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:37:03.710892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:37:03.710892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:03.710892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:37:03.710892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:03.710892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:37:03.710892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:03.711532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:37:03.711532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:03.711532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:37:03.711532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:03.711532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:37:03.711532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:37:03.711532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:03.711532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:37:03.711532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:03.711532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:37:03.711532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:37:03.711532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:03.711532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:37:03.712138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:03.712138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:37:03.712542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:03.712542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:37:03.712542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:37:03.712542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:37:03.712542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:37:03.712542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:37:03.712542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:37:03.712542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:37:03.712542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:37:03.712542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:37:03.712542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:37:03.712542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:37:03.712542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:37:03.712542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:37:03.712542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:37:03.712542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:37:03.713296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:37:03.713296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:37:03.713296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:03.713296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:37:03.713296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:03.713296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:37:03.713296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:03.713296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:37:03.713296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:03.713296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:37:03.713296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:03.713296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:37:03.713296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:37:03.713296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:03.713296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:37:03.713296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:03.713296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:37:03.713296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:37:03.713296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:03.714301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:37:03.714301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:03.714301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:37:03.714301 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:37:03.714301 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:37:03.714301 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:37:03.714301 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:37:03.714301 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:37:03.714945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:03.715870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:37:03.715870 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:03.715870 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:37:03.715870 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:37:03.715870 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:03.715870 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:37:03.715870 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:03.715870 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:37:03.716620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:03.716620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:37:03.716620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:37:03.716620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:37:03.716620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:03.716620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:37:03.716620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:03.716620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:37:03.716620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:03.716620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:37:03.716620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:03.716620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:37:03.716620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:03.716620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:37:03.716620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:37:03.717263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:03.717263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:37:03.717263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:03.717263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:37:03.717263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:37:03.717263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:03.717263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:37:03.717263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:03.717263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:37:03.717622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:03.717622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:37:03.717622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:37:03.717622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:37:03.717622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:37:03.717622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:37:03.717622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:37:03.717622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:37:03.717622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:37:03.717622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:37:03.717622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:03.717622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:37:03.718106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:03.718106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:37:03.718106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:03.718106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:37:03.718106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:03.718106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:37:03.718106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:03.718106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:37:03.718106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:37:03.718106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:03.718106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:37:03.718106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:03.718106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:37:03.718106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:37:03.718617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:03.718617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:37:03.718617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:03.718617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:37:03.718617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:03.718617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:37:03.718617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:37:03.718617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:37:03.718617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:37:03.718617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:37:03.718617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:37:03.718617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:37:03.718617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:37:03.718617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:37:03.719138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:37:03.719138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:37:03.719138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:37:03.719138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:37:03.719138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:37:03.719138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:37:03.719138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:37:03.719138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:37:03.719138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:37:03.719138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:37:03.719138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:37:03.719138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:03.719138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:37:03.719138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:03.719138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:37:03.719138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:03.719138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:37:03.719138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:03.719138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:37:03.719841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:03.719841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:37:03.719841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:37:03.719841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:03.719841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:37:03.719841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:03.719841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:37:03.719841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:37:03.719841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:03.719841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:37:03.719841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:03.719841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:37:03.723253 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:37:03.723581 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:37:03.723690 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:37:03.723822 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:37:03.724522 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:37:03.724934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:03.725507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:37:03.725507 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:03.725507 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:37:03.725507 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:37:03.725507 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:03.725507 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:37:03.725507 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:03.725507 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:37:03.726286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:03.726286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:37:03.726286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:37:03.726286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:37:03.726286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:03.726286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:37:03.726286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:03.726286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:37:03.726286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:03.726286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:37:03.726286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:03.726286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:37:03.726286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:03.726286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:37:03.726286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:37:03.726847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:03.726847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:37:03.726847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:03.726847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:37:03.726847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:37:03.726847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:03.726847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:37:03.726847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:03.726847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:37:03.727154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:03.727154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:37:03.727154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:37:03.727154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:37:03.727154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:37:03.727154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:37:03.727154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:37:03.727154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:37:03.727154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:37:03.727154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:03.727154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:37:03.727561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:03.727561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:37:03.727561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:03.727561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:37:03.727561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:03.727561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:37:03.727561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:03.727561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:37:03.727561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:37:03.727561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:03.727561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:37:03.727561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:03.727561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:37:03.727561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:37:03.728050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:03.728050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:37:03.728050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:03.728050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:37:03.728234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:03.728234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:37:03.728234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:37:03.728234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:37:03.728234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:37:03.728234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:37:03.728234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:37:03.728234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:37:03.728234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:37:03.728234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:37:03.728234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:37:03.728234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:37:03.728234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:37:03.728234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:37:03.728234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:37:03.728234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:37:03.728234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:37:03.728910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:37:03.728910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:37:03.728910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:37:03.728910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:03.728910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:37:03.728910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:03.728910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:37:03.728910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:03.728910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:37:03.728910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:03.728910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:37:03.728910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:03.728910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:37:03.728910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:37:03.728910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:03.728910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:37:03.728910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:03.728910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:37:03.728910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:37:03.729636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:03.729636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:37:03.729636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:03.729636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:37:03.733664 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:37:03.733961 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:37:03.734074 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:37:03.734413 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:37:03.734835 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:37:03.735100 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:37:03.735311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:03.735949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:37:03.735949 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:03.735949 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:37:03.735949 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:37:03.735949 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:03.735949 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:37:03.735949 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:03.735949 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:37:03.736755 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:03.736755 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:37:03.736755 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:37:03.736755 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:37:03.736755 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:37:03.736755 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:37:03.736755 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:37:03.736755 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:03.736755 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:37:03.736755 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:03.736755 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:37:03.736755 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:03.737213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:37:03.737213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:03.737213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:37:03.737213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:03.737213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:37:03.737213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:37:03.737213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:03.737213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:37:03.737213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:03.737213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:37:03.737213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:37:03.737213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:03.737213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:37:03.737568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:03.737568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:37:03.737714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:03.737714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:37:03.737714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:37:03.737714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:37:03.737714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:37:03.737714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:37:03.737714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:37:03.737714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:37:03.737714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:37:03.737714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:37:03.737714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:37:03.737714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:37:03.737714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:37:03.737714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:37:03.737714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:37:03.737714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:37:03.738265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:37:03.738265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:37:03.738265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:03.738265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:37:03.738265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:03.738265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:37:03.738265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:03.738265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:37:03.738265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:03.738265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:37:03.738265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:03.738265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:37:03.738265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:37:03.738265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:03.738265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:37:03.738265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:03.738265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:37:03.738265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:37:03.738265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:03.738867 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:37:03.738867 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:03.738867 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:37:08.568268 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:37:08.569036 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:37:08.569217 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:37:08.570868 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:37:08.607426 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:37:08.607774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:08.608896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:37:08.608896 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:08.608896 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:37:08.608896 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:37:08.608896 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:08.608896 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:37:08.608896 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:08.608896 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:37:08.609987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:08.609987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:37:08.609987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:37:08.609987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:37:08.609987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:37:08.609987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:37:08.609987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:37:08.609987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:08.609987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:37:08.609987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:08.609987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:37:08.609987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:08.610770 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:37:08.610770 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:08.610770 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:37:08.610770 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:08.610770 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:37:08.610770 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:37:08.610770 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:08.610770 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:37:08.610770 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:08.610770 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:37:08.610770 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:37:08.610770 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:08.610770 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:37:08.611497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:08.611497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:37:08.611497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:08.611497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:37:08.611497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:37:08.611497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:37:08.611497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:37:08.611497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:37:08.611497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:37:08.611497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:37:08.611497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:37:08.611497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:37:08.611497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:37:08.611497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:37:08.611497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:37:08.612444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:37:08.612444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:37:08.612444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:37:08.612444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:37:08.612444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:37:08.612444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:08.612444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:37:08.612444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:08.612444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:37:08.612444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:08.612444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:37:08.612444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:08.612444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:37:08.612444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:08.612444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:37:08.612444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:37:08.612444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:08.612444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:37:08.612444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:08.613470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:37:08.613470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:37:08.613470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:08.613470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:37:08.613470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:08.613470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:37:08.706363 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:37:08.707023 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:37:08.707237 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:37:08.708546 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:37:08.708918 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:37:08.709391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:08.710111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:37:08.710111 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:08.710111 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:37:08.710111 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:37:08.710111 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:08.710111 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:37:08.710111 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:08.710111 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:37:08.712645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:08.712645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:37:08.712645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:37:08.712645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:37:08.712645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:37:08.712645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:37:08.712645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:37:08.712645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:08.712645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:37:08.712645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:08.712645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:37:08.712645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:08.713600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:37:08.713600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:08.713600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:37:08.713600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:08.713600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:37:08.713600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:37:08.713600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:08.713600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:37:08.713600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:08.713600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:37:08.713600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:37:08.713600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:08.713600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:37:08.714479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:08.714479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:37:08.715109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:08.715109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:37:08.715109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:37:08.715109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:37:08.715109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:37:08.715109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:37:08.715109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:37:08.715109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:37:08.715109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:37:08.715109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:37:08.715109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:37:08.715109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:37:08.715109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:37:08.715109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:37:08.715109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:37:08.715109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:37:08.716185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:37:08.716185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:37:08.716185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:08.716185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:37:08.716185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:08.716185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:37:08.716185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:08.716185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:37:08.716185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:08.716185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:37:08.716185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:08.716185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:37:08.716185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:37:08.716185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:08.716185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:37:08.716185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:08.716185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:37:08.716185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:37:08.716185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:08.716960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:37:08.716960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:08.716960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:37:08.716960 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:37:08.716960 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:37:08.716960 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:37:08.716960 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:37:08.716960 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:37:08.717323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:08.720520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:37:08.720520 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:08.720520 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:37:08.720520 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:37:08.720520 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:08.720520 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:37:08.720520 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:08.720520 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:37:08.720520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:08.720520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:37:08.720520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:37:08.720520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:37:08.720520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:08.720520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:37:08.720520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:08.720520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:37:08.720520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:08.720520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:37:08.721323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:08.721323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:37:08.721323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:08.721323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:37:08.721323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:37:08.721323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:08.721323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:37:08.721323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:08.721323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:37:08.721323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:37:08.721323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:08.721323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:37:08.721323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:08.721323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:37:08.721323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:08.722014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:37:08.722014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:37:08.722014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:37:08.722014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:37:08.722014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:37:08.722014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:37:08.722014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:37:08.722014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:37:08.722014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:37:08.722014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:08.722014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:37:08.722014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:08.722014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:37:08.722495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:08.722495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:37:08.722495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:08.722495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:37:08.722495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:08.722495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:37:08.722495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:37:08.722495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:08.722495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:37:08.722495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:08.722495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:37:08.722495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:37:08.722495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:08.722495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:37:08.723026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:08.723026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:37:08.723026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:08.723026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:37:08.723026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:37:08.723026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:37:08.723026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:37:08.723026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:37:08.723026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:37:08.723026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:37:08.723026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:37:08.723026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:37:08.723026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:37:08.723026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:37:08.723026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:37:08.723594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:37:08.723594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:37:08.723594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:37:08.723594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:37:08.723594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:37:08.723594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:37:08.723594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:37:08.723594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:37:08.723594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:08.723594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:37:08.723594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:08.723594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:37:08.723594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:08.723594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:37:08.723594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:08.723594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:37:08.723594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:08.723594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:37:08.723594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:37:08.724389 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:08.724389 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:37:08.724389 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:08.724389 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:37:08.724389 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:37:08.724389 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:08.724389 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:37:08.724389 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:08.724389 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:37:08.724389 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:37:08.724389 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:37:08.724389 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:37:08.724876 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:37:08.724876 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:37:08.725270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:08.725877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:37:08.725877 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:08.725877 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:37:08.725877 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:37:08.725877 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:08.725877 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:37:08.725877 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:08.725877 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:37:08.726456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:08.726456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:37:08.726456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:37:08.726456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:37:08.726456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:08.726456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:37:08.726456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:08.726456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:37:08.726456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:08.726456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:37:08.726456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:08.726456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:37:08.726456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:08.726456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:37:08.726456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:37:08.726871 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:08.726871 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:37:08.726871 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:08.726871 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:37:08.726871 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:37:08.726871 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:08.726871 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:37:08.726871 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:08.726871 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:37:08.727492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:08.727492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:37:08.727492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:37:08.727492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:37:08.727492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:37:08.727492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:37:08.727492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:37:08.727492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:37:08.727492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:37:08.727492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:08.727492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:37:08.727836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:08.727836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:37:08.727836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:08.727836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:37:08.727836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:08.727836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:37:08.727836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:08.727836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:37:08.727836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:37:08.727836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:08.727836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:37:08.727836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:08.727836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:37:08.727836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:37:08.728193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:08.728193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:37:08.728193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:08.728193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:37:08.728515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:08.728515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:37:08.728515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:37:08.728515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:37:08.728515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:37:08.728515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:37:08.728515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:37:08.728515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:37:08.728515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:37:08.728515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:37:08.728515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:37:08.728515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:37:08.728515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:37:08.728515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:37:08.728515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:37:08.728515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:37:08.728515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:37:08.728964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:37:08.728964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:37:08.728964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:37:08.728964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:08.728964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:37:08.728964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:08.728964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:37:08.728964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:08.728964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:37:08.728964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:08.728964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:37:08.728964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:08.728964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:37:08.728964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:37:08.728964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:08.728964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:37:08.728964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:08.728964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:37:08.728964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:37:08.729397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:08.729397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:37:08.729397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:08.729397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:37:08.734135 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:37:08.734482 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:37:08.734593 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:37:08.734884 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:37:08.735535 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:37:08.736009 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:37:08.736263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:08.737309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:37:08.737309 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:08.737309 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:37:08.737309 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:37:08.737309 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:08.737309 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:37:08.737309 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:08.737309 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:37:08.739442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:08.739442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:37:08.739442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:37:08.739442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:37:08.739442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:37:08.739442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:37:08.739442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:37:08.739442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:08.739442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:37:08.739442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:08.739442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:37:08.739442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:08.739855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:37:08.739855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:08.739855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:37:08.739855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:08.739855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:37:08.739855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:37:08.739855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:08.739855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:37:08.739855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:08.739855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:37:08.739855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:37:08.739855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:08.739855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:37:08.740186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:08.740186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:37:08.741401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:08.741401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:37:08.741401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:37:08.741401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:37:08.741401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:37:08.741401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:37:08.741401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:37:08.741401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:37:08.741401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:37:08.741401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:37:08.741401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:37:08.741401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:37:08.741401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:37:08.741401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:37:08.741401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:37:08.741401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:37:08.741873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:37:08.741873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:37:08.741873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:08.741873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:37:08.741873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:08.741873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:37:08.741873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:08.741873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:37:08.741873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:08.741873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:37:08.741873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:08.741873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:37:08.741873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:37:08.741873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:08.741873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:37:08.741873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:08.741873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:37:08.741873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:37:08.741873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:08.742349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:37:08.742349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:08.742349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:37:13.574548 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:37:13.575372 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:37:13.575372 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:37:13.576201 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:37:13.614286 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:37:13.614853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:13.615823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:37:13.615823 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:13.615823 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:37:13.615823 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:37:13.615823 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:13.615823 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:37:13.615823 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:13.615823 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:37:13.616623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:13.616623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:37:13.616623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:37:13.616623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:37:13.616623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:37:13.616623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:37:13.616623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:37:13.616623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:13.616623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:37:13.616623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:13.616623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:37:13.616623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:13.617268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:37:13.617268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:13.617268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:37:13.617268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:13.617268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:37:13.617268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:37:13.617268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:13.617268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:37:13.617268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:13.617268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:37:13.617268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:37:13.617268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:13.617268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:37:13.617967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:13.617967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:37:13.617967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:13.617967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:37:13.617967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:37:13.617967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:37:13.617967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:37:13.617967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:37:13.617967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:37:13.617967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:37:13.617967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:37:13.617967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:37:13.617967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:37:13.617967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:37:13.617967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:37:13.618773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:37:13.618773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:37:13.618773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:37:13.618773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:37:13.618773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:37:13.618773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:13.618773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:37:13.618773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:13.618773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:37:13.618773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:13.618773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:37:13.618773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:13.618773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:37:13.618773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:13.618773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:37:13.618773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:37:13.618773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:13.618773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:37:13.618773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:13.619659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:37:13.619659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:37:13.619659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:13.619659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:37:13.619659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:13.619659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:37:13.707100 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:37:13.707704 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:37:13.707939 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:37:13.709165 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:37:13.709610 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:37:13.709903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:13.710970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:37:13.710970 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:13.710970 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:37:13.710970 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:37:13.710970 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:13.710970 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:37:13.710970 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:13.710970 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:37:13.712475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:13.712475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:37:13.712475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:37:13.712475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:37:13.712475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:37:13.712475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:37:13.712475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:37:13.712475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:13.712475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:37:13.712475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:13.712475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:37:13.712475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:13.713371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:37:13.713371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:13.713371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:37:13.713371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:13.713371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:37:13.713371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:37:13.713371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:13.713371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:37:13.713371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:13.713371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:37:13.713371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:37:13.713371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:13.713371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:37:13.714231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:13.714231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:37:13.714499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:13.714499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:37:13.714499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:37:13.714499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:37:13.714499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:37:13.714499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:37:13.714499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:37:13.714499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:37:13.714499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:37:13.714499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:37:13.714499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:37:13.714499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:37:13.714499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:37:13.714499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:37:13.714499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:37:13.714499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:37:13.715431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:37:13.715431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:37:13.715431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:13.715431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:37:13.715431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:13.715431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:37:13.715431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:13.715431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:37:13.715431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:13.715431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:37:13.715431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:13.715431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:37:13.715431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:37:13.715431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:13.715431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:37:13.715431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:13.715431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:37:13.715431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:37:13.715431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:13.716104 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:37:13.716104 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:13.716104 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:37:13.716104 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:37:13.716284 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:37:13.716442 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:37:13.716560 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:37:13.717139 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:37:13.717469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:13.718323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:37:13.718323 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:13.718323 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:37:13.718323 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:37:13.718323 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:13.718323 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:37:13.718323 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:13.718323 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:37:13.719163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:13.719163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:37:13.719163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:37:13.719163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:37:13.719163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:13.719163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:37:13.719163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:13.719163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:37:13.719163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:13.719163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:37:13.719163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:13.719163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:37:13.719163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:13.719163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:37:13.719163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:37:13.719894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:13.719894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:37:13.719894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:13.719894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:37:13.719894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:37:13.719894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:13.719894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:37:13.719894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:13.719894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:37:13.720294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:13.720294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:37:13.720294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:37:13.720294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:37:13.720294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:37:13.720294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:37:13.720294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:37:13.720294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:37:13.720294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:37:13.720294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:37:13.720294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:13.720294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:37:13.720791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:13.720791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:37:13.720791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:13.720791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:37:13.720791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:13.720791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:37:13.720791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:13.720791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:37:13.720791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:37:13.720791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:13.720791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:37:13.720791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:13.720791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:37:13.720791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:37:13.721342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:13.721342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:37:13.721342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:13.721342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:37:13.721342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:13.721342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:37:13.721342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:37:13.721342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:37:13.721342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:37:13.721342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:37:13.721342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:37:13.721342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:37:13.721342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:37:13.721342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:37:13.721901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:37:13.721901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:37:13.721901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:37:13.721901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:37:13.721901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:37:13.721901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:37:13.721901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:37:13.721901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:37:13.721901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:37:13.721901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:37:13.721901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:37:13.721901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:13.721901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:37:13.721901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:13.721901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:37:13.721901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:13.721901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:37:13.721901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:13.721901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:37:13.722878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:13.722878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:37:13.722878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:37:13.722878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:13.722878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:37:13.722878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:13.722878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:37:13.722878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:37:13.722878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:13.722878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:37:13.722878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:13.722878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:37:13.724030 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:37:13.724450 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:37:13.724641 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:37:13.724896 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:37:13.725430 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:37:13.725938 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:13.727316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:37:13.727316 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:13.727316 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:37:13.727316 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:37:13.727316 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:13.727316 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:37:13.727316 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:13.727316 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:37:13.728522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:13.728522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:37:13.728522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:37:13.728522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:37:13.728522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:13.728522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:37:13.728522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:13.728522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:37:13.728522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:13.728522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:37:13.728522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:13.728522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:37:13.728522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:13.728522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:37:13.728522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:37:13.729230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:13.729230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:37:13.729230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:13.729230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:37:13.729230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:37:13.729230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:13.729230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:37:13.729230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:13.729230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:37:13.730151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:13.730151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:37:13.730151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:37:13.730151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:37:13.730151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:37:13.730151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:37:13.730151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:37:13.730151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:37:13.730151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:37:13.730151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:13.730151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:37:13.730778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:13.730778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:37:13.730778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:13.730778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:37:13.730778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:13.730778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:37:13.730778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:13.730778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:37:13.730778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:37:13.730778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:13.730778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:37:13.730778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:13.730778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:37:13.730778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:37:13.731335 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:13.731335 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:37:13.731335 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:13.731335 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:37:13.731806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:13.731806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:37:13.731806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:37:13.731806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:37:13.731806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:37:13.731806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:37:13.731806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:37:13.731806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:37:13.731806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:37:13.731806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:37:13.731806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:37:13.731806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:37:13.731806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:37:13.731806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:37:13.731806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:37:13.731806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:37:13.731806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:37:13.732290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:37:13.732290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:37:13.732290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:37:13.732290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:13.732290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:37:13.732290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:13.732290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:37:13.732290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:13.732290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:37:13.732290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:13.732290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:37:13.732290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:13.732290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:37:13.732290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:37:13.732290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:13.732290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:37:13.732290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:13.732290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:37:13.732290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:37:13.732833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:13.732833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:37:13.732833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:13.732833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:37:13.734587 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:37:13.734884 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:37:13.735010 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:37:13.735175 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:37:13.735570 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:37:13.735905 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:37:13.736114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:13.736699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:37:13.736699 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:13.736699 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:37:13.736699 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:37:13.736699 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:13.736699 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:37:13.736699 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:13.736699 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:37:13.737588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:13.737588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:37:13.737588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:37:13.737588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:37:13.737588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:37:13.737588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:37:13.737588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:37:13.737588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:13.737588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:37:13.737588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:13.737588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:37:13.737588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:13.737993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:37:13.737993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:13.737993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:37:13.737993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:13.737993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:37:13.737993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:37:13.737993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:13.737993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:37:13.737993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:13.737993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:37:13.737993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:37:13.737993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:13.737993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:37:13.738392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:13.738392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:37:13.738769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:13.738769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:37:13.738769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:37:13.738769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:37:13.738769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:37:13.738769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:37:13.738769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:37:13.738769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:37:13.738769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:37:13.738769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:37:13.738769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:37:13.738769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:37:13.738769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:37:13.738769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:37:13.738769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:37:13.738769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:37:13.739354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:37:13.739354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:37:13.739354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:13.739354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:37:13.739354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:13.739354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:37:13.739354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:13.739354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:37:13.739354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:13.739354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:37:13.739354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:13.739354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:37:13.739354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:37:13.739354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:13.739354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:37:13.739354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:13.739354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:37:13.739354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:37:13.739354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:13.739834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:37:13.739834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:13.739834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:37:18.575429 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:37:18.576426 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:37:18.576426 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:37:18.577066 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:37:18.621926 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:37:18.622349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:18.623583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:37:18.623583 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:18.623583 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:37:18.623583 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:37:18.623583 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:18.623583 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:37:18.623583 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:18.623583 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:37:18.624707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.048 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:18.624707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:37:18.624707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:37:18.624707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:37:18.624707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:37:18.624707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:37:18.624707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:37:18.624707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:18.624707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:37:18.624707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:18.624707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:37:18.624707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:18.625227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:37:18.625227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:18.625227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:37:18.625227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:18.625227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:37:18.625227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:37:18.625227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:18.625227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:37:18.625227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:18.625227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:37:18.625227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:37:18.625227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:18.625227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:37:18.625613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:18.625613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:37:18.625917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:18.625917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:37:18.625917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:37:18.625917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:37:18.625917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:37:18.625917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:37:18.625917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:37:18.625917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:37:18.625917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:37:18.625917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:37:18.625917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:37:18.625917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:37:18.625917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:37:18.625917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:37:18.625917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:37:18.625917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:37:18.626426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:37:18.626426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:37:18.626426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:18.626426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:37:18.626426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:18.626426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:37:18.626426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:18.626426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:37:18.626426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:18.626426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:37:18.626426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:18.626426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:37:18.626426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:37:18.626426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:18.626426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:37:18.626426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:18.626426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:37:18.626426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:37:18.626426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:18.627125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:37:18.627125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:18.627125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:37:18.707917 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:37:18.708831 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:37:18.708831 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:37:18.709298 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:37:18.709639 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:37:18.710009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:18.710965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:37:18.710965 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:18.710965 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:37:18.710965 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:37:18.710965 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:18.710965 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:37:18.710965 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:18.710965 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:37:18.712066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:18.712066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:37:18.712066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:37:18.712066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:37:18.712066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:37:18.712066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:37:18.712066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:37:18.712066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:18.712066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:37:18.712066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:18.712066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:37:18.712066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:18.712711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:37:18.712711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:18.712711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:37:18.712711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:18.712711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:37:18.712711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:37:18.712711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:18.712711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:37:18.712711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:18.712711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:37:18.712711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:37:18.712711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:18.712711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:37:18.713436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:18.713436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:37:18.713436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:18.713436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:37:18.713436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:37:18.713436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:37:18.713436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:37:18.713436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:37:18.713436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:37:18.713436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:37:18.713436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:37:18.713436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:37:18.713436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:37:18.713436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:37:18.713436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:37:18.713974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:37:18.713974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:37:18.713974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:37:18.713974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:37:18.713974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:37:18.713974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:18.713974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:37:18.713974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:18.713974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:37:18.713974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:18.713974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:37:18.713974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:18.713974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:37:18.713974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:18.713974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:37:18.713974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:37:18.713974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:18.713974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:37:18.713974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:18.714502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:37:18.714502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:37:18.714502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:18.714502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:37:18.714502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:18.714502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:37:18.716388 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:37:18.716610 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:37:18.716726 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:37:18.716905 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:37:18.717448 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:37:18.717736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:18.718601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:37:18.718601 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:18.718601 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:37:18.718601 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:37:18.718601 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:18.718601 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:37:18.718601 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:18.718601 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:37:18.719399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:18.719399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:37:18.719399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:37:18.719399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:37:18.719399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:18.719399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:37:18.719399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:18.719399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:37:18.719399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:18.719399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:37:18.719399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:18.719399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:37:18.719399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:18.719399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:37:18.719399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:37:18.719864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:18.719864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:37:18.719864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:18.719864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:37:18.719864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:37:18.719864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:18.719864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:37:18.719864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:18.719864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:37:18.720260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:18.720260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:37:18.720260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:37:18.720260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:37:18.720260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:37:18.720260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:37:18.720260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:37:18.720260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:37:18.720260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:37:18.720260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:37:18.720260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:18.720260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:37:18.720606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:18.720606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:37:18.720606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:18.720606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:37:18.720606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:18.720606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:37:18.720606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:18.720606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:37:18.720606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:37:18.720606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:18.720606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:37:18.720606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:18.720606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:37:18.720606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:37:18.720976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:18.720976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:37:18.720976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:18.720976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:37:18.721354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:18.721354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:37:18.721354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:37:18.721354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:37:18.721354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:37:18.721354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:37:18.721354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:37:18.721354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:37:18.721354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:37:18.721354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:37:18.721354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:37:18.721354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:37:18.721354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:37:18.721354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:37:18.721354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:37:18.721354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:37:18.721842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:37:18.721842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:37:18.721842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:37:18.721842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:37:18.721842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:37:18.721842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:18.721842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:37:18.721842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:18.721842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:37:18.721842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:18.721842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:37:18.721842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:18.721842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:37:18.721842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:18.721842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:37:18.721842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:37:18.721842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:18.721842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:37:18.722349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:18.722349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:37:18.722349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:37:18.722349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:18.722349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:37:18.722349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:18.722349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:37:18.724594 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:37:18.724946 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:37:18.725064 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:37:18.725238 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:37:18.725628 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:37:18.725959 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:18.726551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:37:18.726551 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:18.726551 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:37:18.726551 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:37:18.726551 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:18.726551 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:37:18.726551 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:18.726551 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:37:18.727339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:18.727339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:37:18.727339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:37:18.727339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:37:18.727339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:18.727339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:37:18.727339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:18.727339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:37:18.727339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:18.727339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:37:18.727339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:18.727339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:37:18.727339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:18.727339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:37:18.727339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:37:18.727784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:18.727784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:37:18.727784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:18.727784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:37:18.727784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:37:18.727784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:18.727784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:37:18.727784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:18.727784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:37:18.728276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:18.728276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:37:18.728276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:37:18.728276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:37:18.728276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:37:18.728276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:37:18.728276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:37:18.728276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:37:18.728276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:37:18.728276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:18.728276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:37:18.728611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:18.728611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:37:18.728611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:18.728611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:37:18.728611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:18.728611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:37:18.728611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:18.728611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:37:18.728611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:37:18.728611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:18.728611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:37:18.728611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:18.728611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:37:18.728611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:37:18.729051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:18.729051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:37:18.729051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:18.729051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:37:18.729441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:18.729441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:37:18.729441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:37:18.729441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:37:18.729441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:37:18.729441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:37:18.729441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:37:18.729441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:37:18.729441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:37:18.729441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:37:18.729441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:37:18.729441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:37:18.729441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:37:18.729441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:37:18.729441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:37:18.729441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:37:18.729441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:37:18.729969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:37:18.729969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:37:18.729969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:37:18.729969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:18.729969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:37:18.729969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:18.729969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:37:18.729969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:18.729969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:37:18.729969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:18.729969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:37:18.729969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:18.729969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:37:18.729969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:37:18.729969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:18.729969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:37:18.729969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:18.729969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:37:18.729969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:37:18.730486 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:18.730486 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:37:18.730486 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:18.730486 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:37:18.735061 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:37:18.735426 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:37:18.735547 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:37:18.735720 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:37:18.736236 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:37:18.736630 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:37:18.736912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:18.737658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:37:18.737658 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:18.737658 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:37:18.737658 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:37:18.737658 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:18.737658 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:37:18.737658 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:18.737658 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:37:18.738465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:18.738465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:37:18.738465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:37:18.738465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:37:18.738465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:37:18.738465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:37:18.738465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:37:18.738465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:18.738465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:37:18.738465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:18.738465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:37:18.738465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:18.738832 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:37:18.738832 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:18.738832 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:37:18.738832 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:18.738832 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:37:18.738832 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:37:18.738832 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:18.738832 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:37:18.738832 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:18.738832 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:37:18.738832 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:37:18.738832 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:18.738832 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:37:18.739154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:18.739154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:37:18.739392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:18.739392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:37:18.739392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:37:18.739392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:37:18.739392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:37:18.739392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:37:18.739392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:37:18.739392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:37:18.739392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:37:18.739392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:37:18.739392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:37:18.739392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:37:18.739392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:37:18.739392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:37:18.739392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:37:18.739392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:37:18.739834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:37:18.739834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:37:18.739834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:18.739834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:37:18.739834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:18.739834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:37:18.739834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:18.739834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:37:18.739834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:18.739834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:37:18.739834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:18.739834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:37:18.739834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:37:18.739834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:18.739834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:37:18.739834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:18.739834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:37:18.739834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:37:18.739834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:18.740316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:37:18.740316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:18.740316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:37:23.576533 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:37:23.577649 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:37:23.577649 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:37:23.578595 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:37:23.611559 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:37:23.612018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:23.612897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:37:23.612897 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:23.612897 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:37:23.612897 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:37:23.612897 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:23.612897 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:37:23.612897 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:23.612897 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:37:23.613643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:23.613643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:37:23.613643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:37:23.613643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:37:23.613643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:37:23.613643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:37:23.613643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:37:23.613643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:23.613643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:37:23.613643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:23.613643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:37:23.613643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:23.614647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:37:23.614647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:23.614647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:37:23.614647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:23.614647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:37:23.614647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:37:23.614647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:23.614647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:37:23.614647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:23.614647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:37:23.614647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:37:23.614647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:23.614647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:37:23.615432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:23.615432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:37:23.615432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:23.615432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:37:23.615432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:37:23.615432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:37:23.615432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:37:23.615432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:37:23.615432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:37:23.615432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:37:23.615432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:37:23.615432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:37:23.615432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:37:23.615432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:37:23.615432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:37:23.616202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:37:23.616202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:37:23.616202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:37:23.616202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:37:23.616202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:37:23.616202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:23.616202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:37:23.616202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:23.616202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:37:23.616202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:23.616202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:37:23.616202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:23.616202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:37:23.616202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:23.616202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:37:23.616202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:37:23.616202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:23.616202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:37:23.616202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:23.618446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:37:23.618446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:37:23.618446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:23.618446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:37:23.618446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:23.618446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:37:23.708599 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:37:23.709282 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:37:23.709547 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:37:23.710686 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:37:23.711076 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:37:23.711390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:23.712457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:37:23.712457 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:23.712457 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:37:23.712457 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:37:23.712457 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:23.712457 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:37:23.712457 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:23.712457 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:37:23.713737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:23.713737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:37:23.713737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:37:23.713737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:37:23.713737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:37:23.713737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:37:23.713737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:37:23.713737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:23.713737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:37:23.713737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:23.713737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:37:23.713737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:23.714324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:37:23.714324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:23.714324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:37:23.714324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:23.714324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:37:23.714324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:37:23.714324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:23.714324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:37:23.714324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:23.714324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:37:23.714324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:37:23.714324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:23.714324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:37:23.714830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:23.714830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:37:23.715300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:23.715300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:37:23.715300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:37:23.715300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:37:23.715300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:37:23.715300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:37:23.715300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:37:23.715300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:37:23.715300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:37:23.715300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:37:23.715300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:37:23.715300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:37:23.715300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:37:23.715300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:37:23.715300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:37:23.715300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:37:23.716425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:37:23.716425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:37:23.716425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:23.716425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:37:23.716425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:23.716425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:37:23.716425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:23.716425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:37:23.716425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:23.716425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:37:23.716425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:23.716425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:37:23.716425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:37:23.716425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:23.716425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:37:23.716425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:23.716425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:37:23.716425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:37:23.716425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:23.717410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:37:23.717410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:23.717410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:37:23.717410 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:37:23.717410 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:37:23.717410 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:37:23.717410 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:37:23.717882 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:37:23.718009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:23.718629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:37:23.718629 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:23.718629 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:37:23.718629 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:37:23.718629 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:23.718629 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:37:23.718629 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:23.718629 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:37:23.719680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:23.719680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:37:23.719680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:37:23.719680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:37:23.719680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:23.719680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:37:23.719680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:23.719680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:37:23.719680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:23.719680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:37:23.719680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:23.719680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:37:23.719680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:23.719680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:37:23.719680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:37:23.720955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:23.720955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:37:23.720955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:23.720955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:37:23.720955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:37:23.720955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:23.720955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:37:23.720955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:23.720955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:37:23.721340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:23.721340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:37:23.721340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:37:23.721340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:37:23.721340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:37:23.721340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:37:23.721340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:37:23.721340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:37:23.721340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:37:23.721340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:37:23.721340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:23.721340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:37:23.721836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:23.721836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:37:23.721836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:23.721836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:37:23.721836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:23.721836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:37:23.721836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:23.721836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:37:23.721836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:37:23.721836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:23.721836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:37:23.721836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:23.721836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:37:23.721836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:37:23.722391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:23.722391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:37:23.722391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:23.722391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:37:23.722574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:23.722574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:37:23.722574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:37:23.722574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:37:23.722574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:37:23.722574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:37:23.722574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:37:23.722574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:37:23.722574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:37:23.722574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:37:23.722574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:37:23.722574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:37:23.722574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:37:23.722574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:37:23.722574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:37:23.722574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:37:23.723079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:37:23.723079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:37:23.723079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:37:23.723079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:37:23.723079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:37:23.723079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:23.723079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:37:23.723079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:23.723079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:37:23.723079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:23.723079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:37:23.723079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:23.723079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:37:23.723079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:23.723079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:37:23.723079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:37:23.723079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:23.723079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:37:23.723606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:23.723606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:37:23.723606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:37:23.723606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:23.723606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:37:23.723606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:23.723606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:37:23.725086 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:37:23.725369 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:37:23.725468 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:37:23.725605 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:37:23.726136 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:37:23.726527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:23.727298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:37:23.727298 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:23.727298 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:37:23.727298 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:37:23.727298 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:23.727298 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:37:23.727298 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:23.727298 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:37:23.728234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:23.728234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:37:23.728234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:37:23.728234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:37:23.728234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:23.728234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:37:23.728234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:23.728234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:37:23.728234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:23.728234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:37:23.728234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:23.728234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:37:23.728234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:23.728234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:37:23.728234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:37:23.729274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:23.729274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:37:23.729274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:23.729274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:37:23.729274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:37:23.729274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:23.729274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:37:23.729274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:23.729274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:37:23.729849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:23.729849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:37:23.729849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:37:23.729849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:37:23.729849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:37:23.729849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:37:23.729849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:37:23.729849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:37:23.729849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:37:23.729849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:23.729849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:37:23.730841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:23.730841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:37:23.730841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:23.730841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:37:23.730841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:23.730841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:37:23.730841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:23.730841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:37:23.730841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:37:23.730841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:23.730841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:37:23.730841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:23.730841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:37:23.730841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:37:23.731445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:23.731445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:37:23.731445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:23.731445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:37:23.731445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:23.731445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:37:23.731445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:37:23.731445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:37:23.731445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:37:23.731445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:37:23.731445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:37:23.731445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:37:23.731445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:37:23.731445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:37:23.732043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:37:23.732043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:37:23.732043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:37:23.732043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:37:23.732043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:37:23.732043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:37:23.732043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:37:23.732043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:37:23.732043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:37:23.732043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:37:23.732043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:23.732043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:37:23.732043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:23.732043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:37:23.732043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:23.732043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:37:23.732043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:23.732043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:37:23.732043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:23.733047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:37:23.733047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:37:23.733047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:23.733047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:37:23.733047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:23.733047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:37:23.733047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:37:23.733047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:23.733047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:37:23.733047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:23.733047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:37:23.735557 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:37:23.735940 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:37:23.736086 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:37:23.736353 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:37:23.736921 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:37:23.737301 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:37:23.737537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:23.738479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:37:23.738479 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:23.738479 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:37:23.738479 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:37:23.738479 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:23.738479 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:37:23.738479 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:23.738479 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:37:23.740270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:23.740270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:37:23.740270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:37:23.740270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:37:23.740270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:37:23.740270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:37:23.740270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:37:23.740270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:23.740270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:37:23.740270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:23.740270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:37:23.740270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:23.740720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:37:23.740720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:23.740720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:37:23.740720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:23.740720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:37:23.740720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:37:23.740720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:23.740720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:37:23.740720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:23.740720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:37:23.740720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:37:23.740720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:23.740720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:37:23.741065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:23.741065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:37:23.742187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:23.742187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:37:23.742187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:37:23.742187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:37:23.742187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:37:23.742187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:37:23.742187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:37:23.742187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:37:23.742187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:37:23.742187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:37:23.742187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:37:23.742187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:37:23.742187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:37:23.742187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:37:23.742187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:37:23.742187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:37:23.743130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:37:23.743130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:37:23.743130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:23.743130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:37:23.743130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:23.743130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:37:23.743130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:23.743130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:37:23.743130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:23.743130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:37:23.743130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:23.743130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:37:23.743130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:37:23.743130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:23.743130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:37:23.743130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:23.743130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:37:23.743130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:37:23.743130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:23.743805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:37:23.743805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:23.743805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:37:28.577612 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:37:28.578281 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:37:28.578281 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:37:28.578674 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:37:28.620120 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:37:28.620591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:28.621238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:37:28.621238 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:28.621238 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:37:28.621238 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:37:28.621238 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:28.621238 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:37:28.621238 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:28.621238 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:37:28.622339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.043 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:28.622339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:37:28.622339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:37:28.622339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:37:28.622339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:37:28.622339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:37:28.622339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:37:28.622339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:28.622339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:37:28.622339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:28.622339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:37:28.622339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:28.623143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:37:28.623143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:28.623143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:37:28.623143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:28.623143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:37:28.623143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:37:28.623143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:28.623143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:37:28.623143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:28.623143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:37:28.623143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:37:28.623143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:28.623143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:37:28.623925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:28.623925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:37:28.623925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:28.623925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:37:28.623925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:37:28.623925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:37:28.623925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:37:28.623925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:37:28.623925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:37:28.623925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:37:28.623925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:37:28.623925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:37:28.623925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:37:28.623925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:37:28.623925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:37:28.624803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:37:28.624803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:37:28.624803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:37:28.624803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:37:28.624803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:37:28.624803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:28.624803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:37:28.624803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:28.624803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:37:28.624803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:28.624803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:37:28.624803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:28.624803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:37:28.624803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:28.624803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:37:28.624803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:37:28.624803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:28.624803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:37:28.624803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:28.625895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:37:28.625895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:37:28.625895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:28.625895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:37:28.625895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:28.625895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:37:28.709251 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:37:28.709680 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:37:28.709853 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:37:28.710801 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:37:28.710994 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:37:28.711114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:28.712017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:37:28.712017 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:28.712017 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:37:28.712017 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:37:28.712017 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:28.712017 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:37:28.712017 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:28.712017 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:37:28.713532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:28.713532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:37:28.713532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:37:28.713532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:37:28.713532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:37:28.713532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:37:28.713532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:37:28.713532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:28.713532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:37:28.713532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:28.713532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:37:28.713532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:28.714349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:37:28.714349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:28.714349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:37:28.714349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:28.714349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:37:28.714349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:37:28.714349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:28.714349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:37:28.714349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:28.714349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:37:28.714349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:37:28.714349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:28.714349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:37:28.714996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:28.714996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:37:28.715838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:28.715838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:37:28.715838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:37:28.715838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:37:28.715838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:37:28.715838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:37:28.715838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:37:28.715838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:37:28.715838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:37:28.715838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:37:28.715838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:37:28.715838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:37:28.715838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:37:28.715838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:37:28.715838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:37:28.715838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:37:28.716649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:37:28.716649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:37:28.716649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:28.716649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:37:28.716649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:28.716649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:37:28.716649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:28.716649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:37:28.716649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:28.716649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:37:28.716649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:28.716649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:37:28.716649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:37:28.716649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:28.716649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:37:28.716649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:28.716649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:37:28.716649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:37:28.716649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:28.717410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:37:28.717410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:28.717410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:37:28.717410 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:37:28.717410 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:37:28.717679 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:37:28.717843 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:37:28.718226 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:37:28.718427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:28.718975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:37:28.718975 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:28.718975 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:37:28.718975 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:37:28.718975 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:28.718975 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:37:28.718975 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:28.718975 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:37:28.719803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:28.719803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:37:28.719803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:37:28.719803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:37:28.719803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:28.719803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:37:28.719803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:28.719803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:37:28.719803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:28.719803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:37:28.719803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:28.719803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:37:28.719803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:28.719803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:37:28.719803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:37:28.720512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:28.720512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:37:28.720512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:28.720512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:37:28.720512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:37:28.720512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:28.720512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:37:28.720512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:28.720512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:37:28.720935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:28.720935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:37:28.720935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:37:28.720935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:37:28.720935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:37:28.720935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:37:28.720935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:37:28.720935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:37:28.720935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:37:28.720935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:37:28.720935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:28.720935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:37:28.721525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:28.721525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:37:28.721525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:28.721525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:37:28.721525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:28.721525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:37:28.721525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:28.721525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:37:28.721525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:37:28.721525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:28.721525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:37:28.721525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:28.721525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:37:28.721525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:37:28.721979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:28.721979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:37:28.721979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:28.721979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:37:28.721979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:28.721979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:37:28.721979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:37:28.721979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:37:28.721979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:37:28.721979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:37:28.721979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:37:28.721979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:37:28.721979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:37:28.721979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:37:28.722350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:37:28.722350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:37:28.722350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:37:28.722350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:37:28.722350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:37:28.722350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:37:28.722350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:37:28.722350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:37:28.722350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:37:28.722350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:37:28.722350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:37:28.722350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:28.722350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:37:28.722350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:28.722350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:37:28.722350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:28.722350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:37:28.722350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:28.722350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:37:28.722865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:28.722865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:37:28.722865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:37:28.722865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:28.722865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:37:28.722865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:28.722865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:37:28.722865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:37:28.722865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:28.722865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:37:28.722865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:28.722865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:37:28.725487 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:37:28.725703 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:37:28.725928 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:37:28.726131 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:37:28.726602 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:37:28.726873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:28.727589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:37:28.727589 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:28.727589 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:37:28.727589 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:37:28.727589 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:28.727589 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:37:28.727589 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:28.727589 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:37:28.728671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:28.728671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:37:28.728671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:37:28.728671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:37:28.728671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:28.728671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:37:28.728671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:28.728671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:37:28.728671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:28.728671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:37:28.728671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:28.728671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:37:28.728671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:28.728671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:37:28.728671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:37:28.729030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:28.729030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:37:28.729030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:28.729030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:37:28.729030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:37:28.729030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:28.729030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:37:28.729030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:28.729030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:37:28.729761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:28.729761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:37:28.729761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:37:28.729761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:37:28.729761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:37:28.729761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:37:28.729761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:37:28.729761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:37:28.729761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:37:28.729761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:28.729761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:37:28.730241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:28.730241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:37:28.730241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:28.730241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:37:28.730241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:28.730241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:37:28.730241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:28.730241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:37:28.730241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:37:28.730241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:28.730241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:37:28.730241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:28.730241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:37:28.730241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:37:28.730633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:28.730633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:37:28.730633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:28.730633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:37:28.731174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:28.731174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:37:28.731174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:37:28.731174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:37:28.731174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:37:28.731174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:37:28.731174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:37:28.731174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:37:28.731174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:37:28.731174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:37:28.731174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:37:28.731174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:37:28.731174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:37:28.731174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:37:28.731174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:37:28.731174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:37:28.731174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:37:28.731858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:37:28.731858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:37:28.731858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:37:28.731858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:28.731858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:37:28.731858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:28.731858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:37:28.731858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:28.731858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:37:28.731858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:28.731858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:37:28.731858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:28.731858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:37:28.731858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:37:28.731858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:28.731858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:37:28.731858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:28.731858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:37:28.731858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:37:28.732378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:28.732378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:37:28.732378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:28.732378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:37:28.735928 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:37:28.736163 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:37:28.736273 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:37:28.736448 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:37:28.736957 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:37:28.737329 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:37:28.737620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:28.738445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:37:28.738445 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:28.738445 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:37:28.738445 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:37:28.738445 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:28.738445 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:37:28.738445 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:28.738445 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:37:28.739525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:28.739525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:37:28.739525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:37:28.739525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:37:28.739525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:37:28.739525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:37:28.739525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:37:28.739525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:28.739525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:37:28.739525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:28.739525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:37:28.739525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:28.739912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:37:28.739912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:28.739912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:37:28.739912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:28.739912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:37:28.739912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:37:28.739912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:28.739912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:37:28.739912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:28.739912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:37:28.739912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:37:28.739912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:28.739912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:37:28.740405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:28.740405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:37:28.740585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:28.740585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:37:28.740585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:37:28.740585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:37:28.740585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:37:28.740585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:37:28.740585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:37:28.740585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:37:28.740585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:37:28.740585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:37:28.740585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:37:28.740585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:37:28.740585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:37:28.740585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:37:28.740585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:37:28.740585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:37:28.741150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:37:28.741150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:37:28.741150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:28.741150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:37:28.741150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:28.741150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:37:28.741150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:28.741150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:37:28.741150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:28.741150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:37:28.741150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:28.741150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:37:28.741150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:37:28.741150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:28.741150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:37:28.741150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:28.741150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:37:28.741150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:37:28.741150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:28.741937 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:37:28.741937 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:28.741937 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:37:33.577897 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:37:33.578686 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:37:33.578686 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:37:33.579979 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:37:33.618851 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:37:33.619083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:33.620880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:37:33.620880 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:33.620880 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:37:33.620880 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:37:33.620880 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:33.620880 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:37:33.620880 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:33.620880 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:37:33.621376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:33.621376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:37:33.621376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:37:33.621376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:37:33.621376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:37:33.621376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:37:33.621376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:37:33.621376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:33.621376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:37:33.621376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:33.621376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:37:33.621376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:33.622063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:37:33.622063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:33.622063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:37:33.622063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:33.622063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:37:33.622063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:37:33.622063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:33.622063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:37:33.622063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:33.622063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:37:33.622063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:37:33.622063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:33.622063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:37:33.622827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:33.622827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:37:33.622827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:33.622827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:37:33.622827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:37:33.622827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:37:33.622827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:37:33.622827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:37:33.622827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:37:33.622827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:37:33.622827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:37:33.622827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:37:33.622827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:37:33.622827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:37:33.622827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:37:33.623539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:37:33.623539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:37:33.623539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:37:33.623539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:37:33.623539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:37:33.623539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:33.623539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:37:33.623539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:33.623539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:37:33.623539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:33.623539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:37:33.623539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:33.623539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:37:33.623539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:33.623539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:37:33.623539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:37:33.623539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:33.623539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:37:33.623539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:33.624543 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:37:33.624543 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:37:33.624543 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:33.624543 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:37:33.624543 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:33.624543 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:37:33.729200 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:37:33.729434 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:37:33.729471 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:37:33.729986 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:37:33.730115 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:37:33.730919 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:37:33.731114 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:37:33.731241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:33.731982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:37:33.731982 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:33.731982 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:37:33.731982 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:37:33.731982 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:33.731982 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:37:33.731982 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:33.731982 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:37:33.733192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:33.733192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:37:33.733192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:37:33.733192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:37:33.733192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:37:33.733192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:37:33.733192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:37:33.733192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:33.733192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:37:33.733192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:33.733192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:37:33.733192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:33.733625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:37:33.733625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:33.733625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:37:33.733625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:33.733625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:37:33.733625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:37:33.733625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:33.733625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:37:33.733625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:33.733625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:37:33.733625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:37:33.733625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:33.733625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:37:33.734083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:33.734083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:37:33.734276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:33.734276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:37:33.734276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:37:33.734276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:37:33.734276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:37:33.734276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:37:33.734276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:37:33.734276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:37:33.734276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:37:33.734276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:37:33.734276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:37:33.734276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:37:33.734276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:37:33.734276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:37:33.734276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:37:33.734276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:37:33.734888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:37:33.734888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:37:33.734888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:33.734888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:37:33.734888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:33.734888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:37:33.734888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:33.734888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:37:33.734888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:33.734888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:37:33.734888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:33.734888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:37:33.734888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:37:33.734888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:33.734888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:37:33.734888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:33.734888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:37:33.734888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:37:33.734888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:33.735492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:37:33.735492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:33.735492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:37:33.735492 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:37:33.735492 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:37:33.735492 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:37:33.735492 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:37:33.735743 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:33.736040 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:37:33.736040 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:33.736040 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:37:33.736040 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:37:33.736040 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:33.736040 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:37:33.736040 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:33.736040 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:37:33.736719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:33.736719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:37:33.736719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:37:33.736719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:37:33.736719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:33.736719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:37:33.736719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:33.736719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:37:33.736719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:33.736719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:37:33.736719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:33.736719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:37:33.736719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:33.736719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:37:33.736719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:37:33.737286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:33.737286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:37:33.737286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:33.737286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:37:33.737286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:37:33.737286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:33.737286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:37:33.737286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:33.737286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:37:33.737710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:33.737710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:37:33.737710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:37:33.737710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:37:33.737710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:37:33.737710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:37:33.737710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:37:33.737710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:37:33.737710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:37:33.737710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:37:33.737710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:33.737710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:37:33.738114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:33.738114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:37:33.738114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:33.738114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:37:33.738114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:33.738114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:37:33.738114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:33.738114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:37:33.738114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:37:33.738114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:33.738114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:37:33.738114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:33.738114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:37:33.738114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:37:33.738495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:33.738495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:37:33.738495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:33.738495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:37:33.738495 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:37:33.739383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:33.739383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:37:33.739383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:37:33.739383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:37:33.739383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:37:33.739383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:37:33.739383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:37:33.739383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:37:33.739383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:37:33.739383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:37:33.739383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:37:33.739383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:37:33.739383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:37:33.739383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:37:33.739383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:37:33.739383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:37:33.739869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:37:33.739869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:37:33.739869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:37:33.739869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:37:33.739869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:37:33.739869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:33.739869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:37:33.739869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:33.739869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:37:33.739869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:33.739869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:37:33.739869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:33.739869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:37:33.739869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:33.739869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:37:33.739869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:37:33.739869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:33.739869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:37:33.740377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:33.740377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:37:33.740377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:37:33.740377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:33.740377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:37:33.740377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:33.740377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:37:33.740377 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:37:33.740377 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:37:33.740377 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:37:33.740740 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:37:33.740805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:33.741620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:37:33.741620 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:33.741620 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:37:33.741620 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:37:33.741620 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:33.741620 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:37:33.741620 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:33.741620 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:37:33.742206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:33.742206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:37:33.742206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:37:33.742206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:37:33.742206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:33.742206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:37:33.742206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:33.742206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:37:33.742206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:33.742206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:37:33.742206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:33.742206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:37:33.742206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:33.742206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:37:33.742206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:37:33.742646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:33.742646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:37:33.742646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:33.742646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:37:33.742646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:37:33.742646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:33.742646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:37:33.742646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:33.742646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:37:33.742951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:33.742951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:37:33.742951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:37:33.742951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:37:33.742951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:37:33.742951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:37:33.742951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:37:33.742951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:37:33.742951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:37:33.742951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:33.742951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:37:33.743286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:33.743286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:37:33.743286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:33.743286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:37:33.743286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:33.743286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:37:33.743286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:33.743286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:37:33.743286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:37:33.743286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:33.743286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:37:33.743286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:33.743286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:37:33.743286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:37:33.743666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:33.743666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:37:33.743666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:33.743666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:37:33.743837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:33.743837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:37:33.743837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:37:33.743837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:37:33.743837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:37:33.743837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:37:33.743837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:37:33.743837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:37:33.743837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:37:33.743837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:37:33.743837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:37:33.743837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:37:33.743837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:37:33.743837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:37:33.743837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:37:33.743837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:37:33.743837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:37:33.744395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:37:33.744395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:37:33.744395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:37:33.744395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:33.744395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:37:33.744395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:33.744395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:37:33.744395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:33.744395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:37:33.744395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:33.744395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:37:33.744395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:33.744395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:37:33.744395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:37:33.744395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:33.744395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:37:33.744395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:33.744395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:37:33.744395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:37:33.745011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:33.745011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:37:33.745011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:33.745011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:37:33.745011 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:37:33.745011 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:37:33.745011 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:37:33.745011 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:37:33.745262 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:37:33.745303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:33.746082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:37:33.746082 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:33.746082 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:37:33.746082 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:37:33.746082 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:33.746082 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:37:33.746082 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:33.746082 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:37:33.746786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:33.746786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:37:33.746786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:37:33.746786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:37:33.746786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:37:33.746786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:37:33.746786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:37:33.746786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:33.746786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:37:33.746786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:33.746786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:37:33.746786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:33.747142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:37:33.747142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:33.747142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:37:33.747142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:33.747142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:37:33.747142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:37:33.747142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:33.747142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:37:33.747142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:33.747142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:37:33.747142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:37:33.747142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:33.747142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:37:33.747502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:33.747502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:37:33.747569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:33.747569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:37:33.747569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:37:33.747569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:37:33.747569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:37:33.747569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:37:33.747569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:37:33.747569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:37:33.747569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:37:33.747569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:37:33.747569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:37:33.747569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:37:33.747569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:37:33.747569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:37:33.747569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:37:33.747569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:37:33.748027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:37:33.748027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:37:33.748027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:33.748027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:37:33.748027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:33.748027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:37:33.748027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:33.748027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:37:33.748027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:33.748027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:37:33.748027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:33.748027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:37:33.748027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:37:33.748027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:33.748027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:37:33.748027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:33.748027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:37:33.748027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:37:33.748027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:33.748533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:37:33.748533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:33.748533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:37:38.578702 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:37:38.580634 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:37:38.580634 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:37:38.581206 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:37:38.622312 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:37:38.622734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:38.623707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:37:38.623707 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:38.623707 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:37:38.623707 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:37:38.623707 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:38.623707 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:37:38.623707 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:38.623707 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:37:38.624922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.044 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:38.624922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:37:38.624922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:37:38.624922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:37:38.624922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:37:38.624922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:37:38.624922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:37:38.624922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:38.624922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:37:38.624922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:38.624922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:37:38.624922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:38.625527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:37:38.625527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:38.625527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:37:38.625527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:38.625527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:37:38.625527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:37:38.625527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:38.625527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:37:38.625527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:38.625527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:37:38.625527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:37:38.625527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:38.625527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:37:38.626155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:38.626155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:37:38.626258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:38.626258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:37:38.626258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:37:38.626258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:37:38.626258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:37:38.626258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:37:38.626258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:37:38.626258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:37:38.626258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:37:38.626258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:37:38.626258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:37:38.626258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:37:38.626258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:37:38.626258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:37:38.626258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:37:38.626258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:37:38.626979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:37:38.626979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:37:38.626979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:38.626979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:37:38.626979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:38.626979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:37:38.626979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:38.626979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:37:38.626979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:38.626979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:37:38.626979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:38.626979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:37:38.626979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:37:38.626979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:38.626979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:37:38.626979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:38.626979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:37:38.626979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:37:38.626979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:38.627869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:37:38.627869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:38.627869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:37:38.730126 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:37:38.730635 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:37:38.730932 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:37:38.732581 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:37:38.732581 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:37:38.732581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:38.733566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:37:38.733566 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:38.733566 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:37:38.733566 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:37:38.733566 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:38.733566 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:37:38.733566 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:38.733566 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:37:38.736696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:38.736696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:37:38.736696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:37:38.736696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:37:38.736696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:37:38.736696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:37:38.736696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:37:38.736696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:38.736696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:37:38.736696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:38.736696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:37:38.736696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:38.737197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:37:38.737197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:38.737197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:37:38.737197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:38.737197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:37:38.737197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:37:38.737197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:38.737197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:37:38.737197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:38.737197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:37:38.737197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:37:38.737197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:38.737197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:37:38.737553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:38.737553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:37:38.737553 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:37:38.739340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:38.739340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:37:38.739340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:37:38.739340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:37:38.739340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:37:38.739340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:37:38.739340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:37:38.739340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:37:38.739340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:37:38.739340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:37:38.739340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:37:38.739340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:37:38.739340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:37:38.739340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:37:38.739340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:37:38.739340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:37:38.739861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:37:38.739861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:37:38.739861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:38.739861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:37:38.739861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:38.739861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:37:38.739861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:38.739861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:37:38.739861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:38.739861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:37:38.739861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:38.739861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:37:38.739861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:37:38.739861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:38.739861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:37:38.739861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:38.739861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:37:38.739861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:37:38.739861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:38.740456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:37:38.740456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:38.740456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:37:38.740456 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:37:38.740456 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:37:38.740456 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:37:38.740456 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:37:38.740680 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:37:38.740720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:38.741302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:37:38.741302 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:38.741302 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:37:38.741302 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:37:38.741302 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:38.741302 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:37:38.741302 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:38.741302 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:37:38.741982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:38.741982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:37:38.741982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:37:38.741982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:37:38.741982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:38.741982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:37:38.741982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:38.741982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:37:38.741982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:38.741982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:37:38.741982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:38.741982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:37:38.741982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:38.741982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:37:38.741982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:37:38.742544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:38.742544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:37:38.742544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:38.742544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:37:38.742544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:37:38.742544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:38.742544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:37:38.742544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:38.742544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:37:38.742823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:38.742823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:37:38.742823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:37:38.742823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:37:38.742823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:37:38.742823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:37:38.742823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:37:38.742823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:37:38.742823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:37:38.742823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:37:38.742823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:38.742823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:37:38.743180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:38.743180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:37:38.743180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:38.743180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:37:38.743180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:38.743180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:37:38.743180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:38.743180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:37:38.743180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:37:38.743180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:38.743180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:37:38.743180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:38.743180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:37:38.743180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:37:38.743572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:38.743572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:37:38.743572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:38.743572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:37:38.743769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:38.743769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:37:38.743769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:37:38.743769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:37:38.743769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:37:38.743769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:37:38.743769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:37:38.743769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:37:38.743769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:37:38.743769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:37:38.743769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:37:38.743769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:37:38.743769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:37:38.743769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:37:38.743769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:37:38.743769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:37:38.744190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:37:38.744190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:37:38.744190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:37:38.744190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:37:38.744190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:37:38.744190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:38.744190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:37:38.744190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:38.744190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:37:38.744190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:38.744190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:37:38.744190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:38.744190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:37:38.744190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:38.744190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:37:38.744190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:37:38.744190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:38.744190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:37:38.744706 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:38.744706 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:37:38.744706 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:37:38.744706 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:38.744706 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:37:38.744706 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:38.744706 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:37:38.744706 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:37:38.744706 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:37:38.744706 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:37:38.745020 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:37:38.745020 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:37:38.745020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:38.745469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:37:38.745469 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:38.745469 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:37:38.745469 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:37:38.745469 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:38.745469 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:37:38.745469 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:38.745469 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:37:38.746229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:38.746229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:37:38.746229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:37:38.746229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:37:38.746229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:38.746229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:37:38.746229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:38.746229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:37:38.746229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:38.746229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:37:38.746229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:38.746229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:37:38.746229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:38.746229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:37:38.746229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:37:38.746652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:38.746652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:37:38.746652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:38.746652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:37:38.746652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:37:38.746652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:38.746652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:37:38.746652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:38.746652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:37:38.747057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:38.747057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:37:38.747057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:37:38.747057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:37:38.747057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:37:38.747057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:37:38.747057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:37:38.747057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:37:38.747057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:37:38.747057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:38.747057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:37:38.748850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:38.748850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:37:38.748850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:38.748850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:37:38.748850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:38.748850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:37:38.748850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:38.748850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:37:38.748850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:37:38.748850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:38.748850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:37:38.748850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:38.748850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:37:38.748850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:37:38.749200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:38.749200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:37:38.749200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:38.749200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:37:38.749200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:38.749200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:37:38.749200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:37:38.749200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:37:38.749200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:37:38.749200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:37:38.749200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:37:38.749200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:37:38.749200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:37:38.749200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:37:38.749608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:37:38.749608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:37:38.749608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:37:38.749608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:37:38.749608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:37:38.749608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:37:38.749608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:37:38.749608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:37:38.749608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:37:38.749608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:37:38.749608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:38.749608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:37:38.749608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:38.749608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:37:38.749608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:38.749608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:37:38.749608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:38.749608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:37:38.749608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:38.750114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:37:38.750114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:37:38.750114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:38.750114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:37:38.750114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:38.750114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:37:38.750114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:37:38.750114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:38.750114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:37:38.750114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:38.750114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:37:38.750114 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:37:38.750114 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:37:38.750671 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:37:38.750671 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:37:38.750671 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:37:38.750671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:38.750884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:37:38.750884 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:38.750884 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:37:38.750884 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:37:38.750884 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:38.750884 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:37:38.750884 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:38.750884 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:37:38.752239 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:38.752239 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:37:38.752239 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:37:38.752239 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:37:38.752239 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:37:38.752239 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:37:38.752239 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:37:38.752239 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:38.752239 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:37:38.752239 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:38.752239 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:37:38.752239 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:38.752555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:37:38.752555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:38.752555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:37:38.752555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:38.752555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:37:38.752555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:37:38.752555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:38.752555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:37:38.752555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:38.752555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:37:38.752555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:37:38.752555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:38.752555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:37:38.752822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:38.752822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:37:38.753283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:38.753283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:37:38.753283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:37:38.753283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:37:38.753283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:37:38.753283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:37:38.753283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:37:38.753283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:37:38.753283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:37:38.753283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:37:38.753283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:37:38.753283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:37:38.753283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:37:38.753283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:37:38.753283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:37:38.753283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:37:38.753654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:37:38.753654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:37:38.753654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:38.753654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:37:38.753654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:38.753654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:37:38.753654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:38.753654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:37:38.753654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:38.753654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:37:38.753654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:38.753654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:37:38.753654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:37:38.753654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:38.753654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:37:38.753654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:38.753654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:37:38.753654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:37:38.753654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:38.754066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:37:38.754066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:38.754066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:37:43.579267 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:37:43.580215 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:37:43.580215 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:37:43.581169 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:37:43.616502 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:37:43.616964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:43.617979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:37:43.617979 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:43.617979 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:37:43.617979 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:37:43.617979 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:43.617979 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:37:43.617979 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:43.617979 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:37:43.619209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:43.619209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:37:43.619209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:37:43.619209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:37:43.619209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:37:43.619209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:37:43.619209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:37:43.619209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:43.619209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:37:43.619209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:43.619209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:37:43.619209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:43.619792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:37:43.619792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:43.619792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:37:43.619792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:43.619792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:37:43.619792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:37:43.619792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:43.619792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:37:43.619792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:43.619792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:37:43.619792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:37:43.619792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:43.619792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:37:43.620322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:43.620322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:37:43.622117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:43.622117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:37:43.622117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:37:43.622117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:37:43.622117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:37:43.622117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:37:43.622117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:37:43.622117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:37:43.622117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:37:43.622117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:37:43.622117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:37:43.622117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:37:43.622117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:37:43.622117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:37:43.622117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:37:43.622117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:37:43.622684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:37:43.622684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:37:43.622684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:43.622684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:37:43.622684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:43.622684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:37:43.622684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:43.622684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:37:43.622684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:43.622684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:37:43.622684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:43.622684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:37:43.622684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:37:43.622684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:43.622684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:37:43.622684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:43.622684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:37:43.622684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:37:43.622684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:43.623197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:37:43.623197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:43.623197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:37:43.730868 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:37:43.731562 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:37:43.731769 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:37:43.733089 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:37:43.733392 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:37:43.733616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:43.734471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:37:43.734471 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:43.734471 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:37:43.734471 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:37:43.734471 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:43.734471 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:37:43.734471 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:43.734471 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:37:43.736009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:43.736009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:37:43.736009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:37:43.736009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:37:43.736009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:37:43.736009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:37:43.736009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:37:43.736009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:43.736009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:37:43.736009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:43.736009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:37:43.736009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:43.736788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:37:43.736788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:43.736788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:37:43.736788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:43.736788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:37:43.736788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:37:43.736788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:43.736788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:37:43.736788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:43.736788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:37:43.736788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:37:43.736788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:43.736788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:37:43.737416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:43.737416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:37:43.737624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:43.737624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:37:43.737624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:37:43.737624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:37:43.737624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:37:43.737624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:37:43.737624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:37:43.737624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:37:43.737624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:37:43.737624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:37:43.737624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:37:43.737624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:37:43.737624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:37:43.737624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:37:43.737624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:37:43.737624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:37:43.738449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:37:43.738449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:37:43.738449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:43.738449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:37:43.738449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:43.738449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:37:43.738449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:43.738449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:37:43.738449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:43.738449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:37:43.738449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:43.738449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:37:43.738449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:37:43.738449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:43.738449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:37:43.738449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:43.738449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:37:43.738449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:37:43.738449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:43.739452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:37:43.739452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:43.739452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:37:43.739873 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:37:43.740320 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:37:43.740408 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:37:43.740545 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:37:43.741055 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:37:43.741357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:43.742057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:37:43.742057 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:43.742057 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:37:43.742057 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:37:43.742057 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:43.742057 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:37:43.742057 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:43.742057 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:37:43.742991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:43.742991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:37:43.742991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:37:43.742991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:37:43.742991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:43.742991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:37:43.742991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:43.742991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:37:43.742991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:43.742991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:37:43.742991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:43.742991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:37:43.742991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:43.742991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:37:43.742991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:37:43.744032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:43.744032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:37:43.744032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:43.744032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:37:43.744032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:37:43.744032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:43.744032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:37:43.744032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:43.744032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:37:43.744032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:43.744032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:37:43.744032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:37:43.744032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:37:43.744032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:37:43.744735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:37:43.744735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:37:43.744735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:37:43.744735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:37:43.744735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:37:43.744735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:43.744735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:37:43.744735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:43.744735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:37:43.744735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:43.744735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:37:43.744735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:43.744735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:37:43.745340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:43.745340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:37:43.745340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:37:43.745340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:43.745340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:37:43.745340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:43.745340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:37:43.745340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:37:43.745340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:43.745340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:37:43.745340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:43.745340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:37:43.745900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:43.745900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:37:43.745900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:37:43.745900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:37:43.745900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:37:43.745900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:37:43.745900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:37:43.745900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:37:43.745900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:37:43.745900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:37:43.745900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:37:43.745900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:37:43.745900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:37:43.745900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:37:43.745900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:37:43.745900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:37:43.746787 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:37:43.746787 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:37:43.746787 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:37:43.746787 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:37:43.746787 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:37:43.746787 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:43.746787 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:37:43.746787 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:43.746787 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:37:43.746787 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:43.746787 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:37:43.746787 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:43.746787 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:37:43.746787 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:43.746787 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:37:43.746787 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:37:43.746787 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:43.746787 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:37:43.747669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:43.747669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:37:43.747669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:37:43.747669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:43.747669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:37:43.747669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:43.747669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:37:43.747669 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:37:43.747669 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:37:43.747669 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:37:43.748267 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:37:43.748267 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:37:43.748267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:43.748267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:37:43.748267 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:43.748267 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:37:43.748267 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:37:43.748267 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:43.748267 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:37:43.748267 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:43.748267 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:37:43.748267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:43.748960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:37:43.748960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:37:43.748960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:37:43.748960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:43.748960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:37:43.748960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:43.748960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:37:43.748960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:43.748960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:37:43.748960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:43.748960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:37:43.748960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:43.748960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:37:43.748960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:37:43.748960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:43.748960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:37:43.748960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:43.749777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:37:43.749777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:37:43.749777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:43.749777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:37:43.749777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:43.749777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:37:43.749777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:43.749777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:37:43.749777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:37:43.749777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:37:43.749777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:37:43.749777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:37:43.749777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:37:43.750565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:37:43.750565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:37:43.750565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:43.750565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:37:43.750565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:43.750565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:37:43.750565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:43.750565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:37:43.750565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:43.750565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:37:43.750565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:43.750565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:37:43.750565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:37:43.751250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:43.751250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:37:43.751250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:43.751250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:37:43.751250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:37:43.751250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:43.751250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:37:43.751250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:43.751250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:37:43.751250 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:37:43.751810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:43.751810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:37:43.751810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:37:43.751810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:37:43.751810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:37:43.751810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:37:43.751810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:37:43.751810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:37:43.751810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:37:43.751810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:37:43.751810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:37:43.751810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:37:43.751810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:37:43.751810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:37:43.751810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:37:43.751810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:37:43.751810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:37:43.752497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:37:43.752497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:37:43.752497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:37:43.752497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:43.752497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:37:43.752497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:43.752497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:37:43.752497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:43.752497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:37:43.752497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:43.752497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:37:43.752497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:43.752497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:37:43.752497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:37:43.752497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:43.752497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:37:43.752497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:43.752497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:37:43.752497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:37:43.752843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:43.752843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:37:43.752843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:43.752843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:37:43.752843 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:37:43.752843 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:37:43.752843 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:37:43.752843 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:37:43.753001 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:37:43.753001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:43.753001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:37:43.753001 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:43.753001 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:37:43.753001 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:37:43.753001 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:43.753001 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:37:43.753001 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:43.753001 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:37:43.753462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:43.753462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:37:43.753462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:37:43.753462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:37:43.753462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:37:43.753462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:37:43.753462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:37:43.753462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:43.753462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:37:43.753462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:43.753462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:37:43.753462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:43.753697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:37:43.753697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:43.753697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:37:43.753697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:43.753697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:37:43.753697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:37:43.753697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:43.753697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:37:43.753697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:43.753697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:37:43.753697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:37:43.753697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:43.753697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:37:43.753968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:43.753968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:37:43.754438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:43.754438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:37:43.754438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:37:43.754438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:37:43.754438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:37:43.754438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:37:43.754438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:37:43.754438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:37:43.754438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:37:43.754438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:37:43.754438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:37:43.754438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:37:43.754438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:37:43.754438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:37:43.754438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:37:43.754438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:37:43.754728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:37:43.754728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:37:43.754728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:43.754728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:37:43.754728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:43.754728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:37:43.754728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:43.754728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:37:43.754728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:43.754728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:37:43.754728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:43.754728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:37:43.754728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:37:43.754728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:43.754728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:37:43.754728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:43.754728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:37:43.754728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:37:43.754728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:43.755063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:37:43.755063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:43.755063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:37:48.579912 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:37:48.580459 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:37:48.580520 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:37:48.581435 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:37:48.617072 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:37:48.617226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:48.617892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:37:48.617892 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:48.617892 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:37:48.617892 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:37:48.617892 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:48.617892 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:37:48.617892 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:48.617892 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:37:48.618706 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:48.618706 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:37:48.618706 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:37:48.618706 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:37:48.618706 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:37:48.618706 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:37:48.618706 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:37:48.618706 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:48.618706 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:37:48.618706 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:48.618706 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:37:48.618706 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:48.619209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:37:48.619209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:48.619209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:37:48.619209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:48.619209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:37:48.619209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:37:48.619209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:48.619209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:37:48.619209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:48.619209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:37:48.619209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:37:48.619209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:48.619209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:37:48.619823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:48.619823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:37:48.619823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:48.619823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:37:48.619823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:37:48.619823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:37:48.619823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:37:48.619823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:37:48.619823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:37:48.619823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:37:48.619823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:37:48.619823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:37:48.619823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:37:48.619823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:37:48.619823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:37:48.620343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:37:48.620343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:37:48.620343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:37:48.620343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:37:48.620343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:37:48.620343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:48.620343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:37:48.620343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:48.620343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:37:48.620343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:48.620343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:37:48.620343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:48.620343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:37:48.620343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:48.620343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:37:48.620343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:37:48.620343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:48.620343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:37:48.620343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:48.621244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:37:48.621244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:37:48.621244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:48.621244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:37:48.621244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:48.621244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:37:48.732377 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:37:48.732987 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:37:48.733262 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:37:48.734032 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:37:48.734267 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:37:48.734471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:48.735323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:37:48.735323 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:48.735323 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:37:48.735323 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:37:48.735323 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:48.735323 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:37:48.735323 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:48.735323 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:37:48.736480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:48.736480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:37:48.736480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:37:48.736480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:37:48.736480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:37:48.736480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:37:48.736480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:37:48.736480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:48.736480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:37:48.736480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:48.736480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:37:48.736480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:48.736918 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:37:48.736918 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:48.736918 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:37:48.736918 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:48.736918 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:37:48.736918 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:37:48.736918 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:48.736918 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:37:48.736918 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:48.736918 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:37:48.736918 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:37:48.736918 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:48.736918 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:37:48.737260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:48.737260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:37:48.737599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:48.737599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:37:48.737599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:37:48.737599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:37:48.737599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:37:48.737599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:37:48.737599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:37:48.737599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:37:48.737599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:37:48.737599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:37:48.737599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:37:48.737599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:37:48.737599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:37:48.737599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:37:48.737599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:37:48.737599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:37:48.738075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:37:48.738075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:37:48.738075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:48.738075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:37:48.738075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:48.738075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:37:48.738075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:48.738075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:37:48.738075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:48.738075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:37:48.738075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:48.738075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:37:48.738075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:37:48.738075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:48.738075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:37:48.738075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:48.738075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:37:48.738075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:37:48.738075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:48.738622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:37:48.738622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:48.738622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:37:48.740395 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:37:48.740717 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:37:48.740854 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:37:48.740978 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:37:48.742142 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:37:48.742690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:48.743762 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:37:48.743762 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:48.743762 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:37:48.743762 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:37:48.743762 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:48.743762 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:37:48.743762 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:48.743762 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:37:48.744652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:48.744652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:37:48.744652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:37:48.744652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:37:48.744652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:48.744652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:37:48.744652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:48.744652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:37:48.744652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:48.744652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:37:48.744652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:48.744652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:37:48.744652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:48.744652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:37:48.744652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:37:48.745193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:48.745193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:37:48.745193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:48.745193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:37:48.745193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:37:48.745193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:48.745193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:37:48.745193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:48.745193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:37:48.745903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:48.745903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:37:48.745903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:37:48.745903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:37:48.745903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:37:48.745903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:37:48.745903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:37:48.745903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:37:48.745903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:37:48.745903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:37:48.745903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:48.745903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:37:48.746265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:48.746265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:37:48.746265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:48.746265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:37:48.746265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:48.746265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:37:48.746265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:48.746265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:37:48.746265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:37:48.746265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:48.746265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:37:48.746265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:48.746265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:37:48.746265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:37:48.746592 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:48.746592 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:37:48.746592 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:48.746592 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:37:48.747530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:48.747530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:37:48.747530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:37:48.747530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:37:48.747530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:37:48.747530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:37:48.747530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:37:48.747530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:37:48.747530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:37:48.747530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:37:48.747530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:37:48.747530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:37:48.747530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:37:48.747530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:37:48.747530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:37:48.747530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:37:48.748023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:37:48.748023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:37:48.748023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:37:48.748023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:37:48.748023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:37:48.748023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:48.748023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:37:48.748023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:48.748023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:37:48.748023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:48.748023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:37:48.748023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:48.748023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:37:48.748023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:48.748023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:37:48.748023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:37:48.748023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:48.748023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:37:48.748487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:48.748487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:37:48.748487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:37:48.748487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:48.748487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:37:48.748487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:48.748487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:37:48.748487 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:37:48.748487 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:37:48.748487 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:37:48.748817 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:37:48.749256 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:37:48.749656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:48.750662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:37:48.750662 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:48.750662 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:37:48.750662 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:37:48.750662 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:48.750662 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:37:48.750662 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:48.750662 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:37:48.750954 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:37:48.752519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:48.752519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:37:48.752519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:37:48.752519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:37:48.752519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:48.752519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:37:48.752519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:48.752519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:37:48.752519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:48.752519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:37:48.752519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:48.752519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:37:48.752519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:48.752519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:37:48.752519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:37:48.752915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:48.752915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:37:48.752915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:48.752915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:37:48.752915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:37:48.752915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:48.752915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:37:48.752915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:48.752915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:37:48.753562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:48.753562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:37:48.753562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:37:48.753562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:37:48.753562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:37:48.753562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:37:48.753562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:37:48.753562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:37:48.753562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:37:48.753562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:48.753562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:37:48.753885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:48.753885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:37:48.753885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:48.753885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:37:48.753885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:48.753885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:37:48.753885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:48.753885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:37:48.753885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:37:48.753885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:48.753885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:37:48.753885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:48.753885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:37:48.753885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:37:48.754208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:48.754208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:37:48.754208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:48.754208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:37:48.754792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:48.754792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:37:48.754792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:37:48.754792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:37:48.754792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:37:48.754792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:37:48.754792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:37:48.754792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:37:48.754792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:37:48.754792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:37:48.754792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:37:48.754792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:37:48.754792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:37:48.754792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:37:48.754792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:37:48.754792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:37:48.754792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:37:48.755209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:37:48.755209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:37:48.755209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:37:48.755209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:48.755209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:37:48.755209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:48.755209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:37:48.755209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:48.755209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:37:48.755209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:48.755209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:37:48.755209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:48.755209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:37:48.755209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:37:48.755209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:48.755209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:37:48.755209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:48.755209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:37:48.755209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:37:48.755568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:48.755568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:37:48.755568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:48.755568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:37:48.755568 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:37:48.755568 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:37:48.755568 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:37:48.755896 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:37:48.756262 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:37:48.756495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:48.756977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:37:48.756977 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:48.756977 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:37:48.756977 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:37:48.756977 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:48.756977 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:37:48.756977 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:48.756977 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:37:48.758081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:48.758081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:37:48.758081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:37:48.758081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:37:48.758081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:37:48.758081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:37:48.758081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:37:48.758081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:48.758081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:37:48.758081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:48.758081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:37:48.758081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:48.758421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:37:48.758421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:48.758421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:37:48.758421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:48.758421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:37:48.758421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:37:48.758421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:48.758421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:37:48.758421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:48.758421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:37:48.758421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:37:48.758421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:48.758421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:37:48.758735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:48.758735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:37:48.759355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:48.759355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:37:48.759355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:37:48.759355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:37:48.759355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:37:48.759355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:37:48.759355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:37:48.759355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:37:48.759355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:37:48.759355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:37:48.759355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:37:48.759355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:37:48.759355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:37:48.759355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:37:48.759355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:37:48.759355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:37:48.759805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:37:48.759805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:37:48.759805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:48.759805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:37:48.759805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:48.759805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:37:48.759805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:48.759805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:37:48.759805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:48.759805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:37:48.759805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:48.759805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:37:48.759805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:37:48.759805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:48.759805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:37:48.759805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:48.759805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:37:48.759805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:37:48.759805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:48.760257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:37:48.760257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:48.760257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:37:53.580602 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:37:53.581548 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:37:53.581548 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:37:53.582818 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:37:53.619010 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:37:53.619495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:53.620385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:37:53.620385 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:53.620385 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:37:53.620385 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:37:53.620385 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:53.620385 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:37:53.620385 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:53.620385 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:37:53.621392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:53.621392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:37:53.621392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:37:53.621392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:37:53.621392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:37:53.621392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:37:53.621392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:37:53.621392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:53.621392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:37:53.621392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:53.621392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:37:53.621392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:53.622259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:37:53.622259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:53.622259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:37:53.622259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:53.622259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:37:53.622259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:37:53.622259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:53.622259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:37:53.622259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:53.622259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:37:53.622259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:37:53.622259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:53.622259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:37:53.623086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:53.623086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:37:53.623086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:53.623086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:37:53.623086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:37:53.623086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:37:53.623086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:37:53.623086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:37:53.623086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:37:53.623086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:37:53.623086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:37:53.623086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:37:53.623086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:37:53.623086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:37:53.623086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:37:53.624152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:37:53.624152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:37:53.624152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:37:53.624152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:37:53.624152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:37:53.624152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:53.624152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:37:53.624152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:53.624152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:37:53.624152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:53.624152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:37:53.624152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:53.624152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:37:53.624152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:53.624152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:37:53.624152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:37:53.624152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:53.624152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:37:53.624152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:53.626062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:37:53.626062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:37:53.626062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:53.626062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:37:53.626062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:53.626062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:37:53.733058 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:37:53.733699 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:37:53.733842 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:37:53.734460 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:37:53.734636 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:37:53.734809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:53.735364 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:37:53.735364 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:53.735364 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:37:53.735364 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:37:53.735364 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:53.735364 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:37:53.735364 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:53.735364 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:37:53.736075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:53.736075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:37:53.736075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:37:53.736075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:37:53.736075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:37:53.736075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:37:53.736075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:37:53.736075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:53.736075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:37:53.736075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:53.736075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:37:53.736075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:53.736458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:37:53.736458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:53.736458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:37:53.736458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:53.736458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:37:53.736458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:37:53.736458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:53.736458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:37:53.736458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:53.736458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:37:53.736458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:37:53.736458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:53.736458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:37:53.736828 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:53.736828 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:37:53.736896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:53.736896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:37:53.736896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:37:53.736896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:37:53.736896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:37:53.736896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:37:53.736896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:37:53.736896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:37:53.736896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:37:53.736896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:37:53.736896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:37:53.736896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:37:53.736896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:37:53.736896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:37:53.736896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:37:53.736896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:37:53.737325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:37:53.737325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:37:53.737325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:53.737325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:37:53.737325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:53.737325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:37:53.737325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:53.737325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:37:53.737325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:53.737325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:37:53.737325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:53.737325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:37:53.737325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:37:53.737325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:53.737325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:37:53.737325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:53.737325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:37:53.737325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:37:53.737325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:53.737853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:37:53.737853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:53.737853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:37:53.740987 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:37:53.741432 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:37:53.741610 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:37:53.741846 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:37:53.742401 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:37:53.742603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:53.743238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:37:53.743238 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:53.743238 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:37:53.743238 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:37:53.743238 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:53.743238 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:37:53.743238 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:53.743238 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:37:53.744289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:53.744289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:37:53.744289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:37:53.744289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:37:53.744289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:53.744289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:37:53.744289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:53.744289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:37:53.744289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:53.744289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:37:53.744289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:53.744289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:37:53.744289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:53.744289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:37:53.744289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:37:53.744956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:53.744956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:37:53.744956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:53.744956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:37:53.744956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:37:53.744956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:53.744956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:37:53.744956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:53.744956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:37:53.745359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:53.745359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:37:53.745359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:37:53.745359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:37:53.745359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:37:53.745359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:37:53.745359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:37:53.745359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:37:53.745359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:37:53.745359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:37:53.745359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:53.745359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:37:53.746334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:53.746334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:37:53.746334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:53.746334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:37:53.746334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:53.746334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:37:53.746334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:53.746334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:37:53.746334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:37:53.746334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:53.746334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:37:53.746334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:53.746334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:37:53.746334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:37:53.746783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:53.746783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:37:53.746783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:53.746783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:37:53.746783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:53.746783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:37:53.746783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:37:53.746783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:37:53.746783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:37:53.746783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:37:53.746783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:37:53.746783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:37:53.746783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:37:53.746783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:37:53.747230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:37:53.747230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:37:53.747230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:37:53.747230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:37:53.747230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:37:53.747230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:37:53.747230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:37:53.747230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:37:53.747230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:37:53.747230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:37:53.747230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:37:53.747230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:53.747230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:37:53.747230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:53.747230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:37:53.747230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:53.747230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:37:53.747230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:53.747230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:37:53.747987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:53.747987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:37:53.747987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:37:53.747987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:53.747987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:37:53.747987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:53.747987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:37:53.747987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:37:53.747987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:53.747987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:37:53.747987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:53.747987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:37:53.748356 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:37:53.748825 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:37:53.748825 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:37:53.748968 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:37:53.749272 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:37:53.749538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:53.750056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:37:53.750056 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:53.750056 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:37:53.750056 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:37:53.750056 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:53.750056 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:37:53.750056 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:53.750056 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:37:53.750810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:53.750810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:37:53.750810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:37:53.750810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:37:53.750810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:53.750810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:37:53.750810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:53.750810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:37:53.750810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:53.750810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:37:53.750810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:53.750810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:37:53.750810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:53.750810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:37:53.750810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:37:53.751112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:53.751112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:37:53.751112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:53.751112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:37:53.751112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:37:53.751112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:53.751112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:37:53.751112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:53.751112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:37:53.751870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:53.751870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:37:53.751870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:37:53.751870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:37:53.751870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:37:53.751870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:37:53.751870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:37:53.751870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:37:53.751870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:37:53.751870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:53.751870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:37:53.752203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:53.752203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:37:53.752203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:53.752203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:37:53.752203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:53.752203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:37:53.752203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:53.752203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:37:53.752203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:37:53.752203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:53.752203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:37:53.752203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:53.752203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:37:53.752203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:37:53.752474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:53.752474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:37:53.752474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:53.752474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:37:53.752975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:53.752975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:37:53.752975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:37:53.752975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:37:53.752975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:37:53.752975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:37:53.752975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:37:53.752975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:37:53.752975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:37:53.752975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:37:53.752975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:37:53.752975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:37:53.752975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:37:53.752975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:37:53.752975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:37:53.752975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:37:53.752975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:37:53.753376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:37:53.753376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:37:53.753376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:37:53.753376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:53.753376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:37:53.753376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:53.753376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:37:53.753376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:53.753376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:37:53.753376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:53.753376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:37:53.753376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:53.753376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:37:53.753376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:37:53.753376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:53.753376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:37:53.753376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:53.753376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:37:53.753376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:37:53.753771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:53.753771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:37:53.753771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:53.753771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:37:53.755089 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:37:53.755383 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:37:53.755599 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:37:53.755779 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:37:53.756166 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:37:53.756398 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:37:53.756546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:53.757149 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:37:53.757149 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:53.757149 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:37:53.757149 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:37:53.757149 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:53.757149 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:37:53.757149 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:53.757149 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:37:53.758878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:53.758878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:37:53.758878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:37:53.758878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:37:53.758878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:37:53.758878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:37:53.758878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:37:53.758878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:53.758878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:37:53.758878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:53.758878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:37:53.758878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:53.759178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:37:53.759178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:53.759178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:37:53.759178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:53.759178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:37:53.759178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:37:53.759178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:53.759178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:37:53.759178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:53.759178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:37:53.759178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:37:53.759178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:53.759178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:37:53.759474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:53.759474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:37:53.760152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:53.760152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:37:53.760152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:37:53.760152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:37:53.760152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:37:53.760152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:37:53.760152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:37:53.760152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:37:53.760152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:37:53.760152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:37:53.760152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:37:53.760152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:37:53.760152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:37:53.760152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:37:53.760152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:37:53.760152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:37:53.760518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:37:53.760518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:37:53.760518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:53.760518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:37:53.760518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:53.760518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:37:53.760518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:53.760518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:37:53.760518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:53.760518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:37:53.760518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:53.760518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:37:53.760518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:37:53.760518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:53.760518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:37:53.760518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:53.760518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:37:53.760518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:37:53.760518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:53.760923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:37:53.760923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:53.760923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:37:58.581303 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:37:58.582699 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:37:58.582699 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:37:58.583507 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:37:58.585262 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG dbcounter [-] [106314] Writing DB stats neutron:SELECT=240 {{(pid=106314) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 09 09:37:58.621409 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:37:58.621849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:58.622987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:37:58.622987 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:58.622987 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:37:58.622987 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:37:58.622987 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:58.622987 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:37:58.622987 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:58.622987 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:37:58.624155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:58.624155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:37:58.624155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:37:58.624155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:37:58.624155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:37:58.624155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:37:58.624155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:37:58.624155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:58.624155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:37:58.624155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:58.624155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:37:58.624155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:58.625178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:37:58.625178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:58.625178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:37:58.625178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:58.625178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:37:58.625178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:37:58.625178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:58.625178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:37:58.625178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:58.625178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:37:58.625178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:37:58.625178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:58.625178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:37:58.626007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:58.626007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:37:58.626007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:58.626007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:37:58.626007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:37:58.626007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:37:58.626007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:37:58.626007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:37:58.626007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:37:58.626007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:37:58.626007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:37:58.626007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:37:58.626007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:37:58.626007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:37:58.626007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:37:58.626813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:37:58.626813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:37:58.626813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:37:58.626813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:37:58.626813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:37:58.626813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:58.626813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:37:58.626813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:58.626813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:37:58.626813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:58.626813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:37:58.626813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:58.626813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:37:58.626813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:58.626813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:37:58.626813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:37:58.626813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:58.626813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:37:58.626813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:58.628029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:37:58.628029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:37:58.628029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:58.628029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:37:58.628029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:58.628029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:37:58.734337 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:37:58.735041 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:37:58.735159 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:37:58.736116 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:37:58.736402 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:37:58.736613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:58.737480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:37:58.737480 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:58.737480 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:37:58.737480 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:37:58.737480 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:58.737480 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:37:58.737480 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:58.737480 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:37:58.738869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:58.738869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:37:58.738869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:37:58.738869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:37:58.738869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:37:58.738869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:37:58.738869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:37:58.738869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:58.738869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:37:58.738869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:58.738869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:37:58.738869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:58.739287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:37:58.739287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:58.739287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:37:58.739287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:58.739287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:37:58.739287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:37:58.739287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:58.739287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:37:58.739287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:58.739287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:37:58.739287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:37:58.739287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:58.739287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:37:58.739682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:58.739682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:37:58.741840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:58.741840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:37:58.741840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:37:58.741840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:37:58.741840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:37:58.741840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:37:58.741840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:37:58.741840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:37:58.741840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:37:58.741840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:37:58.741840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:37:58.741840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:37:58.741840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:37:58.741840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:37:58.741840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:37:58.741840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:37:58.742410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:37:58.742410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:37:58.742410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:58.742410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:37:58.742410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:58.742410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:37:58.742410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:58.742410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:37:58.742410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:58.742410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:37:58.742410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:58.742410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:37:58.742410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:37:58.742410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:58.742410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:37:58.742410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:58.742410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:37:58.742410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:37:58.742410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:58.742978 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:37:58.742978 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:58.742978 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:37:58.742978 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:37:58.742978 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:37:58.742978 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:37:58.742978 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:37:58.742978 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:37:58.743235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:58.744034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:37:58.744034 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:58.744034 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:37:58.744034 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:37:58.744034 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:58.744034 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:37:58.744034 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:58.744034 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:37:58.745390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:58.745390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:37:58.745390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:37:58.745390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:37:58.745390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:58.745390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:37:58.745390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:58.745390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:37:58.745390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:58.745390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:37:58.745390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:58.745390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:37:58.745390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:58.745390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:37:58.745390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:37:58.745868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:58.745868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:37:58.745868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:58.745868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:37:58.745868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:37:58.745868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:58.745868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:37:58.745868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:58.745868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:37:58.746633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:58.746633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:37:58.746633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:37:58.746633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:37:58.746633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:37:58.746633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:37:58.746633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:37:58.746633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:37:58.746633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:37:58.746633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:37:58.746633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:58.746633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:37:58.747513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:58.747513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:37:58.747513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:58.747513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:37:58.747513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:58.747513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:37:58.747513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:58.747513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:37:58.747513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:37:58.747513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:58.747513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:37:58.747513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:58.747513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:37:58.747513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:37:58.748406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:58.748406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:37:58.748406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:58.748406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:37:58.748692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:58.748692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:37:58.748692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:37:58.748692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:37:58.748692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:37:58.748692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:37:58.748692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:37:58.748692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:37:58.748692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:37:58.748692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:37:58.748692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:37:58.748692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:37:58.748692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:37:58.748692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:37:58.748692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:37:58.748692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:37:58.749854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:37:58.749854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:37:58.749854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:37:58.749854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:37:58.749854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:37:58.749854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:58.749854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:37:58.749854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:58.749854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:37:58.749854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:58.749854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:37:58.749854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:58.749854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:37:58.749854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:58.749854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:37:58.749854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:37:58.749854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:58.749854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:37:58.751065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:58.751065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:37:58.751065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:37:58.751065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:58.751065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:37:58.751065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:58.751065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:37:58.751065 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:37:58.751065 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:37:58.751065 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:37:58.751599 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:37:58.751599 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:37:58.751599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:58.752117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:37:58.752117 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:58.752117 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:37:58.752117 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:37:58.752117 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:58.752117 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:37:58.752117 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:58.752117 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:37:58.752854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:58.752854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:37:58.752854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:37:58.752854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:37:58.752854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:58.752854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:37:58.752854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:58.752854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:37:58.752854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:58.752854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:37:58.752854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:58.752854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:37:58.752854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:58.752854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:37:58.752854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:37:58.754247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:58.754247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:37:58.754247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:58.754247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:37:58.754247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:37:58.754247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:58.754247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:37:58.754247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:58.754247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:37:58.754247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:58.754247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:37:58.754247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:37:58.754247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:37:58.754247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:37:58.755238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:37:58.755238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:37:58.755238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:37:58.755238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:37:58.755238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:58.755238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:37:58.755238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:58.755238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:37:58.755238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:58.755238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:37:58.755238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:58.755238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:37:58.755238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:58.755685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:37:58.755685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:37:58.755685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:58.755685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:37:58.755685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:58.755685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:37:58.755685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:37:58.755685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:58.755685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:37:58.755685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:58.755685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:37:58.755685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:58.756172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:37:58.756172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:37:58.756172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:37:58.756172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:37:58.756172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:37:58.756172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:37:58.756172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:37:58.756172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:37:58.756172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:37:58.756172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:37:58.756172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:37:58.756172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:37:58.756172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:37:58.756172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:37:58.756172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:37:58.756172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:37:58.756172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:37:58.756172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:37:58.756172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:37:58.756599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:58.756599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:37:58.756599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:58.756599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:37:58.756599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:58.756599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:37:58.756599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:58.756599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:37:58.756599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:58.756599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:37:58.756599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:37:58.756599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:58.756599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:37:58.756599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:58.756599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:37:58.756599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:37:58.756599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:58.756599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:37:58.756599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:58.756599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:37:58.756980 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:37:58.756980 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:37:58.756980 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:37:58.756980 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:37:58.756980 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:37:58.756980 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:37:58.757125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:58.757253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:37:58.757253 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:58.757253 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:37:58.757253 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:37:58.757253 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:58.757253 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:37:58.757253 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:58.757253 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:37:58.758321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:58.758321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:37:58.758321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:37:58.758321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:37:58.758321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:37:58.758321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:37:58.758321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:37:58.758321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:58.758321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:37:58.758321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:58.758321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:37:58.758321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:58.758571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:37:58.758571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:58.758571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:37:58.758571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:58.758571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:37:58.758571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:37:58.758571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:58.758571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:37:58.758571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:58.758571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:37:58.758571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:37:58.758571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:58.758571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:37:58.758900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:58.758900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:37:58.759600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:58.759600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:37:58.759600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:37:58.759600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:37:58.759600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:37:58.759600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:37:58.759600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:37:58.759600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:37:58.759600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:37:58.759600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:37:58.759600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:37:58.759600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:37:58.759600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:37:58.759600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:37:58.759600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:37:58.759600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:37:58.759977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:37:58.759977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:37:58.759977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:58.759977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:37:58.759977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:37:58.759977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:37:58.759977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:37:58.759977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:37:58.759977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:37:58.759977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:37:58.759977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:37:58.759977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:37:58.759977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:37:58.759977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:37:58.759977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:37:58.759977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:37:58.759977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:37:58.759977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:37:58.759977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:37:58.760371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:37:58.760371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:37:58.760371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:38:03.582679 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:38:03.583683 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:38:03.583683 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:38:03.585704 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:38:03.621318 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:38:03.621640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:03.622629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:38:03.622629 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:03.622629 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:38:03.622629 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:38:03.622629 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:03.622629 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:38:03.622629 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:03.622629 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:38:03.623647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:03.623647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:38:03.623647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:38:03.623647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:38:03.623647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:38:03.623647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:38:03.623647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:38:03.623647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:03.623647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:38:03.623647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:03.623647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:38:03.623647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:03.624092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:38:03.624092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:03.624092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:38:03.624092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:03.624092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:38:03.624092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:38:03.624092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:03.624092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:38:03.624092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:03.624092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:38:03.624092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:38:03.624092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:03.624092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:38:03.624601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:03.624601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:38:03.625277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:03.625277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:38:03.625277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:38:03.625277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:38:03.625277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:38:03.625277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:38:03.625277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:38:03.625277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:38:03.625277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:38:03.625277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:38:03.625277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:38:03.625277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:38:03.625277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:38:03.625277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:38:03.625277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:38:03.625277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:38:03.625965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:38:03.625965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:38:03.625965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:03.625965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:38:03.625965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:03.625965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:38:03.625965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:03.625965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:38:03.625965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:03.625965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:38:03.625965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:03.625965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:38:03.625965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:38:03.625965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:03.625965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:38:03.625965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:03.625965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:38:03.625965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:38:03.625965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:03.626686 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:38:03.626686 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:03.626686 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:38:03.735112 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:38:03.735933 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:38:03.736219 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:38:03.737673 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:38:03.738177 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:38:03.738452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:03.739320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:38:03.739320 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:03.739320 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:38:03.739320 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:38:03.739320 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:03.739320 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:38:03.739320 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:03.739320 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:38:03.740821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:03.740821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:38:03.740821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:38:03.740821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:38:03.740821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:38:03.740821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:38:03.740821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:38:03.740821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:03.740821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:38:03.740821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:03.740821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:38:03.740821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:03.741523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:38:03.741523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:03.741523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:38:03.741523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:03.741523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:38:03.741523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:38:03.741523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:03.741523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:38:03.741523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:03.741523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:38:03.741523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:38:03.741523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:03.741523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:38:03.742188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:03.742188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:38:03.742319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:03.742319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:38:03.742319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:38:03.742319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:38:03.742319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:38:03.742319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:38:03.742319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:38:03.742319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:38:03.742319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:38:03.742319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:38:03.742319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:38:03.742319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:38:03.742319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:38:03.742319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:38:03.742319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:38:03.742319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:38:03.743198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:38:03.743198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:38:03.743198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:03.743198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:38:03.743198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:03.743198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:38:03.743198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:03.743198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:38:03.743198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:03.743198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:38:03.743198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:03.743198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:38:03.743198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:38:03.743198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:03.743198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:38:03.743198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:03.743198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:38:03.743198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:38:03.743198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:03.744191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:38:03.744191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:03.744191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:38:03.744191 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:38:03.744191 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:38:03.744191 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:38:03.744191 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:38:03.744191 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:38:03.744615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:03.745243 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:38:03.745243 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:03.745243 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:38:03.745243 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:38:03.745243 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:03.745243 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:38:03.745243 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:03.745243 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:38:03.746180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:03.746180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:38:03.746180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:38:03.746180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:38:03.746180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:03.746180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:38:03.746180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:03.746180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:38:03.746180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:03.746180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:38:03.746180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:03.746180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:38:03.746180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:03.746180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:38:03.746180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:38:03.747105 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:03.747105 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:38:03.747105 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:03.747105 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:38:03.747105 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:38:03.747105 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:03.747105 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:38:03.747105 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:03.747105 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:38:03.747502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:03.747502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:38:03.747502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:38:03.747502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:38:03.747502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:38:03.747502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:38:03.747502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:38:03.747502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:38:03.747502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:38:03.747502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:38:03.747502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:03.747502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:38:03.748313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:03.748313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:38:03.748313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:03.748313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:38:03.748313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:03.748313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:38:03.748313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:03.748313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:38:03.748313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:38:03.748313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:03.748313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:38:03.748313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:03.748313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:38:03.748313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:38:03.749012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:03.749012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:38:03.749012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:03.749012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:38:03.749012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:03.749012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:38:03.749012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:38:03.749012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:38:03.749012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:38:03.749012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:38:03.749012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:38:03.749012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:38:03.749012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:38:03.749012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:38:03.749368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:38:03.749368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:38:03.749368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:38:03.749368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:38:03.749368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:38:03.749368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:38:03.749368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:38:03.749368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:38:03.749368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:38:03.749368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:38:03.749368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:38:03.749368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:03.749368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:38:03.749368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:03.749368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:38:03.749368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:03.749368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:38:03.749368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:03.749368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:38:03.750240 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:03.750240 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:38:03.750240 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:38:03.750240 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:03.750240 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:38:03.750240 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:03.750240 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:38:03.750240 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:38:03.750240 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:03.750240 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:38:03.750240 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:03.750240 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:38:03.750240 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:38:03.750240 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:38:03.750682 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:38:03.750682 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:38:03.750814 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:38:03.751085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:03.751647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:38:03.751647 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:03.751647 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:38:03.751647 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:38:03.751647 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:03.751647 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:38:03.751647 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:03.751647 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:38:03.752506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:03.752506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:38:03.752506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:38:03.752506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:38:03.752506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:03.752506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:38:03.752506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:03.752506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:38:03.752506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:03.752506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:38:03.752506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:03.752506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:38:03.752506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:03.752506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:38:03.752506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:38:03.753002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:03.753002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:38:03.753002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:03.753002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:38:03.753002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:38:03.753002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:03.753002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:38:03.753002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:03.753002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:38:03.753449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:03.753449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:38:03.753449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:38:03.753449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:38:03.753449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:38:03.753449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:38:03.753449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:38:03.753449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:38:03.753449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:38:03.753449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:03.753449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:38:03.753772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:03.753772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:38:03.753772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:03.753772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:38:03.753772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:03.753772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:38:03.753772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:03.753772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:38:03.753772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:38:03.753772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:03.753772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:38:03.753772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:03.753772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:38:03.753772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:38:03.754137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:03.754137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:38:03.754137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:03.754137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:38:03.754552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:03.754552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:38:03.754552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:38:03.754552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:38:03.754552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:38:03.754552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:38:03.754552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:38:03.754552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:38:03.754552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:38:03.754552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:38:03.754552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:38:03.754552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:38:03.754552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:38:03.754552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:38:03.754552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:38:03.754552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:38:03.754552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:38:03.755077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:38:03.755077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:38:03.755077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:38:03.755077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:03.755077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:38:03.755077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:03.755077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:38:03.755077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:03.755077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:38:03.755077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:03.755077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:38:03.755077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:03.755077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:38:03.755077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:38:03.755077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:03.755077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:38:03.755077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:03.755077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:38:03.755077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:38:03.755555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:03.755555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:38:03.755555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:03.755555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:38:03.755935 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:38:03.756226 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:38:03.756334 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:38:03.756535 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:38:03.756899 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:38:03.757120 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:38:03.757268 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:03.757834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:38:03.757834 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:03.757834 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:38:03.757834 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:38:03.757834 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:03.757834 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:38:03.757834 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:03.757834 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:38:03.758773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:03.758773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:38:03.758773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:38:03.758773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:38:03.758773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:38:03.758773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:38:03.758773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:38:03.758773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:03.758773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:38:03.758773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:03.758773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:38:03.758773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:03.759094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:38:03.759094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:03.759094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:38:03.759094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:03.759094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:38:03.759094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:38:03.759094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:03.759094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:38:03.759094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:03.759094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:38:03.759094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:38:03.759094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:03.759094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:38:03.759444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:03.759444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:38:03.759764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:03.759764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:38:03.759764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:38:03.759764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:38:03.759764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:38:03.759764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:38:03.759764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:38:03.759764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:38:03.759764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:38:03.759764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:38:03.759764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:38:03.759764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:38:03.759764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:38:03.759764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:38:03.759764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:38:03.759764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:38:03.760278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:38:03.760278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:38:03.760278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:03.760278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:38:03.760278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:03.760278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:38:03.760278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:03.760278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:38:03.760278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:03.760278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:38:03.760278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:03.760278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:38:03.760278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:38:03.760278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:03.760278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:38:03.760278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:03.760278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:38:03.760278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:38:03.760278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:03.760793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:38:03.760793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:03.760793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:38:08.582960 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:38:08.583701 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:38:08.583701 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:38:08.584132 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:38:08.612196 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:38:08.612387 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:08.613986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:38:08.613986 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:08.613986 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:38:08.613986 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:38:08.613986 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:08.613986 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:38:08.613986 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:08.613986 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:38:08.613986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.029 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:08.613986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:38:08.613986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:38:08.613986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:38:08.613986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:38:08.613986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:38:08.613986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:38:08.614601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:08.614601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:38:08.614601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:08.614601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:38:08.614601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:08.614601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:38:08.614601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:08.614601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:38:08.614601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:08.614601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:38:08.614601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:38:08.614601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:08.614601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:38:08.614601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:08.615141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:38:08.615141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:38:08.615141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:08.615141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:38:08.615141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:08.615141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:38:08.615141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:08.615141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:38:08.615141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:38:08.615141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:38:08.615141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:38:08.615141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:38:08.615141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:38:08.615141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:38:08.615765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:38:08.615765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:38:08.615765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:38:08.615765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:38:08.615765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:38:08.615765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:38:08.615765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:38:08.615765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:38:08.615765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:38:08.615765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:38:08.615765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:08.615765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:38:08.615765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:08.615765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:38:08.615765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:08.615765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:38:08.615765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:08.615765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:38:08.615765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:08.615765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:38:08.615765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:38:08.616483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:08.616483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:38:08.616483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:08.616483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:38:08.616483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:38:08.616483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:08.616483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:38:08.616483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:08.616483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:38:08.735702 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:38:08.736473 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:38:08.736790 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:38:08.738721 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:38:08.739051 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:38:08.739219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:08.740119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:38:08.740119 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:08.740119 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:38:08.740119 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:38:08.740119 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:08.740119 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:38:08.740119 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:08.740119 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:38:08.742235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:08.742235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:38:08.742235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:38:08.742235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:38:08.742235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:38:08.742235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:38:08.742235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:38:08.742235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:08.742235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:38:08.742235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:08.742235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:38:08.742235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:08.742760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:38:08.742760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:08.742760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:38:08.742760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:08.742760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:38:08.742760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:38:08.742760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:08.742760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:38:08.742760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:08.742760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:38:08.742760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:38:08.742760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:08.742760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:38:08.743301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:08.743301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:38:08.744036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:08.744036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:38:08.744036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:38:08.744036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:38:08.744036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:38:08.744036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:38:08.744036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:38:08.744036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:38:08.744036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:38:08.744036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:38:08.744036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:38:08.744036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:38:08.744036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:38:08.744036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:38:08.744036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:38:08.744036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:38:08.744817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:38:08.744817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:38:08.744817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:08.744817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:38:08.744817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:08.744817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:38:08.744817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:08.744817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:38:08.744817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:08.744817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:38:08.744817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:08.744817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:38:08.744817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:38:08.744817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:08.744817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:38:08.744817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:08.744817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:38:08.744817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:38:08.744817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:08.745618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:38:08.745618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:08.745618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:38:08.745618 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:38:08.745618 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:38:08.745618 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:38:08.745618 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:38:08.746023 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:38:08.746023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:08.746704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:38:08.746704 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:08.746704 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:38:08.746704 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:38:08.746704 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:08.746704 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:38:08.746704 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:08.746704 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:38:08.747866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:08.747866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:38:08.747866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:38:08.747866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:38:08.747866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:08.747866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:38:08.747866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:08.747866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:38:08.747866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:08.747866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:38:08.747866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:08.747866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:38:08.747866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:08.747866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:38:08.747866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:38:08.748799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:08.748799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:38:08.748799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:08.748799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:38:08.748799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:38:08.748799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:08.748799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:38:08.748799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:08.748799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:38:08.749649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:08.749649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:38:08.749649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:38:08.749649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:38:08.749649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:38:08.749649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:38:08.749649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:38:08.749649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:38:08.749649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:38:08.749649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:38:08.749649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:08.749649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:38:08.750343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:08.750343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:38:08.750343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:08.750343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:38:08.750343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:08.750343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:38:08.750343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:08.750343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:38:08.750343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:38:08.750343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:08.750343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:38:08.750343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:08.750343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:38:08.750343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:38:08.751184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:08.751184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:38:08.751184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:08.751184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:38:08.751477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:08.751477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:38:08.751477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:38:08.751477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:38:08.751477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:38:08.751477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:38:08.751477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:38:08.751477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:38:08.751477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:38:08.751477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:38:08.751477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:38:08.751477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:38:08.751477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:38:08.751477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:38:08.751477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:38:08.751477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:38:08.752286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:38:08.752286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:38:08.752286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:38:08.752286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:38:08.752286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:38:08.752286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:08.752286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:38:08.752286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:08.752286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:38:08.752286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:08.752286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:38:08.752286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:08.752286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:38:08.752286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:08.752286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:38:08.752286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:38:08.752286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:08.752286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:38:08.753175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:08.753175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:38:08.753175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:38:08.753175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:08.753175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:38:08.753175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:08.753175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:38:08.753175 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:38:08.753175 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:38:08.753175 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:38:08.753698 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:38:08.753698 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:38:08.753698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:08.753698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:38:08.753698 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:08.753698 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:38:08.753698 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:38:08.753698 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:08.753698 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:38:08.753698 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:08.753698 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:38:08.754472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:08.754472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:38:08.754472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:38:08.754472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:38:08.754472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:08.754472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:38:08.754472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:08.754472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:38:08.754472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:08.754472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:38:08.754472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:08.754472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:38:08.754472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:08.754472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:38:08.754472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:38:08.755305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:08.755305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:38:08.755305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:08.755305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:38:08.755305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:38:08.755305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:08.755305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:38:08.755305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:08.755305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:38:08.755305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:08.755305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:38:08.755305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:38:08.755305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:38:08.755305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:38:08.755806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:38:08.755806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:38:08.755806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:38:08.755806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:38:08.755806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:08.755806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:38:08.755806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:08.755806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:38:08.755806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:08.755806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:38:08.755806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:08.755806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:38:08.755806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:08.756284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:38:08.756284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:38:08.756284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:08.756284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:38:08.756284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:08.756284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:38:08.756284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:38:08.756284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:08.756284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:38:08.756284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:08.756284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:38:08.756284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:08.756701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:38:08.756701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:38:08.756701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:38:08.756701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:38:08.756701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:38:08.756701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:38:08.756701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:38:08.756701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:38:08.756701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:38:08.756701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:38:08.756701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:38:08.756701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:38:08.756701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:38:08.756701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:38:08.756701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:38:08.756701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:38:08.756701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:38:08.756701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:38:08.756701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:38:08.757277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:08.757277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:38:08.757277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:08.757277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:38:08.757277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:08.757277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:38:08.757277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:08.757277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:38:08.757277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:08.757277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:38:08.757277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:38:08.757277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:08.757277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:38:08.757277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:08.757277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:38:08.757277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:38:08.757277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:08.757277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:38:08.757277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:08.757277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:38:08.759116 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:38:08.759385 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:38:08.759494 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:38:08.759685 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:38:08.762779 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:38:08.762779 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:38:08.762779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:08.762779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:38:08.762779 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:08.762779 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:38:08.762779 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:38:08.762779 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:08.762779 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:38:08.762779 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:08.762779 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:38:08.762779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:08.763662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:38:08.763662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:38:08.763662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:38:08.763662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:38:08.763662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:38:08.763662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:38:08.763662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:08.763662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:38:08.763662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:08.763662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:38:08.763662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:08.763662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:38:08.763662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:08.763662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:38:08.764107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:08.764107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:38:08.764107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:38:08.764107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:08.764107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:38:08.764107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:08.764107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:38:08.764107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:38:08.764107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:08.764107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:38:08.764107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:08.764107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:38:08.764450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:08.764450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:38:08.764450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:38:08.764450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:38:08.764450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:38:08.764450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:38:08.764450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:38:08.764450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:38:08.764450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:38:08.764450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:38:08.764450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:38:08.764450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:38:08.764450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:38:08.764450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:38:08.764450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:38:08.764450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:38:08.764929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:38:08.764929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:38:08.764929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:08.764929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:38:08.764929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:08.764929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:38:08.764929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:08.764929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:38:08.764929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:08.764929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:38:08.764929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:08.764929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:38:08.764929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:38:08.764929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:08.764929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:38:08.764929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:08.764929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:38:08.764929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:38:08.764929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:08.765457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:38:08.765457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:08.765457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:38:13.584401 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:38:13.585331 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:38:13.585425 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:38:13.586381 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:38:13.620494 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:38:13.620791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:13.621435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:38:13.621435 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:13.621435 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:38:13.621435 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:38:13.621435 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:13.621435 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:38:13.621435 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:13.621435 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:38:13.622715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:13.622715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:38:13.622715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:38:13.622715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:38:13.622715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:38:13.622715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:38:13.622715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:38:13.622715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:13.622715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:38:13.622715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:13.622715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:38:13.622715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:13.623555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:38:13.623555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:13.623555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:38:13.623555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:13.623555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:38:13.623555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:38:13.623555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:13.623555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:38:13.623555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:13.623555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:38:13.623555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:38:13.623555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:13.623555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:38:13.624006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:13.624006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:38:13.624006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:13.624006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:38:13.624006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:38:13.624006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:38:13.624006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:38:13.624006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:38:13.624006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:38:13.624006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:38:13.624006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:38:13.624006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:38:13.624006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:38:13.624006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:38:13.624006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:38:13.624602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:38:13.624602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:38:13.624602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:38:13.624602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:38:13.624602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:38:13.624602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:13.624602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:38:13.624602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:13.624602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:38:13.624602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:13.624602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:38:13.624602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:13.624602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:38:13.624602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:13.624602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:38:13.624602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:38:13.624602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:13.624602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:38:13.624602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:13.625260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:38:13.625260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:38:13.625260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:13.625260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:38:13.625260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:13.625260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:38:13.736288 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:38:13.736766 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:38:13.736868 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:38:13.737798 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:38:13.737987 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:38:13.738106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:13.738672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:38:13.738672 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:13.738672 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:38:13.738672 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:38:13.738672 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:13.738672 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:38:13.738672 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:13.738672 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:38:13.739496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:13.739496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:38:13.739496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:38:13.739496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:38:13.739496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:38:13.739496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:38:13.739496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:38:13.739496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:13.739496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:38:13.739496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:13.739496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:38:13.739496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:13.740333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:38:13.740333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:13.740333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:38:13.740333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:13.740333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:38:13.740333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:38:13.740333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:13.740333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:38:13.740333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:13.740333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:38:13.740333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:38:13.740333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:13.740333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:38:13.741348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:13.741348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:38:13.741348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:13.741348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:38:13.741348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:38:13.741348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:38:13.741348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:38:13.741348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:38:13.741348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:38:13.741348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:38:13.741348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:38:13.741348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:38:13.741348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:38:13.741348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:38:13.741348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:38:13.742071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:38:13.742071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:38:13.742071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:38:13.742071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:38:13.742071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:38:13.742071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:13.742071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:38:13.742071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:13.742071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:38:13.742071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:13.742071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:38:13.742071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:13.742071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:38:13.742071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:13.742071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:38:13.742071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:38:13.742071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:13.742071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:38:13.742071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:13.742547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:38:13.742547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:38:13.742547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:13.742547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:38:13.742547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:13.742547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:38:13.744592 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:38:13.744834 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:38:13.744931 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:38:13.745129 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:38:13.745638 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:38:13.745976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:13.746725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:38:13.746725 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:13.746725 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:38:13.746725 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:38:13.746725 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:13.746725 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:38:13.746725 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:13.746725 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:38:13.748234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:13.748234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:38:13.748234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:38:13.748234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:38:13.748234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:13.748234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:38:13.748234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:13.748234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:38:13.748234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:13.748234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:38:13.748234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:13.748234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:38:13.748234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:13.748234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:38:13.748234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:38:13.748833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:13.748833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:38:13.748833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:13.748833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:38:13.748833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:38:13.748833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:13.748833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:38:13.748833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:13.748833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:38:13.749030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:13.749030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:38:13.749030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:38:13.749030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:38:13.749030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:38:13.749030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:38:13.749030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:38:13.749030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:38:13.749030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:38:13.749030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:38:13.749030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:13.749030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:38:13.749279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:13.749279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:38:13.749279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:13.749279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:38:13.749279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:13.749279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:38:13.749279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:13.749279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:38:13.749279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:38:13.749279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:13.749279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:38:13.749279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:13.749279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:38:13.749279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:38:13.749546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:13.749546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:38:13.749546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:13.749546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:38:13.749799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:13.749799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:38:13.749799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:38:13.749799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:38:13.749799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:38:13.749799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:38:13.749799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:38:13.749799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:38:13.749799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:38:13.749799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:38:13.749799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:38:13.749799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:38:13.749799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:38:13.749799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:38:13.749799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:38:13.749799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:38:13.750374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:38:13.750374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:38:13.750374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:38:13.750374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:38:13.750374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:38:13.750374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:13.750374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:38:13.750374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:13.750374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:38:13.750374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:13.750374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:38:13.750374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:13.750374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:38:13.750374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:13.750374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:38:13.750374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:38:13.750374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:13.750374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:38:13.750793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:13.750793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:38:13.750793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:38:13.750793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:13.750793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:38:13.750793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:13.750793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:38:13.752923 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:38:13.752923 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:38:13.752923 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:38:13.752923 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:38:13.752923 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:38:13.753121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:13.753515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:38:13.753515 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:13.753515 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:38:13.753515 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:38:13.753515 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:13.753515 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:38:13.753515 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:13.753515 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:38:13.754144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:13.754144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:38:13.754144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:38:13.754144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:38:13.754144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:13.754144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:38:13.754144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:13.754144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:38:13.754144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:13.754144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:38:13.754144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:13.754144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:38:13.754144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:13.754144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:38:13.754144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:38:13.754452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:13.754452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:38:13.754452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:13.754452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:38:13.754452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:38:13.754452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:13.754452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:38:13.754452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:13.754452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:38:13.754652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:13.754652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:38:13.754652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:38:13.754652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:38:13.754652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:38:13.754652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:38:13.754652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:38:13.754652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:38:13.754652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:38:13.754652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:13.754652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:38:13.754920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:13.754920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:38:13.754920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:13.754920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:38:13.754920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:13.754920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:38:13.754920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:13.754920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:38:13.754920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:38:13.754920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:13.754920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:38:13.754920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:13.754920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:38:13.754920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:38:13.755198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:13.755198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:38:13.755198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:13.755198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:38:13.755431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:13.755431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:38:13.755431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:38:13.755431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:38:13.755431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:38:13.755431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:38:13.755431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:38:13.755431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:38:13.755431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:38:13.755431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:38:13.755431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:38:13.755431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:38:13.755431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:38:13.755431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:38:13.755431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:38:13.755431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:38:13.755431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:38:13.755827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:38:13.755827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:38:13.755827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:38:13.755827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:13.755827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:38:13.755827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:13.755827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:38:13.755827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:13.755827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:38:13.755827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:13.755827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:38:13.755827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:13.755827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:38:13.755827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:38:13.755827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:13.755827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:38:13.755827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:13.755827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:38:13.755827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:38:13.756206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:13.756206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:38:13.756206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:13.756206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:38:13.759495 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:38:13.759691 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:38:13.759795 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:38:13.759959 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:38:13.760359 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:38:13.760583 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:38:13.760705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:13.761207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:38:13.761207 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:13.761207 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:38:13.761207 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:38:13.761207 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:13.761207 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:38:13.761207 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:13.761207 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:38:13.761970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:13.761970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:38:13.761970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:38:13.761970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:38:13.761970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:38:13.761970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:38:13.761970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:38:13.761970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:13.761970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:38:13.761970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:13.761970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:38:13.761970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:13.762565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:38:13.762565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:13.762565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:38:13.762565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:13.762565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:38:13.762565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:38:13.762565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:13.762565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:38:13.762565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:13.762565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:38:13.762565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:38:13.762565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:13.762565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:38:13.763165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:13.763165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:38:13.763165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:13.763165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:38:13.763165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:38:13.763165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:38:13.763165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:38:13.763165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:38:13.763165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:38:13.763165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:38:13.763165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:38:13.763165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:38:13.763165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:38:13.763165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:38:13.763165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:38:13.763881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:38:13.763881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:38:13.763881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:38:13.763881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:38:13.763881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:38:13.763881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:13.763881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:38:13.763881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:13.763881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:38:13.763881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:13.763881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:38:13.763881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:13.763881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:38:13.763881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:13.763881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:38:13.763881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:38:13.763881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:13.763881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:38:13.763881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:13.764665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:38:13.764665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:38:13.764665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:13.764665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:38:13.764665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:13.764665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:38:18.585150 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:38:18.586054 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:38:18.586054 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:38:18.586342 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:38:18.620256 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:38:18.620427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:18.620974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:38:18.620974 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:18.620974 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:38:18.620974 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:38:18.620974 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:18.620974 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:38:18.620974 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:18.620974 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:38:18.621877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:18.621877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:38:18.621877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:38:18.621877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:38:18.621877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:38:18.621877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:38:18.621877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:38:18.621877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:18.621877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:38:18.621877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:18.621877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:38:18.621877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:18.622327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:38:18.622327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:18.622327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:38:18.622327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:18.622327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:38:18.622327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:38:18.622327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:18.622327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:38:18.622327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:18.622327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:38:18.622327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:38:18.622327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:18.622327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:38:18.622707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:18.622707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:38:18.622707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:18.622707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:38:18.622707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:38:18.622707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:38:18.622707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:38:18.622707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:38:18.622707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:38:18.622707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:38:18.622707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:38:18.622707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:38:18.622707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:38:18.622707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:38:18.622707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:38:18.623072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:38:18.623072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:38:18.623072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:38:18.623072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:38:18.623072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:38:18.623072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:18.623072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:38:18.623072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:18.623072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:38:18.623072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:18.623072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:38:18.623072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:18.623072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:38:18.623072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:18.623072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:38:18.623072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:38:18.623072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:18.623072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:38:18.623072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:18.623514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:38:18.623514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:38:18.623514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:18.623514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:38:18.623514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:18.623514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:38:18.736921 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:38:18.737443 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:38:18.737658 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:38:18.738651 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:38:18.738938 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:38:18.739115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:18.739837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:38:18.739837 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:18.739837 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:38:18.739837 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:38:18.739837 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:18.739837 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:38:18.739837 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:18.739837 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:38:18.741296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:18.741296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:38:18.741296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:38:18.741296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:38:18.741296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:38:18.741296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:38:18.741296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:38:18.741296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:18.741296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:38:18.741296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:18.741296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:38:18.741296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:18.741802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:38:18.741802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:18.741802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:38:18.741802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:18.741802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:38:18.741802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:38:18.741802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:18.741802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:38:18.741802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:18.741802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:38:18.741802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:38:18.741802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:18.741802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:38:18.742334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:18.742334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:38:18.743102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:18.743102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:38:18.743102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:38:18.743102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:38:18.743102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:38:18.743102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:38:18.743102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:38:18.743102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:38:18.743102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:38:18.743102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:38:18.743102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:38:18.743102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:38:18.743102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:38:18.743102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:38:18.743102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:38:18.743102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:38:18.743670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:38:18.743670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:38:18.743670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:18.743670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:38:18.743670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:18.743670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:38:18.743670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:18.743670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:38:18.743670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:18.743670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:38:18.743670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:18.743670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:38:18.743670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:38:18.743670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:18.743670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:38:18.743670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:18.743670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:38:18.743670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:38:18.743670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:18.744256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:38:18.744256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:18.744256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:38:18.745075 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:38:18.745452 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:38:18.745577 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:38:18.745773 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:38:18.746523 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:38:18.746941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:18.748125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:38:18.748125 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:18.748125 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:38:18.748125 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:38:18.748125 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:18.748125 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:38:18.748125 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:18.748125 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:38:18.748986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:18.748986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:38:18.748986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:38:18.748986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:38:18.748986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:18.748986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:38:18.748986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:18.748986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:38:18.748986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:18.748986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:38:18.748986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:18.748986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:38:18.748986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:18.748986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:38:18.748986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:38:18.749646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:18.749646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:38:18.749646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:18.749646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:38:18.749646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:38:18.749646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:18.749646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:38:18.749646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:18.749646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:38:18.750056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:18.750056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:38:18.750056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:38:18.750056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:38:18.750056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:38:18.750056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:38:18.750056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:38:18.750056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:38:18.750056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:38:18.750056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:38:18.750056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:18.750056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:38:18.750472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:18.750472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:38:18.750472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:18.750472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:38:18.750472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:18.750472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:38:18.750472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:18.750472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:38:18.750472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:38:18.750472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:18.750472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:38:18.750472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:18.750472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:38:18.750472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:38:18.750952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:18.750952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:38:18.750952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:18.750952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:38:18.751179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:18.751179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:38:18.751179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:38:18.751179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:38:18.751179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:38:18.751179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:38:18.751179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:38:18.751179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:38:18.751179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:38:18.751179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:38:18.751179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:38:18.751179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:38:18.751179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:38:18.751179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:38:18.751179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:38:18.751179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:38:18.751869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:38:18.751869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:38:18.751869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:38:18.751869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:38:18.751869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:38:18.751869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:18.751869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:38:18.751869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:18.751869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:38:18.751869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:18.751869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:38:18.751869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:18.751869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:38:18.751869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:18.751869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:38:18.751869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:38:18.751869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:18.751869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:38:18.752374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:18.752374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:38:18.752374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:38:18.752374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:18.752374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:38:18.752374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:18.752374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:38:18.752656 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:38:18.752791 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:38:18.752953 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:38:18.753093 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:38:18.753676 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:38:18.754638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:18.755454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:38:18.755454 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:18.755454 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:38:18.755454 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:38:18.755454 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:18.755454 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:38:18.755454 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:18.755454 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:38:18.756404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:18.756404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:38:18.756404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:38:18.756404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:38:18.756404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:18.756404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:38:18.756404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:18.756404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:38:18.756404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:18.756404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:38:18.756404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:18.756404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:38:18.756404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:18.756404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:38:18.756404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:38:18.758701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:18.758701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:38:18.758701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:18.758701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:38:18.758701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:38:18.758701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:18.758701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:38:18.758701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:18.758701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:38:18.758701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:18.758701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:38:18.758701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:38:18.758701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:38:18.758701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:38:18.759194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:38:18.759194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:38:18.759194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:38:18.759194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:38:18.759194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:18.759194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:38:18.759194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:18.759194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:38:18.759194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:18.759194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:38:18.759194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:18.759194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:38:18.759194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:18.759661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:38:18.759661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:38:18.759661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:18.759661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:38:18.759661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:18.759661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:38:18.759661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:38:18.759661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:18.759661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:38:18.759661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:18.759661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:38:18.759661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:18.760548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:38:18.760548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:38:18.760548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:38:18.760548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:38:18.760548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:38:18.760548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:38:18.760548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:38:18.760548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:38:18.760548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:38:18.760548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:38:18.760548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:38:18.760548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:38:18.760548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:38:18.760548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:38:18.760548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:38:18.760548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:38:18.760548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:38:18.760548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:38:18.760548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:38:18.761112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:18.761112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:38:18.761112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:18.761112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:38:18.761112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:18.761112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:38:18.761112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:18.761112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:38:18.761112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:18.761112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:38:18.761112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:38:18.761112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:18.761112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:38:18.761112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:18.761112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:38:18.761112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:38:18.761112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:18.761112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:38:18.761112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:18.761112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:38:18.761637 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:38:18.761637 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:38:18.761637 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:38:18.761637 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:38:18.761983 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:38:18.762385 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:38:18.762641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:18.765141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:38:18.765141 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:18.765141 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:38:18.765141 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:38:18.765141 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:18.765141 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:38:18.765141 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:18.765141 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:38:18.766127 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:18.766127 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:38:18.766127 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:38:18.766127 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:38:18.766127 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:38:18.766127 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:38:18.766127 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:38:18.766127 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:18.766127 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:38:18.766127 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:18.766127 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:38:18.766127 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:18.766554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:38:18.766554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:18.766554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:38:18.766554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:18.766554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:38:18.766554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:38:18.766554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:18.766554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:38:18.766554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:18.766554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:38:18.766554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:38:18.766554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:18.766554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:38:18.766848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:18.766848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:38:18.767193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:18.767193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:38:18.767193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:38:18.767193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:38:18.767193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:38:18.767193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:38:18.767193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:38:18.767193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:38:18.767193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:38:18.767193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:38:18.767193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:38:18.767193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:38:18.767193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:38:18.767193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:38:18.767193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:38:18.767193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:38:18.767671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:38:18.767671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:38:18.767671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:18.767671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:38:18.767671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:18.767671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:38:18.767671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:18.767671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:38:18.767671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:18.767671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:38:18.767671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:18.767671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:38:18.767671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:38:18.767671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:18.767671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:38:18.767671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:18.767671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:38:18.767671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:38:18.767671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:18.768213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:38:18.768213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:18.768213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:38:23.587513 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:38:23.588373 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:38:23.588373 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:38:23.589149 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:38:23.627655 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:38:23.627655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:23.628054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:38:23.628054 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:23.628054 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:38:23.628054 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:38:23.628054 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:23.628054 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:38:23.628054 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:23.628054 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:38:23.628907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:23.628907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:38:23.628907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:38:23.628907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:38:23.628907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:38:23.628907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:38:23.628907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:38:23.628907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:23.628907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:38:23.628907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:23.628907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:38:23.628907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:23.629484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:38:23.629484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:23.629484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:38:23.629484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:23.629484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:38:23.629484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:38:23.629484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:23.629484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:38:23.629484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:23.629484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:38:23.629484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:38:23.629484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:23.629484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:38:23.629998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:23.629998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:38:23.629998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:23.629998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:38:23.629998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:38:23.629998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:38:23.629998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:38:23.629998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:38:23.629998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:38:23.629998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:38:23.629998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:38:23.629998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:38:23.629998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:38:23.629998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:38:23.629998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:38:23.630530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:38:23.630530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:38:23.630530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:38:23.630530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:38:23.630530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:38:23.630530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:23.630530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:38:23.630530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:23.630530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:38:23.630530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:23.630530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:38:23.630530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:23.630530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:38:23.630530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:23.630530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:38:23.630530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:38:23.630530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:23.630530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:38:23.630530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:23.631197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:38:23.631197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:38:23.631197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:23.631197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:38:23.631197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:23.631197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:38:23.737668 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:38:23.738360 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:38:23.738567 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:38:23.739653 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:38:23.739941 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:38:23.740124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:23.740773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:38:23.740773 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:23.740773 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:38:23.740773 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:38:23.740773 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:23.740773 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:38:23.740773 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:23.740773 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:38:23.742429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:23.742429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:38:23.742429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:38:23.742429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:38:23.742429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:38:23.742429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:38:23.742429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:38:23.742429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:23.742429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:38:23.742429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:23.742429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:38:23.742429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:23.743203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:38:23.743203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:23.743203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:38:23.743203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:23.743203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:38:23.743203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:38:23.743203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:23.743203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:38:23.743203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:23.743203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:38:23.743203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:38:23.743203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:23.743203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:38:23.744206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:23.744206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:38:23.744388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:23.744388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:38:23.744388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:38:23.744388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:38:23.744388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:38:23.744388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:38:23.744388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:38:23.744388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:38:23.744388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:38:23.744388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:38:23.744388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:38:23.744388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:38:23.744388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:38:23.744388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:38:23.744388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:38:23.744388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:38:23.745740 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:38:23.745740 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:38:23.745740 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:23.745740 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:38:23.745740 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:23.745740 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:38:23.745740 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:23.745740 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:38:23.745740 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:23.745740 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:38:23.745740 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:23.745740 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:38:23.745740 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:38:23.745740 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:23.745740 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:38:23.745740 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:23.745740 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:38:23.745740 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:38:23.745740 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:23.747175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:38:23.747175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:23.747175 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:38:23.747175 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:38:23.747175 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:38:23.747175 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:38:23.747175 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:38:23.747175 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:38:23.747599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:23.747599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:38:23.747599 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:23.747599 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:38:23.747599 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:38:23.747599 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:23.747599 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:38:23.747599 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:23.747599 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:38:23.748524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:23.748524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:38:23.748524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:38:23.748524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:38:23.748524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:23.748524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:38:23.748524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:23.748524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:38:23.748524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:23.748524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:38:23.748524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:23.748524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:38:23.748524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:23.748524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:38:23.748524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:38:23.750392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:23.750392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:38:23.750392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:23.750392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:38:23.750392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:38:23.750392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:23.750392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:38:23.750392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:23.750392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:38:23.750392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:23.750392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:38:23.750392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:38:23.750392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:38:23.750392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:38:23.751325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:38:23.751325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:38:23.751325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:38:23.751325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:38:23.751325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:38:23.751325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:23.751325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:38:23.751325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:23.751325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:38:23.751325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:23.751325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:38:23.751325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:23.751325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:38:23.751669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:23.751669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:38:23.751669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:38:23.751669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:23.751669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:38:23.751669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:23.751669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:38:23.751669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:38:23.751669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:23.751669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:38:23.751669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:23.751669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:38:23.752014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:23.752014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:38:23.752014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:38:23.752014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:38:23.752014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:38:23.752014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:38:23.752014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:38:23.752014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:38:23.752014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:38:23.752014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:38:23.752014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:38:23.752014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:38:23.752014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:38:23.752014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:38:23.752014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:38:23.752014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:38:23.752435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:38:23.752435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:38:23.752435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:38:23.752435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:38:23.752435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:38:23.752435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:23.752435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:38:23.752435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:23.752435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:38:23.752435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:23.752435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:38:23.752435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:23.752435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:38:23.752435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:23.752435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:38:23.752435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:38:23.752435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:23.752435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:38:23.753627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:23.753627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:38:23.753627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:38:23.753627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:23.753627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:38:23.753627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:23.753627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:38:23.753627 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:38:23.753627 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:38:23.753627 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:38:23.753994 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:38:23.753994 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:38:23.754598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:23.755171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:38:23.755171 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:23.755171 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:38:23.755171 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:38:23.755171 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:23.755171 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:38:23.755171 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:23.755171 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:38:23.756071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:23.756071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:38:23.756071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:38:23.756071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:38:23.756071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:23.756071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:38:23.756071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:23.756071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:38:23.756071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:23.756071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:38:23.756071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:23.756071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:38:23.756071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:23.756071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:38:23.756071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:38:23.756442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:23.756442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:38:23.756442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:23.756442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:38:23.756442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:38:23.756442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:23.756442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:38:23.756442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:23.756442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:38:23.757130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:23.757130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:38:23.757130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:38:23.757130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:38:23.757130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:38:23.757130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:38:23.757130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:38:23.757130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:38:23.757130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:38:23.757130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:23.757130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:38:23.757546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:23.757546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:38:23.757546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:23.757546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:38:23.757546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:23.757546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:38:23.757546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:23.757546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:38:23.757546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:38:23.757546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:23.757546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:38:23.757546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:23.757546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:38:23.757546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:38:23.758459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:23.758459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:38:23.758459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:23.758459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:38:23.758459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:23.758459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:38:23.758459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:38:23.758459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:38:23.758459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:38:23.758459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:38:23.758459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:38:23.758459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:38:23.758459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:38:23.758459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:38:23.758900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:38:23.758900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:38:23.758900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:38:23.758900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:38:23.758900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:38:23.758900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:38:23.758900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:38:23.758900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:38:23.758900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:38:23.758900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:38:23.758900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:23.758900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:38:23.758900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:23.758900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:38:23.758900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:23.758900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:38:23.758900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:23.758900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:38:23.758900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:23.760397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:38:23.760397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:38:23.760397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:23.760397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:38:23.760397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:23.760397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:38:23.760397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:38:23.760397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:23.760397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:38:23.760397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:23.760397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:38:23.760397 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:38:23.761462 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:38:23.761462 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:38:23.761462 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:38:23.761462 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:38:23.761902 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:38:23.761902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:23.762596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:38:23.762596 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:23.762596 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:38:23.762596 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:38:23.762596 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:23.762596 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:38:23.762596 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:23.762596 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:38:23.763483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:23.763483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:38:23.763483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:38:23.763483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:38:23.763483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:38:23.763483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:38:23.763483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:38:23.763483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:23.763483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:38:23.763483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:23.763483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:38:23.763483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:23.763864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:38:23.763864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:23.763864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:38:23.763864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:23.763864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:38:23.763864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:38:23.763864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:23.763864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:38:23.763864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:23.763864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:38:23.763864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:38:23.763864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:23.763864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:38:23.764226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:23.764226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:38:23.764473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:23.764473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:38:23.764473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:38:23.764473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:38:23.764473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:38:23.764473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:38:23.764473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:38:23.764473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:38:23.764473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:38:23.764473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:38:23.764473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:38:23.764473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:38:23.764473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:38:23.764473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:38:23.764473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:38:23.764473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:38:23.765467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:38:23.765467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:38:23.765467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:23.765467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:38:23.765467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:23.765467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:38:23.765467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:23.765467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:38:23.765467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:23.765467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:38:23.765467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:23.765467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:38:23.765467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:38:23.765467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:23.765467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:38:23.765467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:23.765467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:38:23.765467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:38:23.765467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:23.766736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:38:23.766736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:23.766736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:38:28.588267 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:38:28.588668 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:38:28.589899 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:38:28.589899 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:38:28.628646 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:38:28.629016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:28.630011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:38:28.630011 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:28.630011 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:38:28.630011 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:38:28.630011 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:28.630011 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:38:28.630011 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:28.630011 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:38:28.630922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:28.630922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:38:28.630922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:38:28.630922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:38:28.630922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:38:28.630922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:38:28.630922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:38:28.630922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:28.630922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:38:28.630922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:28.630922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:38:28.630922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:28.631385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:38:28.631385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:28.631385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:38:28.631385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:28.631385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:38:28.631385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:38:28.631385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:28.631385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:38:28.631385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:28.631385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:38:28.631385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:38:28.631385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:28.631385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:38:28.631851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:28.631851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:38:28.631851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:28.631851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:38:28.631851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:38:28.631851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:38:28.631851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:38:28.631851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:38:28.631851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:38:28.631851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:38:28.631851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:38:28.631851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:38:28.631851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:38:28.631851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:38:28.631851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:38:28.632454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:38:28.632454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:38:28.632454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:38:28.632454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:38:28.632454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:38:28.632454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:28.632454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:38:28.632454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:28.632454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:38:28.632454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:28.632454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:38:28.632454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:28.632454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:38:28.632454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:28.632454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:38:28.632454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:38:28.632454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:28.632454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:38:28.632454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:28.633095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:38:28.633095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:38:28.633095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:28.633095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:38:28.633095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:28.633095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:38:28.738675 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:38:28.739356 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:38:28.739526 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:38:28.740962 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:38:28.741297 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:38:28.741491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:28.742377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:38:28.742377 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:28.742377 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:38:28.742377 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:38:28.742377 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:28.742377 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:38:28.742377 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:28.742377 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:38:28.743707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:28.743707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:38:28.743707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:38:28.743707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:38:28.743707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:38:28.743707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:38:28.743707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:38:28.743707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:28.743707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:38:28.743707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:28.743707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:38:28.743707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:28.744129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:38:28.744129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:28.744129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:38:28.744129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:28.744129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:38:28.744129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:38:28.744129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:28.744129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:38:28.744129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:28.744129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:38:28.744129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:38:28.744129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:28.744129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:38:28.744570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:28.744570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:38:28.744999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:28.744999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:38:28.744999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:38:28.744999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:38:28.744999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:38:28.744999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:38:28.744999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:38:28.744999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:38:28.744999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:38:28.744999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:38:28.744999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:38:28.744999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:38:28.744999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:38:28.744999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:38:28.744999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:38:28.744999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:38:28.745840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:38:28.745840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:38:28.745840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:28.745840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:38:28.745840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:28.745840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:38:28.745840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:28.745840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:38:28.745840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:28.745840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:38:28.745840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:28.745840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:38:28.745840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:38:28.745840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:28.745840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:38:28.745840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:28.745840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:38:28.745840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:38:28.745840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:28.746709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:38:28.746709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:28.746709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:38:28.746709 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:38:28.747118 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:38:28.747210 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:38:28.747306 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:38:28.747881 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:38:28.748220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:28.748998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:38:28.748998 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:28.748998 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:38:28.748998 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:38:28.748998 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:28.748998 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:38:28.748998 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:28.748998 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:38:28.749806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:28.749806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:38:28.749806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:38:28.749806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:38:28.749806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:28.749806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:38:28.749806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:28.749806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:38:28.749806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:28.749806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:38:28.749806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:28.749806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:38:28.749806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:28.749806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:38:28.749806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:38:28.750896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:28.750896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:38:28.750896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:28.750896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:38:28.750896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:38:28.750896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:28.750896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:38:28.750896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:28.750896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:38:28.750896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:28.750896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:38:28.750896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:38:28.750896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:38:28.750896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:38:28.751781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:38:28.751781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:38:28.751781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:38:28.751781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:38:28.751781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:38:28.751781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:28.751781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:38:28.751781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:28.751781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:38:28.751781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:28.751781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:38:28.751781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:28.751781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:38:28.752505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:28.752505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:38:28.752505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:38:28.752505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:28.752505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:38:28.752505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:28.752505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:38:28.752505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:38:28.752505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:28.752505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:38:28.752505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:28.752505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:38:28.753177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:28.753177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:38:28.753177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:38:28.753177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:38:28.753177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:38:28.753177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:38:28.753177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:38:28.753177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:38:28.753177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:38:28.753177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:38:28.753177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:38:28.753177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:38:28.753177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:38:28.753177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:38:28.753177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:38:28.753177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:38:28.754275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:38:28.754275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:38:28.754275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:38:28.754275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:38:28.754275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:38:28.754275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:28.754275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:38:28.754275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:28.754275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:38:28.754275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:28.754275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:38:28.754275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:28.754275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:38:28.754275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:28.754275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:38:28.754275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:38:28.754275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:28.754275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:38:28.755155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:28.755155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:38:28.755155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:38:28.755155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:28.755155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:38:28.755155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:28.755155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:38:28.755155 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:38:28.755155 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:38:28.755155 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:38:28.755654 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:38:28.755822 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:38:28.756255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:28.757012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:38:28.757012 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:28.757012 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:38:28.757012 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:38:28.757012 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:28.757012 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:38:28.757012 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:28.757012 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:38:28.758008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:28.758008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:38:28.758008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:38:28.758008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:38:28.758008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:28.758008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:38:28.758008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:28.758008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:38:28.758008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:28.758008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:38:28.758008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:28.758008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:38:28.758008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:28.758008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:38:28.758008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:38:28.758628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:28.758628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:38:28.758628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:28.758628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:38:28.758628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:38:28.758628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:28.758628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:38:28.758628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:28.758628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:38:28.759322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:28.759322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:38:28.759322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:38:28.759322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:38:28.759322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:38:28.759322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:38:28.759322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:38:28.759322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:38:28.759322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:38:28.759322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:28.759322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:38:28.759858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:28.759858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:38:28.759858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:28.759858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:38:28.759858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:28.759858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:38:28.759858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:28.759858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:38:28.759858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:38:28.759858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:28.759858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:38:28.759858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:28.759858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:38:28.759858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:38:28.760453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:28.760453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:38:28.760453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:28.760453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:38:28.760996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:28.760996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:38:28.760996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:38:28.760996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:38:28.760996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:38:28.760996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:38:28.760996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:38:28.760996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:38:28.760996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:38:28.760996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:38:28.760996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:38:28.760996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:38:28.760996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:38:28.760996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:38:28.760996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:38:28.760996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:38:28.760996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:38:28.761653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:38:28.761653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:38:28.761653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:38:28.761653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:28.761653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:38:28.761653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:28.761653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:38:28.761653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:28.761653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:38:28.761653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:28.761653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:38:28.761653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:28.761653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:38:28.761653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:38:28.761653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:28.761653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:38:28.761653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:28.761653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:38:28.761653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:38:28.762368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:28.762368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:38:28.762368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:28.762368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:38:28.762368 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:38:28.762368 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:38:28.762368 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:38:28.762368 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:38:28.762671 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:38:28.762794 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:38:28.762909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:28.763520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:38:28.763520 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:28.763520 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:38:28.763520 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:38:28.763520 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:28.763520 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:38:28.763520 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:28.763520 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:38:28.764620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:28.764620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:38:28.764620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:38:28.764620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:38:28.764620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:38:28.764620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:38:28.764620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:38:28.764620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:28.764620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:38:28.764620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:28.764620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:38:28.764620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:28.765206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:38:28.765206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:28.765206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:38:28.765206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:28.765206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:38:28.765206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:38:28.765206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:28.765206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:38:28.765206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:28.765206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:38:28.765206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:38:28.765206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:28.765206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:38:28.765804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:28.765804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:38:28.766123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:28.766123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:38:28.766123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:38:28.766123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:38:28.766123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:38:28.766123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:38:28.766123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:38:28.766123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:38:28.766123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:38:28.766123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:38:28.766123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:38:28.766123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:38:28.766123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:38:28.766123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:38:28.766123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:38:28.766123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:38:28.766648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:38:28.766648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:38:28.766648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:28.766648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:38:28.766648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:28.766648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:38:28.766648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:28.766648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:38:28.766648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:28.766648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:38:28.766648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:28.766648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:38:28.766648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:38:28.766648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:28.766648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:38:28.766648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:28.766648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:38:28.766648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:38:28.766648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:28.767210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:38:28.767210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:28.767210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:38:33.589205 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:38:33.589946 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:38:33.590237 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:38:33.591613 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:38:33.628581 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:38:33.628931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:33.629833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:38:33.629833 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:33.629833 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:38:33.629833 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:38:33.629833 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:33.629833 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:38:33.629833 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:33.629833 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:38:33.630776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:33.630776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:38:33.630776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:38:33.630776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:38:33.630776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:38:33.630776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:38:33.630776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:38:33.630776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:33.630776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:38:33.630776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:33.630776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:38:33.630776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:33.631171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:38:33.631171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:33.631171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:38:33.631171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:33.631171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:38:33.631171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:38:33.631171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:33.631171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:38:33.631171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:33.631171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:38:33.631171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:38:33.631171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:33.631171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:38:33.631549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:33.631549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:38:33.631549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:33.631549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:38:33.631549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:38:33.631549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:38:33.631549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:38:33.631549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:38:33.631549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:38:33.631549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:38:33.631549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:38:33.631549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:38:33.631549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:38:33.631549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:38:33.631549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:38:33.631990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:38:33.631990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:38:33.631990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:38:33.631990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:38:33.631990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:38:33.631990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:33.631990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:38:33.631990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:33.631990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:38:33.631990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:33.631990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:38:33.631990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:33.631990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:38:33.631990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:33.631990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:38:33.631990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:38:33.631990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:33.631990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:38:33.631990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:33.632481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:38:33.632481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:38:33.632481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:33.632481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:38:33.632481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:33.632481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:38:33.739664 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:38:33.740639 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:38:33.740953 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:38:33.742443 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:38:33.743099 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:38:33.743316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:33.744134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:38:33.744134 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:33.744134 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:38:33.744134 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:38:33.744134 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:33.744134 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:38:33.744134 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:33.744134 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:38:33.745782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:33.745782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:38:33.745782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:38:33.745782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:38:33.745782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:38:33.745782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:38:33.745782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:38:33.745782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:33.745782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:38:33.745782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:33.745782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:38:33.745782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:33.746415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:38:33.746415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:33.746415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:38:33.746415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:33.746415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:38:33.746415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:38:33.746415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:33.746415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:38:33.746415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:33.746415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:38:33.746415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:38:33.746415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:33.746415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:38:33.747073 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:33.747073 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:38:33.747583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:33.747583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:38:33.747583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:38:33.747583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:38:33.747583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:38:33.747583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:38:33.747583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:38:33.747583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:38:33.747583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:38:33.747583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:38:33.747583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:38:33.747583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:38:33.747583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:38:33.747583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:38:33.747583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:38:33.747583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:38:33.748355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:38:33.748355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:38:33.748355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:33.748355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:38:33.748355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:33.748355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:38:33.748355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:33.748355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:38:33.748355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:33.748355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:38:33.748355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:33.748355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:38:33.748355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:38:33.748355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:33.748355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:38:33.748355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:33.748355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:38:33.748355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:38:33.748355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:33.749448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:38:33.749448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:33.749448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:38:33.749448 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:38:33.749448 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:38:33.749448 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:38:33.749448 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:38:33.749448 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:38:33.749946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:33.750109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:38:33.750109 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:33.750109 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:38:33.750109 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:38:33.750109 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:33.750109 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:38:33.750109 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:33.750109 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:38:33.751016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:33.751016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:38:33.751016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:38:33.751016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:38:33.751016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:33.751016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:38:33.751016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:33.751016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:38:33.751016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:33.751016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:38:33.751016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:33.751016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:38:33.751016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:33.751016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:38:33.751016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:38:33.751794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:33.751794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:38:33.751794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:33.751794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:38:33.751794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:38:33.751794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:33.751794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:38:33.751794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:33.751794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:38:33.752092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:33.752092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:38:33.752092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:38:33.752092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:38:33.752092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:38:33.752092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:38:33.752092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:38:33.752092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:38:33.752092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:38:33.752092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:38:33.752092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:33.752092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:38:33.752471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:33.752471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:38:33.752471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:33.752471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:38:33.752471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:33.752471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:38:33.752471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:33.752471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:38:33.752471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:38:33.752471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:33.752471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:38:33.752471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:33.752471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:38:33.752471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:38:33.752879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:33.752879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:38:33.752879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:33.752879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:38:33.753064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:33.753064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:38:33.753064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:38:33.753064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:38:33.753064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:38:33.753064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:38:33.753064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:38:33.753064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:38:33.753064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:38:33.753064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:38:33.753064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:38:33.753064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:38:33.753064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:38:33.753064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:38:33.753064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:38:33.753064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:38:33.753537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:38:33.753537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:38:33.753537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:38:33.753537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:38:33.753537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:38:33.753537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:33.753537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:38:33.753537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:33.753537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:38:33.753537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:33.753537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:38:33.753537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:33.753537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:38:33.753537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:33.753537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:38:33.753537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:38:33.753537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:33.753537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:38:33.754035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:33.754035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:38:33.754035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:38:33.754035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:33.754035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:38:33.754035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:33.754035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:38:33.754035 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:38:33.754306 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:38:33.754306 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:38:33.754468 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:38:33.754927 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:38:33.755174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:33.755740 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:38:33.755740 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:33.755740 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:38:33.755740 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:38:33.755740 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:33.755740 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:38:33.755740 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:33.755740 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:38:33.756625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:33.756625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:38:33.756625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:38:33.756625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:38:33.756625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:33.756625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:38:33.756625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:33.756625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:38:33.756625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:33.756625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:38:33.756625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:33.756625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:38:33.756625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:33.756625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:38:33.756625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:38:33.757089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:33.757089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:38:33.757089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:33.757089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:38:33.757089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:38:33.757089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:33.757089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:38:33.757089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:33.757089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:38:33.757844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:33.757844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:38:33.757844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:38:33.757844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:38:33.757844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:38:33.757844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:38:33.757844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:38:33.757844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:38:33.757844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:38:33.757844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:33.757844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:38:33.758307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:33.758307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:38:33.758307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:33.758307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:38:33.758307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:33.758307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:38:33.758307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:33.758307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:38:33.758307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:38:33.758307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:33.758307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:38:33.758307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:33.758307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:38:33.758307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:38:33.758814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:33.758814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:38:33.758814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:33.758814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:38:33.759629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:33.759629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:38:33.759629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:38:33.759629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:38:33.759629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:38:33.759629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:38:33.759629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:38:33.759629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:38:33.759629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:38:33.759629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:38:33.759629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:38:33.759629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:38:33.759629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:38:33.759629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:38:33.759629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:38:33.759629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:38:33.759629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:38:33.760130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:38:33.760130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:38:33.760130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:38:33.760130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:33.760130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:38:33.760130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:33.760130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:38:33.760130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:33.760130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:38:33.760130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:33.760130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:38:33.760130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:33.760130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:38:33.760130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:38:33.760130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:33.760130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:38:33.760130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:33.760130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:38:33.760130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:38:33.760676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:33.760676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:38:33.760676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:33.760676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:38:33.761866 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:38:33.762497 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:38:33.762828 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:38:33.763456 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:38:33.763937 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:38:33.764217 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:38:33.764413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:33.765088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:38:33.765088 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:33.765088 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:38:33.765088 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:38:33.765088 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:33.765088 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:38:33.765088 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:33.765088 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:38:33.767049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:33.767049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:38:33.767049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:38:33.767049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:38:33.767049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:38:33.767049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:38:33.767049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:38:33.767049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:33.767049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:38:33.767049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:33.767049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:38:33.767049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:33.767463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:38:33.767463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:33.767463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:38:33.767463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:33.767463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:38:33.767463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:38:33.767463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:33.767463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:38:33.767463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:33.767463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:38:33.767463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:38:33.767463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:33.767463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:38:33.767835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:33.767835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:38:33.768763 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:33.768763 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:38:33.768763 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:38:33.768763 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:38:33.768763 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:38:33.768763 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:38:33.768763 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:38:33.768763 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:38:33.768763 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:38:33.768763 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:38:33.768763 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:38:33.768763 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:38:33.768763 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:38:33.768763 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:38:33.768763 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:38:33.768763 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:38:33.769215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:38:33.769215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:38:33.769215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:33.769215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:38:33.769215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:33.769215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:38:33.769215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:33.769215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:38:33.769215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:33.769215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:38:33.769215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:33.769215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:38:33.769215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:38:33.769215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:33.769215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:38:33.769215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:33.769215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:38:33.769215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:38:33.769215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:33.769736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:38:33.769736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:33.769736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:38:38.589964 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:38:38.591112 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:38:38.591112 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:38:38.592173 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:38:38.636030 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:38:38.636365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:38.637352 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:38:38.637352 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:38.637352 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:38:38.637352 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:38:38.637352 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:38.637352 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:38:38.637352 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:38.637352 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:38:38.638220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.047 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:38.638220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:38:38.638220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:38:38.638220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:38:38.638220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:38:38.638220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:38:38.638220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:38:38.638220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:38.638220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:38:38.638220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:38.638220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:38:38.638220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:38.639068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:38:38.639068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:38.639068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:38:38.639068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:38.639068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:38:38.639068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:38:38.639068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:38.639068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:38:38.639068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:38.639068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:38:38.639068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:38:38.639068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:38.639068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:38:38.639648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:38.639648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:38:38.639648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:38.639648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:38:38.639648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:38:38.639648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:38:38.639648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:38:38.639648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:38:38.639648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:38:38.639648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:38:38.639648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:38:38.639648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:38:38.639648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:38:38.639648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:38:38.639648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:38:38.640181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:38:38.640181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:38:38.640181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:38:38.640181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:38:38.640181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:38:38.640181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:38.640181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:38:38.640181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:38.640181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:38:38.640181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:38.640181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:38:38.640181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:38.640181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:38:38.640181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:38.640181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:38:38.640181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:38:38.640181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:38.640181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:38:38.640181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:38.640795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:38:38.640795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:38:38.640795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:38.640795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:38:38.640795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:38.640795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:38:38.740562 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:38:38.741515 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:38:38.741873 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:38:38.743154 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:38:38.743660 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:38:38.743989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:38.745065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:38:38.745065 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:38.745065 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:38:38.745065 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:38:38.745065 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:38.745065 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:38:38.745065 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:38.745065 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:38:38.747100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:38.747100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:38:38.747100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:38:38.747100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:38:38.747100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:38:38.747100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:38:38.747100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:38:38.747100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:38.747100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:38:38.747100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:38.747100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:38:38.747100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:38.747474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:38:38.747474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:38.747474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:38:38.747474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:38.747474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:38:38.747474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:38:38.747474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:38.747474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:38:38.747474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:38.747474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:38:38.747474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:38:38.747474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:38.747474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:38:38.747803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:38.747803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:38:38.748696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:38.748696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:38:38.748696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:38:38.748696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:38:38.748696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:38:38.748696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:38:38.748696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:38:38.748696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:38:38.748696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:38:38.748696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:38:38.748696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:38:38.748696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:38:38.748696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:38:38.748696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:38:38.748696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:38:38.748696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:38:38.749159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:38:38.749159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:38:38.749159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:38.749159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:38:38.749159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:38.749159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:38:38.749159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:38.749159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:38:38.749159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:38.749159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:38:38.749159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:38.749159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:38:38.749159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:38:38.749159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:38.749159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:38:38.749159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:38.749159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:38:38.749159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:38:38.749159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:38.749676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:38:38.749676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:38.749676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:38:38.749676 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:38:38.749676 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:38:38.749863 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:38:38.750025 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:38:38.753278 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:38:38.753985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:38.754447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:38:38.754447 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:38.754447 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:38:38.754447 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:38:38.754447 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:38.754447 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:38:38.754447 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:38.754447 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:38:38.755696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:38.755696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:38:38.755696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:38:38.755696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:38:38.755696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:38.755696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:38:38.755696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:38.755696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:38:38.755696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:38.755696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:38:38.755696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:38.755696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:38:38.755696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:38.755696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:38:38.755696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:38:38.756171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:38.756171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:38:38.756171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:38.756171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:38:38.756171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:38:38.756171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:38.756171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:38:38.756171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:38.756171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:38:38.756171 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:38:38.757544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.006 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:38.757544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:38:38.757544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:38:38.757544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:38:38.757544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:38:38.757544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:38:38.757544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:38:38.757544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:38:38.757544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:38:38.757544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:38:38.757544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:38.757544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:38:38.757987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:38.757987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:38:38.757987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:38.757987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:38:38.757987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:38.757987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:38:38.757987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:38.757987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:38:38.757987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:38:38.757987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:38.757987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:38:38.757987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:38.757987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:38:38.757987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:38:38.758344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:38.758344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:38:38.758344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:38.758344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:38:38.758725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:38.758725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:38:38.758725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:38:38.758725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:38:38.758725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:38:38.758725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:38:38.758725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:38:38.758725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:38:38.758725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:38:38.758725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:38:38.758725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:38:38.758725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:38:38.758725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:38:38.758725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:38:38.758725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:38:38.758725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:38:38.759261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:38:38.759261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:38:38.759261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:38:38.759261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:38:38.759261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:38:38.759261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:38.759261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:38:38.759261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:38.759261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:38:38.759261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:38.759261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:38:38.759261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:38.759261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:38:38.759261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:38.759261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:38:38.759261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:38:38.759261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:38.759261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:38:38.759707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:38.759707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:38:38.759707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:38:38.759707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:38.759707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:38:38.759707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:38.759707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:38:38.759707 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:38:38.759707 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:38:38.759707 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:38:38.760013 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:38:38.760048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:38.760599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:38:38.760599 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:38.760599 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:38:38.760599 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:38:38.760599 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:38.760599 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:38:38.760599 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:38.760599 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:38:38.761227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:38.761227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:38:38.761227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:38:38.761227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:38:38.761227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:38.761227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:38:38.761227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:38.761227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:38:38.761227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:38.761227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:38:38.761227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:38.761227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:38:38.761227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:38.761227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:38:38.761227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:38:38.761627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:38.761627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:38:38.761627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:38.761627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:38:38.761627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:38:38.761627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:38.761627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:38:38.761627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:38.761627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:38:38.762089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:38.762089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:38:38.762089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:38:38.762089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:38:38.762089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:38:38.762089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:38:38.762089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:38:38.762089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:38:38.762089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:38:38.762089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:38.762089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:38:38.762410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:38.762410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:38:38.762410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:38.762410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:38:38.762410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:38.762410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:38:38.762410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:38.762410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:38:38.762410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:38:38.762410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:38.762410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:38:38.762410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:38.762410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:38:38.762410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:38:38.762736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:38.762736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:38:38.762736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:38.762736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:38:38.763517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:38.763517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:38:38.763517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:38:38.763517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:38:38.763517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:38:38.763517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:38:38.763517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:38:38.763517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:38:38.763517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:38:38.763517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:38:38.763517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:38:38.763517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:38:38.763517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:38:38.763517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:38:38.763517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:38:38.763517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:38:38.763517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:38:38.764131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:38:38.764131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:38:38.764131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:38:38.764131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:38.764131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:38:38.764131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:38.764131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:38:38.764131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:38.764131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:38:38.764131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:38.764131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:38:38.764131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:38.764131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:38:38.764131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:38:38.764131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:38.764131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:38:38.764131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:38.764131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:38:38.764131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:38:38.764767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:38.764767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:38:38.764767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:38.764767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:38:38.764767 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:38:38.764767 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:38:38.764767 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:38:38.764767 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:38:38.765019 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:38:38.765019 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:38:38.765019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:38.765503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:38:38.765503 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:38.765503 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:38:38.765503 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:38:38.765503 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:38.765503 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:38:38.765503 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:38.765503 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:38:38.766811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:38.766811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:38:38.766811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:38:38.766811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:38:38.766811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:38:38.766811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:38:38.766811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:38:38.766811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:38.766811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:38:38.766811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:38.766811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:38:38.766811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:38.767187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:38:38.767187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:38.767187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:38:38.767187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:38.767187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:38:38.767187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:38:38.767187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:38.767187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:38:38.767187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:38.767187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:38:38.767187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:38:38.767187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:38.767187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:38:38.767495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:38.767495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:38:38.768258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:38.768258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:38:38.768258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:38:38.768258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:38:38.768258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:38:38.768258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:38:38.768258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:38:38.768258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:38:38.768258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:38:38.768258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:38:38.768258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:38:38.768258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:38:38.768258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:38:38.768258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:38:38.768258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:38:38.768258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:38:38.768843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:38:38.768843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:38:38.768843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:38.768843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:38:38.768843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:38.768843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:38:38.768843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:38.768843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:38:38.768843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:38.768843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:38:38.768843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:38.768843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:38:38.768843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:38:38.768843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:38.768843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:38:38.768843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:38.768843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:38:38.768843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:38:38.768843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:38.769308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:38:38.769308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:38.769308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:38:43.590605 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:38:43.591256 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:38:43.591369 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:38:43.592394 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:38:43.631289 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:38:43.631629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:43.632527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:38:43.632527 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:43.632527 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:38:43.632527 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:38:43.632527 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:43.632527 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:38:43.632527 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:43.632527 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:38:43.633435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:43.633435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:38:43.633435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:38:43.633435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:38:43.633435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:38:43.633435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:38:43.633435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:38:43.633435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:43.633435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:38:43.633435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:43.633435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:38:43.633435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:43.633793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:38:43.633793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:43.633793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:38:43.633793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:43.633793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:38:43.633793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:38:43.633793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:43.633793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:38:43.633793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:43.633793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:38:43.633793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:38:43.633793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:43.633793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:38:43.634126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:43.634126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:38:43.634305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:43.634305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:38:43.634305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:38:43.634305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:38:43.634305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:38:43.634305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:38:43.634305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:38:43.634305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:38:43.634305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:38:43.634305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:38:43.634305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:38:43.634305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:38:43.634305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:38:43.634305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:38:43.634305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:38:43.634305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:38:43.635400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:38:43.635400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:38:43.635400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:43.635400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:38:43.635400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:43.635400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:38:43.635400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:43.635400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:38:43.635400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:43.635400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:38:43.635400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:43.635400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:38:43.635400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:38:43.635400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:43.635400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:38:43.635400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:43.635400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:38:43.635400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:38:43.635400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:43.636124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:38:43.636124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:43.636124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:38:43.741388 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:38:43.742064 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:38:43.742263 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:38:43.743616 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:38:43.744045 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:38:43.744370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:43.745605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:38:43.745605 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:43.745605 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:38:43.745605 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:38:43.745605 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:43.745605 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:38:43.745605 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:43.745605 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:38:43.747760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:43.747760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:38:43.747760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:38:43.747760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:38:43.747760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:38:43.747760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:38:43.747760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:38:43.747760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:43.747760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:38:43.747760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:43.747760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:38:43.747760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:43.748346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:38:43.748346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:43.748346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:38:43.748346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:43.748346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:38:43.748346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:38:43.748346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:43.748346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:38:43.748346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:43.748346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:38:43.748346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:38:43.748346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:43.748346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:38:43.748883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:43.748883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:38:43.750332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:43.750332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:38:43.750332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:38:43.750332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:38:43.750332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:38:43.750332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:38:43.750332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:38:43.750332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:38:43.750332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:38:43.750332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:38:43.750332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:38:43.750332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:38:43.750332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:38:43.750332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:38:43.750332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:38:43.750332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:38:43.751106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:38:43.751106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:38:43.751106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:43.751106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:38:43.751106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:43.751106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:38:43.751106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:43.751106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:38:43.751106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:43.751106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:38:43.751106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:43.751106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:38:43.751106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:38:43.751106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:43.751106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:38:43.751106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:43.751106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:38:43.751106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:38:43.751106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:43.751693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:38:43.751693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:43.751693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:38:43.751693 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:38:43.751693 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:38:43.751693 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:38:43.751693 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:38:43.752996 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:38:43.752996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:43.753541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:38:43.753541 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:43.753541 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:38:43.753541 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:38:43.753541 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:43.753541 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:38:43.753541 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:43.753541 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:38:43.754150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:43.754150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:38:43.754150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:38:43.754150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:38:43.754150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:43.754150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:38:43.754150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:43.754150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:38:43.754150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:43.754150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:38:43.754150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:43.754150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:38:43.754150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:43.754150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:38:43.754150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:38:43.755030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:43.755030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:38:43.755030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:43.755030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:38:43.755030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:38:43.755030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:43.755030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:38:43.755030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:43.755030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:38:43.755030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:43.755030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:38:43.755030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:38:43.755030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:38:43.755030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:38:43.755858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:38:43.755858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:38:43.755858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:38:43.755858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:38:43.755858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:38:43.755858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:43.755858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:38:43.755858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:43.755858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:38:43.755858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:43.755858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:38:43.755858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:43.755858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:38:43.756343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:43.756343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:38:43.756343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:38:43.756343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:43.756343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:38:43.756343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:43.756343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:38:43.756343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:38:43.756343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:43.756343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:38:43.756343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:43.756343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:38:43.756689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:43.756689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:38:43.756689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:38:43.756689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:38:43.756689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:38:43.756689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:38:43.756689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:38:43.756689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:38:43.756689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:38:43.756689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:38:43.756689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:38:43.756689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:38:43.756689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:38:43.756689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:38:43.756689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:38:43.756689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:38:43.757109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:38:43.757109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:38:43.757109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:38:43.757109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:38:43.757109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:38:43.757109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:43.757109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:38:43.757109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:43.757109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:38:43.757109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:43.757109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:38:43.757109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:43.757109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:38:43.757109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:43.757109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:38:43.757109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:38:43.757109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:43.757109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:38:43.759416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:43.759416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:38:43.759416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:38:43.759416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:43.759416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:38:43.759416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:43.759416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:38:43.759416 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:38:43.759416 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:38:43.759416 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:38:43.759632 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:38:43.759902 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:38:43.760133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:43.760627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:38:43.760627 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:43.760627 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:38:43.760627 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:38:43.760627 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:43.760627 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:38:43.760627 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:43.760627 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:38:43.761428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:43.761428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:38:43.761428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:38:43.761428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:38:43.761428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:43.761428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:38:43.761428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:43.761428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:38:43.761428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:43.761428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:38:43.761428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:43.761428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:38:43.761428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:43.761428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:38:43.761428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:38:43.761692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:43.761692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:38:43.761692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:43.761692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:38:43.761692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:38:43.761692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:43.761692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:38:43.761692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:43.761692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:38:43.762390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:43.762390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:38:43.762390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:38:43.762390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:38:43.762390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:38:43.762390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:38:43.762390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:38:43.762390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:38:43.762390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:38:43.762390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:43.762390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:38:43.762616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:43.762616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:38:43.762616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:43.762616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:38:43.762616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:43.762616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:38:43.762616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:43.762616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:38:43.762616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:38:43.762616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:43.762616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:38:43.762616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:43.762616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:38:43.762616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:38:43.763009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:43.763009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:38:43.763009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:43.763009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:38:43.763620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:43.763620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:38:43.763620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:38:43.763620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:38:43.763620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:38:43.763620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:38:43.763620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:38:43.763620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:38:43.763620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:38:43.763620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:38:43.763620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:38:43.763620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:38:43.763620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:38:43.763620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:38:43.763620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:38:43.763620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:38:43.763620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:38:43.764091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:38:43.764091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:38:43.764091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:38:43.764091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:43.764091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:38:43.764091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:43.764091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:38:43.764091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:43.764091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:38:43.764091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:43.764091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:38:43.764091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:43.764091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:38:43.764091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:38:43.764091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:43.764091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:38:43.764091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:43.764091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:38:43.764091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:38:43.764528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:43.764528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:38:43.764528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:43.764528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:38:43.764528 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:38:43.764528 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:38:43.764528 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:38:43.764528 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:38:43.764859 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:38:43.765104 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:38:43.765253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:43.765816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:38:43.765816 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:43.765816 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:38:43.765816 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:38:43.765816 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:43.765816 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:38:43.765816 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:43.765816 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:38:43.766615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:43.766615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:38:43.766615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:38:43.766615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:38:43.766615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:38:43.766615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:38:43.766615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:38:43.766615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:43.766615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:38:43.766615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:43.766615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:38:43.766615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:43.766945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:38:43.766945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:43.766945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:38:43.766945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:43.766945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:38:43.766945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:38:43.766945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:43.766945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:38:43.766945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:43.766945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:38:43.766945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:38:43.766945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:43.766945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:38:43.767255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:43.767255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:38:43.767740 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:43.767740 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:38:43.767740 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:38:43.767740 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:38:43.767740 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:38:43.767740 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:38:43.767740 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:38:43.767740 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:38:43.767740 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:38:43.767740 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:38:43.767740 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:38:43.767740 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:38:43.767740 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:38:43.767740 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:38:43.767740 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:38:43.767740 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:38:43.768253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:38:43.768253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:38:43.768253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:43.768253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:38:43.768253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:43.768253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:38:43.768253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:43.768253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:38:43.768253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:43.768253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:38:43.768253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:43.768253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:38:43.768253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:38:43.768253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:43.768253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:38:43.768253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:43.768253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:38:43.768253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:38:43.768253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:43.768805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:38:43.768805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:43.768805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:38:48.591643 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:38:48.592627 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:38:48.592627 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:38:48.594233 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:38:48.633943 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:38:48.634182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:48.634893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:38:48.634893 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:48.634893 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:38:48.634893 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:38:48.634893 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:48.634893 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:38:48.634893 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:48.634893 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:38:48.635917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:48.635917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:38:48.635917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:38:48.635917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:38:48.635917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:38:48.635917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:38:48.635917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:38:48.635917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:48.635917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:38:48.635917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:48.635917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:38:48.635917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:48.636566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:38:48.636566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:48.636566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:38:48.636566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:48.636566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:38:48.636566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:38:48.636566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:48.636566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:38:48.636566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:48.636566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:38:48.636566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:38:48.636566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:48.636566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:38:48.637278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:48.637278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:38:48.637278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:48.637278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:38:48.637278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:38:48.637278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:38:48.637278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:38:48.637278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:38:48.637278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:38:48.637278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:38:48.637278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:38:48.637278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:38:48.637278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:38:48.637278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:38:48.637278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:38:48.638095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:38:48.638095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:38:48.638095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:38:48.638095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:38:48.638095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:38:48.638095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:48.638095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:38:48.638095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:48.638095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:38:48.638095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:48.638095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:38:48.638095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:48.638095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:38:48.638095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:48.638095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:38:48.638095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:38:48.638095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:48.638095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:38:48.638095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:48.639036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:38:48.639036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:38:48.639036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:48.639036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:38:48.639036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:48.639036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:38:48.742351 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:38:48.743284 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:38:48.743372 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:38:48.744340 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:38:48.744587 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:38:48.744772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:48.745435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:38:48.745435 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:48.745435 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:38:48.745435 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:38:48.745435 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:48.745435 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:38:48.745435 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:48.745435 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:38:48.746284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:48.746284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:38:48.746284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:38:48.746284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:38:48.746284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:38:48.746284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:38:48.746284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:38:48.746284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:48.746284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:38:48.746284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:48.746284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:38:48.746284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:48.746656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:38:48.746656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:48.746656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:38:48.746656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:48.746656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:38:48.746656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:38:48.746656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:48.746656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:38:48.746656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:48.746656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:38:48.746656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:38:48.746656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:48.746656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:38:48.747045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:48.747045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:38:48.747122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:48.747122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:38:48.747122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:38:48.747122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:38:48.747122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:38:48.747122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:38:48.747122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:38:48.747122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:38:48.747122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:38:48.747122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:38:48.747122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:38:48.747122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:38:48.747122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:38:48.747122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:38:48.747122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:38:48.747122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:38:48.747578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:38:48.747578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:38:48.747578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:48.747578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:38:48.747578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:48.747578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:38:48.747578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:48.747578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:38:48.747578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:48.747578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:38:48.747578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:48.747578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:38:48.747578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:38:48.747578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:48.747578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:38:48.747578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:48.747578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:38:48.747578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:38:48.747578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:48.748103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:38:48.748103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:48.748103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:38:48.751374 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:38:48.751716 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:38:48.751814 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:38:48.751893 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:38:48.752494 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:38:48.752494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:48.753033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:38:48.753033 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:48.753033 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:38:48.753033 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:38:48.753033 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:48.753033 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:38:48.753033 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:48.753033 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:38:48.753598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:48.753598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:38:48.753598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:38:48.753598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:38:48.753598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:48.753598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:38:48.753598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:48.753598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:38:48.753598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:48.753598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:38:48.753598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:48.753598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:38:48.753598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:48.753598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:38:48.753598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:38:48.754161 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:48.754161 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:38:48.754161 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:48.754161 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:38:48.754161 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:38:48.754161 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:48.754161 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:38:48.754161 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:48.754161 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:38:48.754452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:48.754452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:38:48.754452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:38:48.754452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:38:48.754452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:38:48.754452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:38:48.754452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:38:48.754452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:38:48.754452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:38:48.754452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:38:48.754452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:48.754452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:38:48.754864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:48.754864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:38:48.754864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:48.754864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:38:48.754864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:48.754864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:38:48.754864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:48.754864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:38:48.754864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:38:48.754864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:48.754864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:38:48.754864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:48.754864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:38:48.754864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:38:48.755292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:48.755292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:38:48.755292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:48.755292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:38:48.755292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:48.755292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:38:48.755292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:38:48.755292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:38:48.755292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:38:48.755292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:38:48.755292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:38:48.755292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:38:48.755292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:38:48.755292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:38:48.755704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:38:48.755704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:38:48.755704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:38:48.755704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:38:48.755704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:38:48.755704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:38:48.755704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:38:48.755704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:38:48.755704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:38:48.755704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:38:48.755704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:38:48.755704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:48.755704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:38:48.755704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:48.755704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:38:48.755704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:48.755704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:38:48.755704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:48.755704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:38:48.756329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:48.756329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:38:48.756329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:38:48.756329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:48.756329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:38:48.756329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:48.756329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:38:48.756329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:38:48.756329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:48.756329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:38:48.756329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:48.756329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:38:48.759580 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:38:48.760020 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:38:48.760020 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:38:48.760125 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:38:48.760564 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:38:48.760974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:48.761817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:38:48.761817 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:48.761817 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:38:48.761817 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:38:48.761817 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:48.761817 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:38:48.761817 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:48.761817 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:38:48.763052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:48.763052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:38:48.763052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:38:48.763052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:38:48.763052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:48.763052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:38:48.763052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:48.763052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:38:48.763052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:48.763052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:38:48.763052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:48.763052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:38:48.763052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:48.763052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:38:48.763052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:38:48.763393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:48.763393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:38:48.763393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:48.763393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:38:48.763393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:38:48.763393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:48.763393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:38:48.763393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:48.763393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:38:48.764526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:48.764526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:38:48.764526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:38:48.764526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:38:48.764526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:38:48.764526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:38:48.764526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:38:48.764526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:38:48.764526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:38:48.764526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:48.764526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:38:48.764950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:48.764950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:38:48.764950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:48.764950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:38:48.764950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:48.764950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:38:48.764950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:48.764950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:38:48.764950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:38:48.764950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:48.764950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:38:48.764950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:48.764950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:38:48.764950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:38:48.765348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:48.765348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:38:48.765348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:48.765348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:38:48.767202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:48.767202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:38:48.767202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:38:48.767202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:38:48.767202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:38:48.767202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:38:48.767202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:38:48.767202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:38:48.767202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:38:48.767202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:38:48.767202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:38:48.767202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:38:48.767202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:38:48.767202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:38:48.767202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:38:48.767202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:38:48.767202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:38:48.767979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:38:48.767979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:38:48.767979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:38:48.767979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:48.767979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:38:48.767979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:48.767979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:38:48.767979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:48.767979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:38:48.767979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:48.767979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:38:48.767979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:48.767979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:38:48.767979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:38:48.767979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:48.767979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:38:48.767979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:48.767979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:38:48.767979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:38:48.768550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:48.768550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:38:48.768550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:48.768550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:38:48.768550 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:38:48.768550 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:38:48.768550 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:38:48.768550 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:38:48.769043 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:38:48.769407 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:38:48.769647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:48.770481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:38:48.770481 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:48.770481 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:38:48.770481 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:38:48.770481 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:48.770481 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:38:48.770481 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:48.770481 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:38:48.771393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:48.771393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:38:48.771393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:38:48.771393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:38:48.771393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:38:48.771393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:38:48.771393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:38:48.771393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:48.771393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:38:48.771393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:48.771393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:38:48.771393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:48.771965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:38:48.771965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:48.771965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:38:48.771965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:48.771965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:38:48.771965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:38:48.771965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:48.771965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:38:48.771965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:48.771965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:38:48.771965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:38:48.771965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:48.771965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:38:48.772338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:48.772338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:38:48.772338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:48.772338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:38:48.772338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:38:48.772338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:38:48.772338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:38:48.772338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:38:48.772338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:38:48.772338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:38:48.772338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:38:48.772338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:38:48.772338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:38:48.772338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:38:48.772338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:38:48.772830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:38:48.772830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:38:48.772830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:38:48.772830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:38:48.772830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:38:48.772830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:48.772830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:38:48.772830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:48.772830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:38:48.772830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:48.772830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:38:48.772830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:48.772830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:38:48.772830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:48.772830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:38:48.772830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:38:48.772830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:48.772830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:38:48.772830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:48.773383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:38:48.773383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:38:48.773383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:48.773383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:38:48.773383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:48.773383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:38:53.592350 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:38:53.593579 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:38:53.593579 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:38:53.594665 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:38:53.627890 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:38:53.627890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:53.628113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:38:53.628113 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:53.628113 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:38:53.628113 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:38:53.628113 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:53.628113 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:38:53.628113 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:53.628113 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:38:53.628980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:53.628980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:38:53.628980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:38:53.628980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:38:53.628980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:38:53.628980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:38:53.628980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:38:53.628980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:53.628980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:38:53.628980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:53.628980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:38:53.628980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:53.629493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:38:53.629493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:53.629493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:38:53.629493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:53.629493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:38:53.629493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:38:53.629493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:53.629493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:38:53.629493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:53.629493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:38:53.629493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:38:53.629493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:53.629493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:38:53.630086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:53.630086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:38:53.630086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:53.630086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:38:53.630086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:38:53.630086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:38:53.630086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:38:53.630086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:38:53.630086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:38:53.630086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:38:53.630086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:38:53.630086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:38:53.630086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:38:53.630086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:38:53.630086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:38:53.630723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:38:53.630723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:38:53.630723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:38:53.630723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:38:53.630723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:38:53.630723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:53.630723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:38:53.630723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:53.630723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:38:53.630723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:53.630723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:38:53.630723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:53.630723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:38:53.630723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:53.630723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:38:53.630723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:38:53.630723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:53.630723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:38:53.630723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:53.631525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:38:53.631525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:38:53.631525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:53.631525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:38:53.631525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:53.631525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:38:53.743430 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:38:53.743969 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:38:53.744181 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:38:53.745853 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:38:53.745853 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:38:53.746144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:53.747120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:38:53.747120 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:53.747120 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:38:53.747120 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:38:53.747120 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:53.747120 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:38:53.747120 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:53.747120 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:38:53.748842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:53.748842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:38:53.748842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:38:53.748842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:38:53.748842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:38:53.748842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:38:53.748842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:38:53.748842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:53.748842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:38:53.748842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:53.748842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:38:53.748842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:53.749206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:38:53.749206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:53.749206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:38:53.749206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:53.749206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:38:53.749206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:38:53.749206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:53.749206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:38:53.749206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:53.749206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:38:53.749206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:38:53.749206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:53.749206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:38:53.749553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:53.749553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:38:53.750457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:53.750457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:38:53.750457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:38:53.750457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:38:53.750457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:38:53.750457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:38:53.750457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:38:53.750457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:38:53.750457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:38:53.750457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:38:53.750457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:38:53.750457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:38:53.750457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:38:53.750457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:38:53.750457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:38:53.750457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:38:53.750928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:38:53.750928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:38:53.750928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:53.750928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:38:53.750928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:53.750928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:38:53.750928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:53.750928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:38:53.750928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:53.750928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:38:53.750928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:53.750928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:38:53.750928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:38:53.750928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:53.750928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:38:53.750928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:53.750928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:38:53.750928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:38:53.750928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:53.751389 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:38:53.751389 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:53.751389 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:38:53.751762 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:38:53.752086 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:38:53.752166 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:38:53.752365 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:38:53.752899 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:38:53.753253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:53.753960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:38:53.753960 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:53.753960 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:38:53.753960 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:38:53.753960 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:53.753960 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:38:53.753960 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:53.753960 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:38:53.754768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:53.754768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:38:53.754768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:38:53.754768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:38:53.754768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:53.754768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:38:53.754768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:53.754768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:38:53.754768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:53.754768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:38:53.754768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:53.754768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:38:53.754768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:53.754768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:38:53.754768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:38:53.755150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:53.755150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:38:53.755150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:53.755150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:38:53.755150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:38:53.755150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:53.755150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:38:53.755150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:53.755150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:38:53.755687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:53.755687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:38:53.755687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:38:53.755687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:38:53.755687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:38:53.755687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:38:53.755687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:38:53.755687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:38:53.755687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:38:53.755687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:38:53.755687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:53.755687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:38:53.756078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:53.756078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:38:53.756078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:53.756078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:38:53.756078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:53.756078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:38:53.756078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:53.756078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:38:53.756078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:38:53.756078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:53.756078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:38:53.756078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:53.756078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:38:53.756078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:38:53.756460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:53.756460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:38:53.756460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:53.756460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:38:53.756755 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:53.756755 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:38:53.756755 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:38:53.756755 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:38:53.756755 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:38:53.756755 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:38:53.756755 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:38:53.756755 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:38:53.756755 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:38:53.756755 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:38:53.756755 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:38:53.756755 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:38:53.756755 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:38:53.756755 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:38:53.756755 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:38:53.756755 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:38:53.757209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:38:53.757209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:38:53.757209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:38:53.757209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:38:53.757209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:38:53.757209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:53.757209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:38:53.757209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:53.757209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:38:53.757209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:53.757209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:38:53.757209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:53.757209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:38:53.757209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:53.757209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:38:53.757209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:38:53.757209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:53.757209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:38:53.757682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:53.757682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:38:53.757682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:38:53.757682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:53.757682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:38:53.757682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:53.757682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:38:53.759967 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:38:53.760255 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:38:53.760330 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:38:53.760639 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:38:53.760956 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:38:53.761550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:53.762218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:38:53.762218 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:53.762218 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:38:53.762218 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:38:53.762218 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:53.762218 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:38:53.762218 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:53.762218 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:38:53.763089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:53.763089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:38:53.763089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:38:53.763089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:38:53.763089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:53.763089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:38:53.763089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:53.763089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:38:53.763089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:53.763089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:38:53.763089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:53.763089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:38:53.763089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:53.763089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:38:53.763089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:38:53.763676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:53.763676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:38:53.763676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:53.763676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:38:53.763676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:38:53.763676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:53.763676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:38:53.763676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:53.763676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:38:53.764053 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:53.764053 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:38:53.764053 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:38:53.764053 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:38:53.764053 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:38:53.764053 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:38:53.764053 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:38:53.764053 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:38:53.764053 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:38:53.764053 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:53.764053 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:38:53.764473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:53.764473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:38:53.764473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:53.764473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:38:53.764473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:53.764473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:38:53.764473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:53.764473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:38:53.764473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:38:53.764473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:53.764473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:38:53.764473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:53.764473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:38:53.764473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:38:53.765350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:53.765350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:38:53.765350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:53.765350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:38:53.765350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:53.765350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:38:53.765350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:38:53.765350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:38:53.765350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:38:53.765350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:38:53.765350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:38:53.765350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:38:53.765350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:38:53.765350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:38:53.766342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:38:53.766342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:38:53.766342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:38:53.766342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:38:53.766342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:38:53.766342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:38:53.766342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:38:53.766342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:38:53.766342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:38:53.766342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:38:53.766342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:53.766342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:38:53.766342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:53.766342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:38:53.766342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:53.766342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:38:53.766342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:53.766342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:38:53.766342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:53.767297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:38:53.767297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:38:53.767297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:53.767297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:38:53.767297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:53.767297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:38:53.767297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:38:53.767297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:53.767297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:38:53.767297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:53.767297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:38:53.768067 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:38:53.768351 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:38:53.768479 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:38:53.768668 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:38:53.769212 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:38:53.769770 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:38:53.770918 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:53.770918 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:38:53.770918 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:53.770918 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:38:53.770918 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:38:53.770918 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:53.770918 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:38:53.770918 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:53.770918 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:38:53.772037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:53.772037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:38:53.772037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:38:53.772037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:38:53.772037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:38:53.772037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:38:53.772037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:38:53.772037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:53.772037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:38:53.772037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:53.772037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:38:53.772037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:53.772605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:38:53.772605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:53.772605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:38:53.772605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:53.772605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:38:53.772605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:38:53.772605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:53.772605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:38:53.772605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:53.772605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:38:53.772605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:38:53.772605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:53.772605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:38:53.773289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:53.773289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:38:53.773289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:53.773289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:38:53.773289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:38:53.773289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:38:53.773289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:38:53.773289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:38:53.773289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:38:53.773289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:38:53.773289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:38:53.773289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:38:53.773289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:38:53.773289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:38:53.773289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:38:53.773951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:38:53.773951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:38:53.773951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:38:53.773951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:38:53.773951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:38:53.773951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:53.773951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:38:53.773951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:53.773951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:38:53.773951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:53.773951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:38:53.773951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:53.773951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:38:53.773951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:53.773951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:38:53.773951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:38:53.773951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:53.773951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:38:53.773951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:53.774932 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:38:53.774932 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:38:53.774932 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:53.774932 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:38:53.774932 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:53.774932 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:38:58.593790 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:38:58.594420 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:38:58.594420 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:38:58.595229 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:38:58.598078 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG dbcounter [-] [106314] Writing DB stats neutron:SELECT=240 {{(pid=106314) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 09 09:38:58.632525 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:38:58.632937 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:58.633873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:38:58.633873 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:58.633873 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:38:58.633873 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:38:58.633873 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:58.633873 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:38:58.633873 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:58.633873 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:38:58.635508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:58.635508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:38:58.635508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:38:58.635508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:38:58.635508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:38:58.635508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:38:58.635508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:38:58.635508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:58.635508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:38:58.635508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:58.635508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:38:58.635508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:58.636276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:38:58.636276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:58.636276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:38:58.636276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:58.636276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:38:58.636276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:38:58.636276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:58.636276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:38:58.636276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:58.636276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:38:58.636276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:38:58.636276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:58.636276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:38:58.637371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:58.637371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:38:58.637609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:58.637609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:38:58.637609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:38:58.637609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:38:58.637609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:38:58.637609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:38:58.637609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:38:58.637609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:38:58.637609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:38:58.637609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:38:58.637609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:38:58.637609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:38:58.637609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:38:58.637609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:38:58.637609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:38:58.637609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:38:58.638534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:38:58.638534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:38:58.638534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:58.638534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:38:58.638534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:58.638534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:38:58.638534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:58.638534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:38:58.638534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:58.638534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:38:58.638534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:58.638534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:38:58.638534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:38:58.638534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:58.638534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:38:58.638534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:58.638534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:38:58.638534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:38:58.638534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:58.641273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:38:58.641273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:58.641273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:38:58.744149 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:38:58.744544 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:38:58.744700 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:38:58.745434 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:38:58.745627 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:38:58.745796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:58.746467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:38:58.746467 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:58.746467 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:38:58.746467 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:38:58.746467 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:58.746467 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:38:58.746467 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:58.746467 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:38:58.747178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:58.747178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:38:58.747178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:38:58.747178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:38:58.747178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:38:58.747178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:38:58.747178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:38:58.747178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:58.747178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:38:58.747178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:58.747178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:38:58.747178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:58.747936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:38:58.747936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:58.747936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:38:58.747936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:58.747936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:38:58.747936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:38:58.747936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:58.747936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:38:58.747936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:58.747936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:38:58.747936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:38:58.747936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:58.747936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:38:58.748941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:58.748941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:38:58.748941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:58.748941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:38:58.748941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:38:58.748941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:38:58.748941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:38:58.748941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:38:58.748941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:38:58.748941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:38:58.748941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:38:58.748941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:38:58.748941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:38:58.748941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:38:58.748941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:38:58.750165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:38:58.750165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:38:58.750165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:38:58.750165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:38:58.750165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:38:58.750165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:58.750165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:38:58.750165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:58.750165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:38:58.750165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:58.750165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:38:58.750165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:58.750165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:38:58.750165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:58.750165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:38:58.750165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:38:58.750165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:58.750165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:38:58.750165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:58.751252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:38:58.751252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:38:58.751252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:58.751252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:38:58.751252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:58.751252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:38:58.752212 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:38:58.754140 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:38:58.754256 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:38:58.754330 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:38:58.754759 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:38:58.754980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:58.755530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:38:58.755530 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:58.755530 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:38:58.755530 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:38:58.755530 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:58.755530 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:38:58.755530 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:58.755530 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:38:58.756114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:58.756114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:38:58.756114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:38:58.756114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:38:58.756114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:58.756114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:38:58.756114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:58.756114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:38:58.756114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:58.756114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:38:58.756114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:58.756114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:38:58.756114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:58.756114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:38:58.756114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:38:58.757228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:58.757228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:38:58.757228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:58.757228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:38:58.757228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:38:58.757228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:58.757228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:38:58.757228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:58.757228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:38:58.757228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:58.757228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:38:58.757228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:38:58.757228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:38:58.757228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:38:58.757644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:38:58.757644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:38:58.757644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:38:58.757644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:38:58.757644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:38:58.757644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:58.757644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:38:58.757644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:58.757644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:38:58.757644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:58.757644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:38:58.757644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:58.757644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:38:58.758028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:58.758028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:38:58.758028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:38:58.758028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:58.758028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:38:58.758028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:58.758028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:38:58.758028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:38:58.758028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:58.758028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:38:58.758028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:58.758028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:38:58.758331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:58.758331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:38:58.758331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:38:58.758331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:38:58.758331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:38:58.758331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:38:58.758331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:38:58.758331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:38:58.758331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:38:58.758331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:38:58.758331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:38:58.758331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:38:58.758331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:38:58.758331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:38:58.758331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:38:58.758331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:38:58.758738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:38:58.758738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:38:58.758738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:38:58.758738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:38:58.758738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:38:58.758738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:58.758738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:38:58.758738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:58.758738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:38:58.758738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:58.758738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:38:58.758738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:58.758738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:38:58.758738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:58.758738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:38:58.758738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:38:58.758738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:58.758738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:38:58.759238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:58.759238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:38:58.759238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:38:58.759238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:58.759238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:38:58.759238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:58.759238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:38:58.760475 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:38:58.760874 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:38:58.760978 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:38:58.761112 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:38:58.761574 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:38:58.761846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:58.762320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:38:58.762320 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:58.762320 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:38:58.762320 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:38:58.762320 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:58.762320 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:38:58.762320 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:58.762320 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:38:58.762883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:58.762883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:38:58.762883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:38:58.762883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:38:58.762883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:58.762883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:38:58.762883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:58.762883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:38:58.762883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:58.762883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:38:58.762883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:58.762883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:38:58.762883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:58.762883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:38:58.762883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:38:58.763795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:58.763795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:38:58.763795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:58.763795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:38:58.763795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:38:58.763795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:58.763795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:38:58.763795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:58.763795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:38:58.763795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:58.763795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:38:58.763795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:38:58.763795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:38:58.763795 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:38:58.764577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:38:58.764577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:38:58.764577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:38:58.764577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:38:58.764577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:58.764577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:38:58.764577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:58.764577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:38:58.764577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:58.764577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:38:58.764577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:58.764577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:38:58.764577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:58.765338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:38:58.765338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:38:58.765338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:58.765338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:38:58.765338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:58.765338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:38:58.765338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:38:58.765338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:58.765338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:38:58.765338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:58.765338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:38:58.765338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:58.766126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:38:58.766126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:38:58.766126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:38:58.766126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:38:58.766126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:38:58.766126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:38:58.766126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:38:58.766126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:38:58.766126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:38:58.766126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:38:58.766126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:38:58.766126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:38:58.766126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:38:58.766126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:38:58.766126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:38:58.766126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:38:58.766126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:38:58.766126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:38:58.766126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:38:58.766637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:58.766637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:38:58.766637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:58.766637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:38:58.766637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:58.766637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:38:58.766637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:58.766637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:38:58.766637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:58.766637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:38:58.766637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:38:58.766637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:58.766637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:38:58.766637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:58.766637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:38:58.766637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:38:58.766637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:58.766637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:38:58.766637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:58.766637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:38:58.768450 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:38:58.768711 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:38:58.768803 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:38:58.768942 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:38:58.769184 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:38:58.769438 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:38:58.769555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:58.770191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:38:58.770191 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:58.770191 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:38:58.770191 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:38:58.770191 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:58.770191 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:38:58.770191 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:58.770191 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:38:58.771340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:58.771340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:38:58.771340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:38:58.771340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:38:58.771340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:38:58.771340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:38:58.771340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:38:58.771340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:58.771340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:38:58.771340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:58.771340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:38:58.771340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:58.771711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:38:58.771711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:58.771711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:38:58.771711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:58.771711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:38:58.771711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:38:58.771711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:58.771711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:38:58.771711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:58.771711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:38:58.771711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:38:58.771711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:58.771711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:38:58.772062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:58.772062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:38:58.772592 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:58.772592 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:38:58.772592 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:38:58.772592 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:38:58.772592 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:38:58.772592 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:38:58.772592 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:38:58.772592 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:38:58.772592 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:38:58.772592 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:38:58.772592 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:38:58.772592 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:38:58.772592 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:38:58.772592 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:38:58.772592 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:38:58.772592 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:38:58.773057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:38:58.773057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:38:58.773057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:58.773057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:38:58.773057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:38:58.773057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:38:58.773057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:38:58.773057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:38:58.773057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:38:58.773057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:38:58.773057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:38:58.773057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:38:58.773057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:38:58.773057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:38:58.773057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:38:58.773057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:38:58.773057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:38:58.773057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:38:58.773057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:38:58.773820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:38:58.773820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:38:58.773820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:39:03.594320 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:39:03.595512 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:39:03.595725 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:39:03.597136 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:39:03.630228 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:39:03.630681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:03.631612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:39:03.631612 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:03.631612 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:39:03.631612 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:39:03.631612 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:03.631612 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:39:03.631612 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:03.631612 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:39:03.632377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:03.632377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:39:03.632377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:39:03.632377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:39:03.632377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:39:03.632377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:39:03.632377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:39:03.632377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:03.632377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:39:03.632377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:03.632377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:39:03.632377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:03.633036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:39:03.633036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:03.633036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:39:03.633036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:03.633036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:39:03.633036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:39:03.633036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:03.633036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:39:03.633036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:03.633036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:39:03.633036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:39:03.633036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:03.633036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:39:03.633402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:03.633402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:39:03.633402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:03.633402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:39:03.633402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:39:03.633402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:39:03.633402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:39:03.633402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:39:03.633402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:39:03.633402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:39:03.633402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:39:03.633402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:39:03.633402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:39:03.633402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:39:03.633402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:39:03.633810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:39:03.633810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:39:03.633810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:39:03.633810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:39:03.633810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:39:03.633810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:03.633810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:39:03.633810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:03.633810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:39:03.633810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:03.633810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:39:03.633810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:03.633810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:39:03.633810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:03.633810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:39:03.633810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:39:03.633810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:03.633810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:39:03.633810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:03.634291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:39:03.634291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:39:03.634291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:03.634291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:39:03.634291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:03.634291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:39:03.745122 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:39:03.745973 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:39:03.746249 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:39:03.747584 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:39:03.747997 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:39:03.748291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:03.749206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:39:03.749206 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:03.749206 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:39:03.749206 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:39:03.749206 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:03.749206 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:39:03.749206 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:03.749206 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:39:03.750402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:03.750402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:39:03.750402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:39:03.750402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:39:03.750402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:39:03.750402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:39:03.750402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:39:03.750402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:03.750402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:39:03.750402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:03.750402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:39:03.750402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:03.751068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:39:03.751068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:03.751068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:39:03.751068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:03.751068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:39:03.751068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:39:03.751068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:03.751068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:39:03.751068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:03.751068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:39:03.751068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:39:03.751068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:03.751068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:39:03.751879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:03.751879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:39:03.751879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:03.751879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:39:03.751879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:39:03.751879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:39:03.751879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:39:03.751879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:39:03.751879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:39:03.751879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:39:03.751879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:39:03.751879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:39:03.751879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:39:03.751879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:39:03.751879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:39:03.752597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:39:03.752597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:39:03.752597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:39:03.752597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:39:03.752597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:39:03.752597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:03.752597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:39:03.752597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:03.752597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:39:03.752597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:03.752597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:39:03.752597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:03.752597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:39:03.752597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:03.752597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:39:03.752597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:39:03.752597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:03.752597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:39:03.752597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:03.753437 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:39:03.753437 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:39:03.753437 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:03.753437 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:39:03.753437 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:03.753437 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:39:03.754125 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:39:03.754394 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:39:03.754502 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:39:03.754620 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:39:03.755171 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:39:03.755487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:03.756217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:39:03.756217 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:03.756217 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:39:03.756217 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:39:03.756217 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:03.756217 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:39:03.756217 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:03.756217 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:39:03.757271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:03.757271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:39:03.757271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:39:03.757271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:39:03.757271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:03.757271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:39:03.757271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:03.757271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:39:03.757271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:03.757271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:39:03.757271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:03.757271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:39:03.757271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:03.757271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:39:03.757271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:39:03.758129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:03.758129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:39:03.758129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:03.758129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:39:03.758129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:39:03.758129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:03.758129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:39:03.758129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:03.758129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:39:03.758578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:03.758578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:39:03.758578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:39:03.758578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:39:03.758578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:39:03.758578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:39:03.758578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:39:03.758578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:39:03.758578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:39:03.758578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:39:03.758578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:03.758578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:39:03.758970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:03.758970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:39:03.758970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:03.758970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:39:03.758970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:03.758970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:39:03.758970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:03.758970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:39:03.758970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:39:03.758970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:03.758970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:39:03.758970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:03.758970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:39:03.758970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:39:03.759363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:03.759363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:39:03.759363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:03.759363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:39:03.760075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:03.760075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:39:03.760075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:39:03.760075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:39:03.760075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:39:03.760075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:39:03.760075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:39:03.760075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:39:03.760075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:39:03.760075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:39:03.760075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:39:03.760075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:39:03.760075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:39:03.760075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:39:03.760075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:39:03.760075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:39:03.760526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:39:03.760526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:39:03.760526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:39:03.760526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:39:03.760526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:39:03.760526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:03.760526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:39:03.760526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:03.760526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:39:03.760526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:03.760526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:39:03.760526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:03.760526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:39:03.760526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:03.760526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:39:03.760526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:39:03.760526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:03.760526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:39:03.761059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:03.761059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:39:03.761059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:39:03.761059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:03.761059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:39:03.761059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:03.761059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:39:03.761059 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:39:03.761346 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:39:03.761497 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:39:03.761634 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:39:03.762164 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:39:03.762555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:03.763345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:39:03.763345 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:03.763345 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:39:03.763345 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:39:03.763345 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:03.763345 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:39:03.763345 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:03.763345 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:39:03.764152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:03.764152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:39:03.764152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:39:03.764152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:39:03.764152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:03.764152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:39:03.764152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:03.764152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:39:03.764152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:03.764152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:39:03.764152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:03.764152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:39:03.764152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:03.764152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:39:03.764152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:39:03.764588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:03.764588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:39:03.764588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:03.764588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:39:03.764588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:39:03.764588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:03.764588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:39:03.764588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:03.764588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:39:03.765062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:03.765062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:39:03.765062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:39:03.765062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:39:03.765062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:39:03.765062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:39:03.765062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:39:03.765062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:39:03.765062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:39:03.765062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:03.765062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:39:03.765408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:03.765408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:39:03.765408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:03.765408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:39:03.765408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:03.765408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:39:03.765408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:03.765408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:39:03.765408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:39:03.765408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:03.765408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:39:03.765408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:03.765408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:39:03.765408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:39:03.765827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:03.765827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:39:03.765827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:03.765827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:39:03.766213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:03.766213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:39:03.766213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:39:03.766213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:39:03.766213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:39:03.766213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:39:03.766213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:39:03.766213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:39:03.766213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:39:03.766213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:39:03.766213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:39:03.766213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:39:03.766213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:39:03.766213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:39:03.766213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:39:03.766213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:39:03.766213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:39:03.766697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:39:03.766697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:39:03.766697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:39:03.766697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:03.766697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:39:03.766697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:03.766697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:39:03.766697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:03.766697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:39:03.766697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:03.766697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:39:03.766697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:03.766697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:39:03.766697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:39:03.766697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:03.766697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:39:03.766697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:03.766697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:39:03.766697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:39:03.767237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:03.767237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:39:03.767237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:03.767237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:39:03.768835 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:39:03.769114 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:39:03.769215 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:39:03.769364 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:39:03.769832 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:39:03.770178 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:39:03.770398 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:03.771183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:39:03.771183 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:03.771183 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:39:03.771183 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:39:03.771183 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:03.771183 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:39:03.771183 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:03.771183 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:39:03.772051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:03.772051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:39:03.772051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:39:03.772051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:39:03.772051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:39:03.772051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:39:03.772051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:39:03.772051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:03.772051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:39:03.772051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:03.772051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:39:03.772051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:03.772427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:39:03.772427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:03.772427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:39:03.772427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:03.772427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:39:03.772427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:39:03.772427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:03.772427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:39:03.772427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:03.772427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:39:03.772427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:39:03.772427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:03.772427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:39:03.772783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:03.772783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:39:03.773075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:03.773075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:39:03.773075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:39:03.773075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:39:03.773075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:39:03.773075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:39:03.773075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:39:03.773075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:39:03.773075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:39:03.773075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:39:03.773075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:39:03.773075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:39:03.773075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:39:03.773075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:39:03.773075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:39:03.773075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:39:03.773533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:39:03.773533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:39:03.773533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:03.773533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:39:03.773533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:03.773533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:39:03.773533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:03.773533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:39:03.773533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:03.773533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:39:03.773533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:03.773533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:39:03.773533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:39:03.773533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:03.773533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:39:03.773533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:03.773533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:39:03.773533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:39:03.773533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:03.774083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:39:03.774083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:03.774083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:39:08.596455 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:39:08.597519 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:39:08.597519 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:39:08.598793 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:39:08.634358 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:39:08.634771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:08.635457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:39:08.635457 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:08.635457 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:39:08.635457 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:39:08.635457 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:08.635457 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:39:08.635457 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:08.635457 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:39:08.636400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:08.636400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:39:08.636400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:39:08.636400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:39:08.636400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:39:08.636400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:39:08.636400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:39:08.636400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:08.636400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:39:08.636400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:08.636400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:39:08.636400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:08.637067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:39:08.637067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:08.637067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:39:08.637067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:08.637067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:39:08.637067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:39:08.637067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:08.637067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:39:08.637067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:08.637067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:39:08.637067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:39:08.637067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:08.637067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:39:08.637469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:08.637469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:39:08.637469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:08.637469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:39:08.637469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:39:08.637469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:39:08.637469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:39:08.637469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:39:08.637469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:39:08.637469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:39:08.637469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:39:08.637469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:39:08.637469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:39:08.637469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:39:08.637469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:39:08.637976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:39:08.637976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:39:08.637976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:39:08.637976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:39:08.637976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:39:08.637976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:08.637976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:39:08.637976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:08.637976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:39:08.637976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:08.637976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:39:08.637976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:08.637976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:39:08.637976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:08.637976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:39:08.637976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:39:08.637976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:08.637976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:39:08.637976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:08.638610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:39:08.638610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:39:08.638610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:08.638610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:39:08.638610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:08.638610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:39:08.745975 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:39:08.746663 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:39:08.746956 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:39:08.747577 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:39:08.747709 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:39:08.747830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:08.748661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:39:08.748661 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:08.748661 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:39:08.748661 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:39:08.748661 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:08.748661 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:39:08.748661 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:08.748661 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:39:08.750683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:08.750683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:39:08.750683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:39:08.750683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:39:08.750683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:39:08.750683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:39:08.750683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:39:08.750683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:08.750683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:39:08.750683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:08.750683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:39:08.750683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:08.751105 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:39:08.751105 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:08.751105 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:39:08.751105 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:08.751105 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:39:08.751105 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:39:08.751105 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:08.751105 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:39:08.751105 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:08.751105 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:39:08.751105 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:39:08.751105 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:08.751105 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:39:08.751423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:08.751423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:39:08.752086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:08.752086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:39:08.752086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:39:08.752086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:39:08.752086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:39:08.752086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:39:08.752086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:39:08.752086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:39:08.752086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:39:08.752086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:39:08.752086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:39:08.752086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:39:08.752086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:39:08.752086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:39:08.752086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:39:08.752086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:39:08.752524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:39:08.752524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:39:08.752524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:08.752524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:39:08.752524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:08.752524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:39:08.752524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:08.752524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:39:08.752524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:08.752524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:39:08.752524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:08.752524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:39:08.752524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:39:08.752524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:08.752524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:39:08.752524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:08.752524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:39:08.752524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:39:08.752524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:08.753100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:39:08.753100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:08.753100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:39:08.754592 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:39:08.754956 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:39:08.755075 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:39:08.755208 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:39:08.756002 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:39:08.756184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:08.756635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:39:08.756635 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:08.756635 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:39:08.756635 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:39:08.756635 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:08.756635 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:39:08.756635 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:08.756635 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:39:08.757565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:08.757565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:39:08.757565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:39:08.757565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:39:08.757565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:08.757565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:39:08.757565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:08.757565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:39:08.757565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:08.757565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:39:08.757565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:08.757565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:39:08.757565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:08.757565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:39:08.757565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:39:08.758029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:08.758029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:39:08.758029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:08.758029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:39:08.758029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:39:08.758029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:08.758029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:39:08.758029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:08.758029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:39:08.758732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:08.758732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:39:08.758732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:39:08.758732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:39:08.758732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:39:08.758732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:39:08.758732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:39:08.758732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:39:08.758732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:39:08.758732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:39:08.758732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:08.758732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:39:08.759103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:08.759103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:39:08.759103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:08.759103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:39:08.759103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:08.759103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:39:08.759103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:08.759103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:39:08.759103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:39:08.759103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:08.759103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:39:08.759103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:08.759103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:39:08.759103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:39:08.759432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:08.759432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:39:08.759432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:08.759432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:39:08.760391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:08.760391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:39:08.760391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:39:08.760391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:39:08.760391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:39:08.760391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:39:08.760391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:39:08.760391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:39:08.760391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:39:08.760391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:39:08.760391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:39:08.760391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:39:08.760391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:39:08.760391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:39:08.760391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:39:08.760391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:39:08.760841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:39:08.760841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:39:08.760841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:39:08.760841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:39:08.760841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:39:08.760841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:08.760841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:39:08.760841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:08.760841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:39:08.760841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:08.760841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:39:08.760841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:08.760841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:39:08.760841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:08.760841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:39:08.760841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:39:08.760841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:08.760841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:39:08.761347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:08.761347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:39:08.761347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:39:08.761347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:08.761347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:39:08.761347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:08.761347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:39:08.761528 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:39:08.761766 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:39:08.761945 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:39:08.762114 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:39:08.762391 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:39:08.762607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:08.763052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:39:08.763052 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:08.763052 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:39:08.763052 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:39:08.763052 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:08.763052 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:39:08.763052 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:08.763052 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:39:08.763921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:08.763921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:39:08.763921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:39:08.763921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:39:08.763921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:08.763921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:39:08.763921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:08.763921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:39:08.763921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:08.763921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:39:08.763921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:08.763921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:39:08.763921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:08.763921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:39:08.763921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:39:08.764306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:08.764306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:39:08.764306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:08.764306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:39:08.764306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:39:08.764306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:08.764306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:39:08.764306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:08.764306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:39:08.765140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:08.765140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:39:08.765140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:39:08.765140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:39:08.765140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:39:08.765140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:39:08.765140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:39:08.765140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:39:08.765140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:39:08.765140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:08.765140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:39:08.765452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:08.765452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:39:08.765452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:08.765452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:39:08.765452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:08.765452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:39:08.765452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:08.765452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:39:08.765452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:39:08.765452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:08.765452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:39:08.765452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:08.765452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:39:08.765452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:39:08.765761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:08.765761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:39:08.765761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:08.765761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:39:08.766948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:08.766948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:39:08.766948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:39:08.766948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:39:08.766948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:39:08.766948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:39:08.766948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:39:08.766948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:39:08.766948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:39:08.766948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:39:08.766948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:39:08.766948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:39:08.766948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:39:08.766948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:39:08.766948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:39:08.766948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:39:08.766948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:39:08.767518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:39:08.767518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:39:08.767518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:39:08.767518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:08.767518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:39:08.767518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:08.767518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:39:08.767518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:08.767518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:39:08.767518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:08.767518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:39:08.767518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:08.767518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:39:08.767518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:39:08.767518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:08.767518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:39:08.767518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:08.767518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:39:08.767518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:39:08.768160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:08.768160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:39:08.768160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:08.768160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:39:08.769373 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:39:08.769727 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:39:08.769912 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:39:08.770166 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:39:08.770700 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:39:08.771292 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:39:08.771632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:08.772556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:39:08.772556 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:08.772556 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:39:08.772556 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:39:08.772556 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:08.772556 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:39:08.772556 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:08.772556 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:39:08.773565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:08.773565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:39:08.773565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:39:08.773565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:39:08.773565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:39:08.773565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:39:08.773565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:39:08.773565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:08.773565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:39:08.773565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:08.773565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:39:08.773565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:08.774118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:39:08.774118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:08.774118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:39:08.774118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:08.774118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:39:08.774118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:39:08.774118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:08.774118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:39:08.774118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:08.774118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:39:08.774118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:39:08.774118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:08.774118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:39:08.774569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:08.774569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:39:08.775219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:08.775219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:39:08.775219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:39:08.775219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:39:08.775219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:39:08.775219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:39:08.775219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:39:08.775219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:39:08.775219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:39:08.775219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:39:08.775219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:39:08.775219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:39:08.775219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:39:08.775219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:39:08.775219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:39:08.775219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:39:08.775722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:39:08.775722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:39:08.775722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:08.775722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:39:08.775722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:08.775722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:39:08.775722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:08.775722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:39:08.775722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:08.775722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:39:08.775722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:08.775722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:39:08.775722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:39:08.775722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:08.775722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:39:08.775722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:08.775722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:39:08.775722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:39:08.775722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:08.776288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:39:08.776288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:08.776288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:39:13.597155 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:39:13.597967 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:39:13.597967 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:39:13.598722 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:39:13.637360 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:39:13.637894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:13.638846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:39:13.638846 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:13.638846 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:39:13.638846 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:39:13.638846 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:13.638846 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:39:13.638846 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:13.638846 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:39:13.639625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:13.639625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:39:13.639625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:39:13.639625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:39:13.639625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:39:13.639625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:39:13.639625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:39:13.639625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:13.639625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:39:13.639625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:13.639625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:39:13.639625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:13.640350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:39:13.640350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:13.640350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:39:13.640350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:13.640350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:39:13.640350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:39:13.640350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:13.640350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:39:13.640350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:13.640350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:39:13.640350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:39:13.640350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:13.640350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:39:13.641078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:13.641078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:39:13.641078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:13.641078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:39:13.641078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:39:13.641078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:39:13.641078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:39:13.641078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:39:13.641078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:39:13.641078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:39:13.641078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:39:13.641078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:39:13.641078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:39:13.641078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:39:13.641078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:39:13.641899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:39:13.641899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:39:13.641899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:39:13.641899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:39:13.641899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:39:13.641899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:13.641899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:39:13.641899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:13.641899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:39:13.641899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:13.641899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:39:13.641899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:13.641899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:39:13.641899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:13.641899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:39:13.641899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:39:13.641899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:13.641899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:39:13.641899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:13.645269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:39:13.645269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:39:13.645269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:13.645269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:39:13.645269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:13.645269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:39:13.746828 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:39:13.747475 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:39:13.747649 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:39:13.748777 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:39:13.749147 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:39:13.749407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:13.750284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:39:13.750284 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:13.750284 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:39:13.750284 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:39:13.750284 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:13.750284 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:39:13.750284 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:13.750284 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:39:13.751473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:13.751473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:39:13.751473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:39:13.751473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:39:13.751473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:39:13.751473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:39:13.751473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:39:13.751473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:13.751473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:39:13.751473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:13.751473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:39:13.751473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:13.753190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:39:13.753190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:13.753190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:39:13.753190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:13.753190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:39:13.753190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:39:13.753190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:13.753190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:39:13.753190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:13.753190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:39:13.753190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:39:13.753190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:13.753190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:39:13.754396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:13.754396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:39:13.754396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:13.754396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:39:13.754396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:39:13.754396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:39:13.754396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:39:13.754396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:39:13.754396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:39:13.754396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:39:13.754396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:39:13.754396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:39:13.754396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:39:13.754396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:39:13.754396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:39:13.755248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:39:13.755248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:39:13.755248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:39:13.755248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:39:13.755248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:39:13.755248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:13.755248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:39:13.755248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:13.755248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:39:13.755248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:13.755248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:39:13.755248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:13.755248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:39:13.755248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:13.755248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:39:13.755248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:39:13.755248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:13.755248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:39:13.755248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:13.756919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:39:13.756919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:39:13.756919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:13.756919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:39:13.756919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:13.756919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:39:13.756919 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:39:13.756919 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:39:13.756919 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:39:13.756919 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:39:13.757918 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:39:13.757918 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:13.757918 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:39:13.757918 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:13.757918 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:39:13.757918 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:39:13.757918 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:13.757918 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:39:13.757918 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:13.757918 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:39:13.758318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:13.758318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:39:13.758318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:39:13.758318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:39:13.758318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:13.758318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:39:13.758318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:13.758318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:39:13.758318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:13.758318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:39:13.758318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:13.758318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:39:13.758318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:13.758318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:39:13.758318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:39:13.759044 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:13.759044 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:39:13.759044 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:13.759044 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:39:13.759044 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:39:13.759044 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:13.759044 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:39:13.759044 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:13.759044 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:39:13.759478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:13.759478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:39:13.759478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:39:13.759478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:39:13.759478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:39:13.759478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:39:13.759478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:39:13.759478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:39:13.759478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:39:13.759478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:39:13.759478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:13.759478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:39:13.760531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:13.760531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:39:13.760531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:13.760531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:39:13.760531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:13.760531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:39:13.760531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:13.760531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:39:13.760531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:39:13.760531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:13.760531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:39:13.760531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:13.760531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:39:13.760531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:39:13.761127 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:13.761127 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:39:13.761127 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:13.761127 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:39:13.761127 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:13.761127 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:39:13.761127 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:39:13.761127 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:39:13.761127 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:39:13.761127 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:39:13.761127 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:39:13.761127 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:39:13.761127 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:39:13.761127 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:39:13.761552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:39:13.761552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:39:13.761552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:39:13.761552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:39:13.761552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:39:13.761552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:39:13.761552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:39:13.761552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:39:13.761552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:39:13.761552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:39:13.761552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:39:13.761552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:13.761552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:39:13.761552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:13.761552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:39:13.761552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:13.761552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:39:13.761552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:13.761552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:39:13.761968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:13.761968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:39:13.761968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:39:13.761968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:13.761968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:39:13.761968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:13.761968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:39:13.761968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:39:13.761968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:13.761968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:39:13.761968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:13.761968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:39:13.761968 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:39:13.762243 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:39:13.762408 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:39:13.762471 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:39:13.763307 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:39:13.763473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:13.764323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:39:13.764323 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:13.764323 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:39:13.764323 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:39:13.764323 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:13.764323 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:39:13.764323 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:13.764323 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:39:13.765284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:13.765284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:39:13.765284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:39:13.765284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:39:13.765284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:13.765284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:39:13.765284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:13.765284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:39:13.765284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:13.765284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:39:13.765284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:13.765284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:39:13.765284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:13.765284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:39:13.765284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:39:13.765931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:13.765931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:39:13.765931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:13.765931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:39:13.765931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:39:13.765931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:13.765931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:39:13.765931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:13.765931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:39:13.766280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:13.766280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:39:13.766280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:39:13.766280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:39:13.766280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:39:13.766280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:39:13.766280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:39:13.766280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:39:13.766280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:39:13.766280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:13.766280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:39:13.766651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:13.766651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:39:13.766651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:13.766651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:39:13.766651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:13.766651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:39:13.766651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:13.766651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:39:13.766651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:39:13.766651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:13.766651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:39:13.766651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:13.766651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:39:13.766651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:39:13.767021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:13.767021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:39:13.767021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:13.767021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:39:13.767529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:13.767529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:39:13.767529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:39:13.767529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:39:13.767529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:39:13.767529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:39:13.767529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:39:13.767529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:39:13.767529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:39:13.767529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:39:13.767529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:39:13.767529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:39:13.767529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:39:13.767529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:39:13.767529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:39:13.767529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:39:13.767529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:39:13.768818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:39:13.768818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:39:13.768818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:39:13.768818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:13.768818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:39:13.768818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:13.768818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:39:13.768818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:13.768818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:39:13.768818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:13.768818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:39:13.768818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:13.768818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:39:13.768818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:39:13.768818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:13.768818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:39:13.768818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:13.768818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:39:13.768818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:39:13.769927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:13.769927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:39:13.769927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:13.769927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:39:13.769927 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:39:13.770265 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:39:13.770371 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:39:13.770547 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:39:13.770940 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:39:13.771175 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:39:13.771319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:13.771945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:39:13.771945 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:13.771945 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:39:13.771945 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:39:13.771945 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:13.771945 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:39:13.771945 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:13.771945 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:39:13.772870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:13.772870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:39:13.772870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:39:13.772870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:39:13.772870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:39:13.772870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:39:13.772870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:39:13.772870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:13.772870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:39:13.772870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:13.772870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:39:13.772870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:13.773225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:39:13.773225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:13.773225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:39:13.773225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:13.773225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:39:13.773225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:39:13.773225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:13.773225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:39:13.773225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:13.773225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:39:13.773225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:39:13.773225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:13.773225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:39:13.773557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:13.773557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:39:13.773952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:13.773952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:39:13.773952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:39:13.773952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:39:13.773952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:39:13.773952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:39:13.773952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:39:13.773952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:39:13.773952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:39:13.773952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:39:13.773952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:39:13.773952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:39:13.773952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:39:13.773952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:39:13.773952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:39:13.773952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:39:13.774887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:39:13.774887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:39:13.774887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:13.774887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:39:13.774887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:13.774887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:39:13.774887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:13.774887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:39:13.774887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:13.774887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:39:13.774887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:13.774887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:39:13.774887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:39:13.774887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:13.774887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:39:13.774887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:13.774887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:39:13.774887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:39:13.774887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:13.775588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:39:13.775588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:13.775588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:39:18.597909 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:39:18.599094 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:39:18.599094 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:39:18.600484 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:39:18.637968 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:39:18.638304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:18.638987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:39:18.638987 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:18.638987 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:39:18.638987 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:39:18.638987 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:18.638987 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:39:18.638987 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:18.638987 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:39:18.640438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:18.640438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:39:18.640438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:39:18.640438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:39:18.640438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:39:18.640438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:39:18.640438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:39:18.640438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:18.640438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:39:18.640438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:18.640438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:39:18.640438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:18.641211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:39:18.641211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:18.641211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:39:18.641211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:18.641211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:39:18.641211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:39:18.641211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:18.641211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:39:18.641211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:18.641211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:39:18.641211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:39:18.641211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:18.641211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:39:18.641884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:18.641884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:39:18.642654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:18.642654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:39:18.642654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:39:18.642654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:39:18.642654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:39:18.642654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:39:18.642654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:39:18.642654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:39:18.642654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:39:18.642654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:39:18.642654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:39:18.642654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:39:18.642654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:39:18.642654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:39:18.642654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:39:18.642654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:39:18.644873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:39:18.644873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:39:18.644873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:18.644873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:39:18.644873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:18.644873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:39:18.644873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:18.644873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:39:18.644873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:18.644873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:39:18.644873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:18.644873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:39:18.644873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:39:18.644873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:18.644873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:39:18.644873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:18.644873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:39:18.644873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:39:18.644873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:18.647087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:39:18.647087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:18.647087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:39:18.747602 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:39:18.748246 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:39:18.748440 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:39:18.749586 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:39:18.749941 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:39:18.750186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:18.751133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:39:18.751133 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:18.751133 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:39:18.751133 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:39:18.751133 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:18.751133 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:39:18.751133 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:18.751133 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:39:18.752737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:18.752737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:39:18.752737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:39:18.752737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:39:18.752737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:39:18.752737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:39:18.752737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:39:18.752737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:18.752737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:39:18.752737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:18.752737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:39:18.752737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:18.753506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:39:18.753506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:18.753506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:39:18.753506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:18.753506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:39:18.753506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:39:18.753506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:18.753506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:39:18.753506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:18.753506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:39:18.753506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:39:18.753506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:18.753506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:39:18.754273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:18.754273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:39:18.757294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:18.757294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:39:18.757294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:39:18.757294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:39:18.757294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:39:18.757294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:39:18.757294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:39:18.757294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:39:18.757294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:39:18.757294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:39:18.757294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:39:18.757294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:39:18.757294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:39:18.757294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:39:18.757294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:39:18.757294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:39:18.757900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:39:18.757900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:39:18.757900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:18.757900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:39:18.757900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:18.757900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:39:18.757900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:18.757900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:39:18.757900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:18.757900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:39:18.757900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:18.757900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:39:18.757900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:39:18.757900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:18.757900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:39:18.757900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:18.757900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:39:18.757900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:39:18.757900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:18.758442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:39:18.758442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:18.758442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:39:18.758442 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:39:18.758442 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:39:18.758442 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:39:18.758442 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:39:18.758949 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:39:18.759307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:18.760033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:39:18.760033 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:18.760033 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:39:18.760033 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:39:18.760033 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:18.760033 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:39:18.760033 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:18.760033 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:39:18.760617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:18.760617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:39:18.760617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:39:18.760617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:39:18.760617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:18.760617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:39:18.760617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:18.760617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:39:18.760617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:18.760617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:39:18.760617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:18.760617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:39:18.760617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:18.760617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:39:18.760617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:39:18.761162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:18.761162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:39:18.761162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:18.761162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:39:18.761162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:39:18.761162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:18.761162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:39:18.761162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:18.761162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:39:18.761445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:18.761445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:39:18.761445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:39:18.761445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:39:18.761445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:39:18.761445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:39:18.761445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:39:18.761445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:39:18.761445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:39:18.761445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:39:18.761445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:18.761445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:39:18.761840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:18.761840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:39:18.761840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:18.761840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:39:18.761840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:18.761840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:39:18.761840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:18.761840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:39:18.761840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:39:18.761840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:18.761840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:39:18.761840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:18.761840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:39:18.761840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:39:18.762222 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:18.762222 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:39:18.762222 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:18.762222 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:39:18.762222 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:18.762222 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:39:18.762222 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:39:18.762222 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:39:18.762222 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:39:18.762222 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:39:18.762222 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:39:18.762222 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:39:18.762222 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:39:18.762222 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:39:18.762627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:39:18.762627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:39:18.762627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:39:18.762627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:39:18.762627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:39:18.762627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:39:18.762627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:39:18.762627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:39:18.762627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:39:18.762627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:39:18.762627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:39:18.762627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:18.762627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:39:18.762627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:18.762627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:39:18.762627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:18.762627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:39:18.762627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:18.762627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:39:18.763237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:18.763237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:39:18.763237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:39:18.763237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:18.763237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:39:18.763237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:18.763237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:39:18.763237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:39:18.763237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:18.763237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:39:18.763237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:18.763237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:39:18.763237 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:39:18.763237 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:39:18.764000 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:39:18.764000 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:39:18.764000 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:39:18.764000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:18.764263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:39:18.764263 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:18.764263 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:39:18.764263 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:39:18.764263 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:18.764263 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:39:18.764263 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:18.764263 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:39:18.764683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:18.764683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:39:18.764683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:39:18.764683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:39:18.764683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:18.764683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:39:18.764683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:18.764683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:39:18.764683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:18.764683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:39:18.764683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:18.764683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:39:18.764683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:18.764683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:39:18.764683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:39:18.765415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:18.765415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:39:18.765415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:18.765415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:39:18.765415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:39:18.765415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:18.765415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:39:18.765415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:18.765415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:39:18.765415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:18.765415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:39:18.765415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:39:18.765415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:39:18.765415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:39:18.766094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:39:18.766094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:39:18.766094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:39:18.766094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:39:18.766094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:18.766094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:39:18.766094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:18.766094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:39:18.766094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:18.766094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:39:18.766094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:18.766094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:39:18.766094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:18.766421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:39:18.766421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:39:18.766421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:18.766421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:39:18.766421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:18.766421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:39:18.766421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:39:18.766421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:18.766421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:39:18.766421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:18.766421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:39:18.766421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:18.767351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:39:18.767351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:39:18.767351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:39:18.767351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:39:18.767351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:39:18.767351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:39:18.767351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:39:18.767351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:39:18.767351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:39:18.767351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:39:18.767351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:39:18.767351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:39:18.767351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:39:18.767351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:39:18.767351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:39:18.767351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:39:18.767351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:39:18.767351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:39:18.767351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:39:18.768114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:18.768114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:39:18.768114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:18.768114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:39:18.768114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:18.768114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:39:18.768114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:18.768114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:39:18.768114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:18.768114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:39:18.768114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:39:18.768114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:18.768114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:39:18.768114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:18.768114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:39:18.768114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:39:18.768114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:18.768114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:39:18.768114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:18.768114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:39:18.770460 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:39:18.770954 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:39:18.771263 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:39:18.771549 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:39:18.772072 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:39:18.772426 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:39:18.772649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:18.773429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:39:18.773429 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:18.773429 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:39:18.773429 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:39:18.773429 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:18.773429 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:39:18.773429 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:18.773429 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:39:18.774870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:18.774870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:39:18.774870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:39:18.774870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:39:18.774870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:39:18.774870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:39:18.774870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:39:18.774870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:18.774870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:39:18.774870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:18.774870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:39:18.774870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:18.775354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:39:18.775354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:18.775354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:39:18.775354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:18.775354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:39:18.775354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:39:18.775354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:18.775354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:39:18.775354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:18.775354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:39:18.775354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:39:18.775354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:18.775354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:39:18.775810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:18.775810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:39:18.776499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:18.776499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:39:18.776499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:39:18.776499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:39:18.776499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:39:18.776499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:39:18.776499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:39:18.776499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:39:18.776499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:39:18.776499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:39:18.776499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:39:18.776499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:39:18.776499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:39:18.776499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:39:18.776499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:39:18.776499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:39:18.776946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:39:18.776946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:39:18.776946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:18.776946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:39:18.776946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:18.776946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:39:18.776946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:18.776946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:39:18.776946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:18.776946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:39:18.776946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:18.776946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:39:18.776946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:39:18.776946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:18.776946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:39:18.776946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:18.776946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:39:18.776946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:39:18.776946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:18.777441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:39:18.777441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:18.777441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:39:23.598416 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:39:23.599851 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:39:23.599851 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:39:23.600464 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:39:23.635796 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:39:23.635796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:23.636533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:39:23.636533 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:23.636533 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:39:23.636533 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:39:23.636533 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:23.636533 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:39:23.636533 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:23.636533 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:39:23.637321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:23.637321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:39:23.637321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:39:23.637321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:39:23.637321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:39:23.637321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:39:23.637321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:39:23.637321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:23.637321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:39:23.637321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:23.637321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:39:23.637321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:23.638476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:39:23.638476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:23.638476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:39:23.638476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:23.638476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:39:23.638476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:39:23.638476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:23.638476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:39:23.638476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:23.638476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:39:23.638476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:39:23.638476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:23.638476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:39:23.639521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:23.639521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:39:23.639521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:23.639521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:39:23.639521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:39:23.639521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:39:23.639521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:39:23.639521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:39:23.639521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:39:23.639521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:39:23.639521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:39:23.639521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:39:23.639521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:39:23.639521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:39:23.639521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:39:23.640267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:39:23.640267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:39:23.640267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:39:23.640267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:39:23.640267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:39:23.640267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:23.640267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:39:23.640267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:23.640267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:39:23.640267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:23.640267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:39:23.640267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:23.640267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:39:23.640267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:23.640267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:39:23.640267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:39:23.640267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:23.640267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:39:23.640267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:23.640849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:39:23.640849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:39:23.640849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:23.640849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:39:23.640849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:23.640849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:39:23.748360 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:39:23.749043 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:39:23.749168 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:39:23.750830 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:39:23.751081 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:39:23.751290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:23.752002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:39:23.752002 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:23.752002 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:39:23.752002 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:39:23.752002 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:23.752002 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:39:23.752002 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:23.752002 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:39:23.752997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:23.752997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:39:23.752997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:39:23.752997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:39:23.752997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:39:23.752997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:39:23.752997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:39:23.752997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:23.752997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:39:23.752997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:23.752997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:39:23.752997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:23.753708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:39:23.753708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:23.753708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:39:23.753708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:23.753708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:39:23.753708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:39:23.753708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:23.753708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:39:23.753708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:23.753708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:39:23.753708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:39:23.753708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:23.753708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:39:23.754533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:23.754533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:39:23.754533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:23.754533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:39:23.754533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:39:23.754533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:39:23.754533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:39:23.754533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:39:23.754533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:39:23.754533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:39:23.754533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:39:23.754533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:39:23.754533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:39:23.754533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:39:23.754533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:39:23.755339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:39:23.755339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:39:23.755339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:39:23.755339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:39:23.755339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:39:23.755339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:23.755339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:39:23.755339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:23.755339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:39:23.755339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:23.755339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:39:23.755339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:23.755339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:39:23.755339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:23.755339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:39:23.755339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:39:23.755339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:23.755339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:39:23.755339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:23.756458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:39:23.756458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:39:23.756458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:23.756458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:39:23.756458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:23.756458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:39:23.758141 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:39:23.758567 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:39:23.758780 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:39:23.759007 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:39:23.759383 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:39:23.759592 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:23.760124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:39:23.760124 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:23.760124 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:39:23.760124 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:39:23.760124 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:23.760124 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:39:23.760124 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:23.760124 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:39:23.761337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:23.761337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:39:23.761337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:39:23.761337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:39:23.761337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:23.761337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:39:23.761337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:23.761337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:39:23.761337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:23.761337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:39:23.761337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:23.761337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:39:23.761337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:23.761337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:39:23.761337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:39:23.762540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:23.762540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:39:23.762540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:23.762540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:39:23.762540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:39:23.762540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:23.762540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:39:23.762540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:23.762540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:39:23.763057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:23.763057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:39:23.763057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:39:23.763057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:39:23.763057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:39:23.763057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:39:23.763057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:39:23.763057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:39:23.763057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:39:23.763057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:39:23.763057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:23.763057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:39:23.763451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:23.763451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:39:23.763451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:23.763451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:39:23.763451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:23.763451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:39:23.763451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:23.763451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:39:23.763451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:39:23.763451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:23.763451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:39:23.763451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:23.763451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:39:23.763451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:39:23.764101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:23.764101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:39:23.764101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:23.764101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:39:23.765350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:23.765350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:39:23.765350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:39:23.765350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:39:23.765350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:39:23.765350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:39:23.765350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:39:23.765350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:39:23.765350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:39:23.765350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:39:23.765350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:39:23.765350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:39:23.765350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:39:23.765350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:39:23.765350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:39:23.765350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:39:23.765989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:39:23.765989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:39:23.765989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:39:23.765989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:39:23.765989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:39:23.765989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:23.765989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:39:23.765989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:23.765989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:39:23.765989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:23.765989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:39:23.765989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:23.765989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:39:23.765989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:23.765989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:39:23.765989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:39:23.765989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:23.765989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:39:23.766663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:23.766663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:39:23.766663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:39:23.766663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:23.766663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:39:23.766663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:23.766663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:39:23.766663 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:39:23.766663 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:39:23.766663 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:39:23.767031 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:39:23.767491 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:39:23.767893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:23.768539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:39:23.768539 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:23.768539 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:39:23.768539 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:39:23.768539 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:23.768539 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:39:23.768539 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:23.768539 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:39:23.769334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:23.769334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:39:23.769334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:39:23.769334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:39:23.769334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:23.769334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:39:23.769334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:23.769334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:39:23.769334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:23.769334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:39:23.769334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:23.769334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:39:23.769334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:23.769334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:39:23.769334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:39:23.769813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:23.769813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:39:23.769813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:23.769813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:39:23.769813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:39:23.769813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:23.769813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:39:23.769813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:23.769813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:39:23.770300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:23.770300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:39:23.770300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:39:23.770300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:39:23.770300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:39:23.770300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:39:23.770300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:39:23.770300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:39:23.770300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:39:23.770300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:23.770300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:39:23.770769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:23.770769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:39:23.770769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:23.770769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:39:23.770769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:23.770769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:39:23.770769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:23.770769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:39:23.770769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:39:23.770769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:23.770769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:39:23.770769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:23.770769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:39:23.770769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:39:23.771199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:23.771199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:39:23.771199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:23.771199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:39:23.771973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:23.771973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:39:23.771973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:39:23.771973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:39:23.771973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:39:23.771973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:39:23.771973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:39:23.771973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:39:23.771973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:39:23.771973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:39:23.771973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:39:23.771973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:39:23.771973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:39:23.771973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:39:23.771973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:39:23.771973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:39:23.771973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:39:23.772605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:39:23.772605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:39:23.772605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:39:23.772605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:23.772605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:39:23.772605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:23.772605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:39:23.772605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:23.772605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:39:23.772605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:23.772605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:39:23.772605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:23.772605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:39:23.772605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:39:23.772605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:23.772605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:39:23.772605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:23.772605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:39:23.772605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:39:23.773329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:23.773329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:39:23.773329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:23.773329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:39:23.773329 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:39:23.773329 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:39:23.773329 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:39:23.773329 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:39:23.773669 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:39:23.773900 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:39:23.774105 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:23.774936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:39:23.774936 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:23.774936 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:39:23.774936 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:39:23.774936 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:23.774936 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:39:23.774936 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:23.774936 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:39:23.775768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:23.775768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:39:23.775768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:39:23.775768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:39:23.775768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:39:23.775768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:39:23.775768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:39:23.775768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:23.775768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:39:23.775768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:23.775768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:39:23.775768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:23.776497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:39:23.776497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:23.776497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:39:23.776497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:23.776497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:39:23.776497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:39:23.776497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:23.776497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:39:23.776497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:23.776497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:39:23.776497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:39:23.776497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:23.776497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:39:23.777113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:23.777113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:39:23.777113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:23.777113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:39:23.777113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:39:23.777113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:39:23.777113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:39:23.777113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:39:23.777113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:39:23.777113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:39:23.777113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:39:23.777113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:39:23.777113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:39:23.777113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:39:23.777113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:39:23.777949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:39:23.777949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:39:23.777949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:39:23.777949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:39:23.777949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:39:23.777949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:23.777949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:39:23.777949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:23.777949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:39:23.777949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:23.777949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:39:23.777949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:23.777949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:39:23.777949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:23.777949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:39:23.777949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:39:23.777949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:23.777949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:39:23.777949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:23.778853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:39:23.778853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:39:23.778853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:23.778853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:39:23.778853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:23.778853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:39:28.599084 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:39:28.600115 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:39:28.600115 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:39:28.601458 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:39:28.638064 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:39:28.638356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:28.639271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:39:28.639271 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:28.639271 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:39:28.639271 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:39:28.639271 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:28.639271 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:39:28.639271 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:28.639271 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:39:28.640909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:28.640909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:39:28.640909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:39:28.640909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:39:28.640909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:39:28.640909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:39:28.640909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:39:28.640909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:28.640909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:39:28.640909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:28.640909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:39:28.640909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:28.641768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:39:28.641768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:28.641768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:39:28.641768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:28.641768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:39:28.641768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:39:28.641768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:28.641768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:39:28.641768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:28.641768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:39:28.641768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:39:28.641768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:28.641768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:39:28.642395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:28.642395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:39:28.642395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:28.642395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:39:28.642395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:39:28.642395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:39:28.642395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:39:28.642395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:39:28.642395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:39:28.642395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:39:28.642395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:39:28.642395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:39:28.642395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:39:28.642395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:39:28.642395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:39:28.643160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:39:28.643160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:39:28.643160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:39:28.643160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:39:28.643160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:39:28.643160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:28.643160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:39:28.643160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:28.643160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:39:28.643160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:28.643160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:39:28.643160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:28.643160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:39:28.643160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:28.643160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:39:28.643160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:39:28.643160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:28.643160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:39:28.643160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:28.644069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:39:28.644069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:39:28.644069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:28.644069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:39:28.644069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:28.644069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:39:28.749146 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:39:28.749814 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:39:28.749915 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:39:28.751108 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:39:28.751483 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:39:28.751724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:28.752648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:39:28.752648 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:28.752648 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:39:28.752648 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:39:28.752648 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:28.752648 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:39:28.752648 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:28.752648 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:39:28.753877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:28.753877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:39:28.753877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:39:28.753877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:39:28.753877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:39:28.753877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:39:28.753877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:39:28.753877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:28.753877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:39:28.753877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:28.753877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:39:28.753877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:28.754456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:39:28.754456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:28.754456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:39:28.754456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:28.754456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:39:28.754456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:39:28.754456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:28.754456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:39:28.754456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:28.754456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:39:28.754456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:39:28.754456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:28.754456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:39:28.755051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:28.755051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:39:28.755051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:28.755051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:39:28.755051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:39:28.755051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:39:28.755051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:39:28.755051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:39:28.755051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:39:28.755051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:39:28.755051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:39:28.755051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:39:28.755051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:39:28.755051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:39:28.755051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:39:28.755838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:39:28.755838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:39:28.755838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:39:28.755838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:39:28.755838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:39:28.755838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:28.755838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:39:28.755838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:28.755838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:39:28.755838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:28.755838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:39:28.755838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:28.755838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:39:28.755838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:28.755838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:39:28.755838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:39:28.755838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:28.755838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:39:28.755838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:28.756684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:39:28.756684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:39:28.756684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:28.756684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:39:28.756684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:28.756684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:39:28.758637 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:39:28.758978 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:39:28.759075 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:39:28.759217 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:39:28.759713 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:39:28.760012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:28.763158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:39:28.763158 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:28.763158 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:39:28.763158 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:39:28.763158 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:28.763158 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:39:28.763158 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:28.763158 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:39:28.764142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:28.764142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:39:28.764142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:39:28.764142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:39:28.764142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:28.764142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:39:28.764142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:28.764142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:39:28.764142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:28.764142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:39:28.764142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:28.764142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:39:28.764142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:28.764142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:39:28.764142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:39:28.764673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:28.764673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:39:28.764673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:28.764673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:39:28.764673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:39:28.764673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:28.764673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:39:28.764673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:28.764673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:39:28.765341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:28.765341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:39:28.765341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:39:28.765341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:39:28.765341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:39:28.765341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:39:28.765341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:39:28.765341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:39:28.765341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:39:28.765341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:39:28.765341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:28.765341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:39:28.765708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:28.765708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:39:28.765708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:28.765708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:39:28.765708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:28.765708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:39:28.765708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:28.765708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:39:28.765708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:39:28.765708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:28.765708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:39:28.765708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:28.765708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:39:28.765708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:39:28.766166 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:28.766166 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:39:28.766166 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:28.766166 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:39:28.767589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:28.767589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:39:28.767589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:39:28.767589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:39:28.767589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:39:28.767589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:39:28.767589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:39:28.767589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:39:28.767589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:39:28.767589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:39:28.767589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:39:28.767589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:39:28.767589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:39:28.767589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:39:28.767589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:39:28.767589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:39:28.768062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:39:28.768062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:39:28.768062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:39:28.768062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:39:28.768062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:39:28.768062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:28.768062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:39:28.768062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:28.768062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:39:28.768062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:28.768062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:39:28.768062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:28.768062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:39:28.768062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:28.768062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:39:28.768062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:39:28.768062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:28.768062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:39:28.768596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:28.768596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:39:28.768596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:39:28.768596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:28.768596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:39:28.768596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:28.768596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:39:28.768596 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:39:28.768596 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:39:28.768887 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:39:28.768887 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:39:28.769343 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:39:28.769765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:28.770487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:39:28.770487 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:28.770487 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:39:28.770487 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:39:28.770487 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:28.770487 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:39:28.770487 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:28.770487 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:39:28.771370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:28.771370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:39:28.771370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:39:28.771370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:39:28.771370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:28.771370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:39:28.771370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:28.771370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:39:28.771370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:28.771370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:39:28.771370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:28.771370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:39:28.771370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:28.771370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:39:28.771370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:39:28.771800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:28.771800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:39:28.771800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:28.771800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:39:28.771800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:39:28.771800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:28.771800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:39:28.771800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:28.771800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:39:28.772342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:28.772342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:39:28.772342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:39:28.772342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:39:28.772342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:39:28.772342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:39:28.772342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:39:28.772342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:39:28.772342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:39:28.772342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:28.772342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:39:28.772683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:28.772683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:39:28.772683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:28.772683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:39:28.772683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:28.772683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:39:28.772683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:28.772683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:39:28.772683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:39:28.772683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:28.772683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:39:28.772683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:28.772683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:39:28.772683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:39:28.773064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:28.773064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:39:28.773064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:28.773064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:39:28.773615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:28.773615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:39:28.773615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:39:28.773615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:39:28.773615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:39:28.773615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:39:28.773615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:39:28.773615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:39:28.773615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:39:28.773615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:39:28.773615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:39:28.773615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:39:28.773615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:39:28.773615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:39:28.773615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:39:28.773615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:39:28.773615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:39:28.774116 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:39:28.774116 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:39:28.774116 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:39:28.774116 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:28.774116 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:39:28.774116 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:28.774116 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:39:28.774116 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:28.774116 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:39:28.774116 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:28.774116 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:39:28.774116 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:28.774116 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:39:28.774116 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:39:28.774116 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:28.774116 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:39:28.774116 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:28.774116 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:39:28.774116 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:39:28.774622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:28.774622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:39:28.774622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:28.774622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:39:28.774622 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:39:28.774622 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:39:28.774622 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:39:28.774622 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:39:28.774973 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:39:28.775218 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:39:28.775381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:28.777053 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:39:28.777053 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:28.777053 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:39:28.777053 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:39:28.777053 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:28.777053 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:39:28.777053 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:28.777053 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:39:28.778136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:28.778136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:39:28.778136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:39:28.778136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:39:28.778136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:39:28.778136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:39:28.778136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:39:28.778136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:28.778136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:39:28.778136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:28.778136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:39:28.778136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:28.778493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:39:28.778493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:28.778493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:39:28.778493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:28.778493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:39:28.778493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:39:28.778493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:28.778493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:39:28.778493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:28.778493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:39:28.778493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:39:28.778493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:28.778493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:39:28.778866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:28.778866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:39:28.779126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:28.779126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:39:28.779126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:39:28.779126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:39:28.779126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:39:28.779126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:39:28.779126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:39:28.779126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:39:28.779126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:39:28.779126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:39:28.779126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:39:28.779126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:39:28.779126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:39:28.779126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:39:28.779126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:39:28.779126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:39:28.779566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:39:28.779566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:39:28.779566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:28.779566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:39:28.779566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:28.779566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:39:28.779566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:28.779566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:39:28.779566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:28.779566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:39:28.779566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:28.779566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:39:28.779566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:39:28.779566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:28.779566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:39:28.779566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:28.779566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:39:28.779566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:39:28.779566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:28.780139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:39:28.780139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:28.780139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:39:33.599664 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:39:33.600581 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:39:33.600691 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:39:33.602182 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:39:33.640261 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:39:33.640646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:33.641388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:39:33.641388 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:33.641388 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:39:33.641388 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:39:33.641388 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:33.641388 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:39:33.641388 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:33.641388 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:39:33.642288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:33.642288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:39:33.642288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:39:33.642288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:39:33.642288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:39:33.642288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:39:33.642288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:39:33.642288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:33.642288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:39:33.642288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:33.642288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:39:33.642288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:33.642851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:39:33.642851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:33.642851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:39:33.642851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:33.642851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:39:33.642851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:39:33.642851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:33.642851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:39:33.642851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:33.642851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:39:33.642851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:39:33.642851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:33.642851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:39:33.643241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:33.643241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:39:33.643442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:33.643442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:39:33.643442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:39:33.643442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:39:33.643442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:39:33.643442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:39:33.643442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:39:33.643442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:39:33.643442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:39:33.643442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:39:33.643442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:39:33.643442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:39:33.643442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:39:33.643442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:39:33.643442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:39:33.643442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:39:33.643822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:39:33.643822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:39:33.643822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:33.643822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:39:33.643822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:33.643822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:39:33.643822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:33.643822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:39:33.643822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:33.643822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:39:33.643822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:33.643822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:39:33.643822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:39:33.643822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:33.643822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:39:33.643822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:33.643822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:39:33.643822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:39:33.643822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:33.644227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:39:33.644227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:33.644227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:39:33.749982 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:39:33.750531 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:39:33.750763 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:39:33.752092 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:39:33.752431 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:39:33.752543 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:33.753458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:39:33.753458 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:33.753458 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:39:33.753458 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:39:33.753458 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:33.753458 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:39:33.753458 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:33.753458 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:39:33.755568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:33.755568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:39:33.755568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:39:33.755568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:39:33.755568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:39:33.755568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:39:33.755568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:39:33.755568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:33.755568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:39:33.755568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:33.755568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:39:33.755568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:33.755955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:39:33.755955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:33.755955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:39:33.755955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:33.755955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:39:33.755955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:39:33.755955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:33.755955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:39:33.755955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:33.755955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:39:33.755955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:39:33.755955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:33.755955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:39:33.756339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:33.756339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:39:33.757629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:33.757629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:39:33.757629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:39:33.757629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:39:33.757629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:39:33.757629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:39:33.757629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:39:33.757629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:39:33.757629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:39:33.757629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:39:33.757629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:39:33.757629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:39:33.757629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:39:33.757629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:39:33.757629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:39:33.757629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:39:33.758416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:39:33.758416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:39:33.758416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:33.758416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:39:33.758416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:33.758416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:39:33.758416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:33.758416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:39:33.758416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:33.758416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:39:33.758416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:33.758416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:39:33.758416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:39:33.758416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:33.758416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:39:33.758416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:33.758416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:39:33.758416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:39:33.758416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:33.759047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:39:33.759047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:33.759047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:39:33.759296 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:39:33.759553 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:39:33.759651 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:39:33.759805 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:39:33.760382 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:39:33.760709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:33.761395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:39:33.761395 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:33.761395 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:39:33.761395 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:39:33.761395 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:33.761395 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:39:33.761395 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:33.761395 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:39:33.762185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:33.762185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:39:33.762185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:39:33.762185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:39:33.762185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:33.762185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:39:33.762185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:33.762185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:39:33.762185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:33.762185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:39:33.762185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:33.762185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:39:33.762185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:33.762185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:39:33.762185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:39:33.762650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:33.762650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:39:33.762650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:33.762650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:39:33.762650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:39:33.762650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:33.762650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:39:33.762650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:33.762650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:39:33.763259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:33.763259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:39:33.763259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:39:33.763259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:39:33.763259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:39:33.763259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:39:33.763259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:39:33.763259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:39:33.763259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:39:33.763259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:39:33.763259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:33.763259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:39:33.763658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:33.763658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:39:33.763658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:33.763658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:39:33.763658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:33.763658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:39:33.763658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:33.763658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:39:33.763658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:39:33.763658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:33.763658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:39:33.763658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:33.763658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:39:33.763658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:39:33.764113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:33.764113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:39:33.764113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:33.764113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:39:33.764427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:33.764427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:39:33.764427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:39:33.764427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:39:33.764427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:39:33.764427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:39:33.764427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:39:33.764427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:39:33.764427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:39:33.764427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:39:33.764427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:39:33.764427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:39:33.764427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:39:33.764427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:39:33.764427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:39:33.764427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:39:33.764910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:39:33.764910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:39:33.764910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:39:33.764910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:39:33.764910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:39:33.764910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:33.764910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:39:33.764910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:33.764910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:39:33.764910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:33.764910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:39:33.764910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:33.764910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:39:33.764910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:33.764910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:39:33.764910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:39:33.764910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:33.764910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:39:33.765450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:33.765450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:39:33.765450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:39:33.765450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:33.765450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:39:33.765450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:33.765450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:39:33.768676 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:39:33.768983 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:39:33.769092 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:39:33.769216 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:39:33.769640 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:39:33.770052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:33.770704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:39:33.770704 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:33.770704 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:39:33.770704 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:39:33.770704 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:33.770704 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:39:33.770704 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:33.770704 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:39:33.771456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:33.771456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:39:33.771456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:39:33.771456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:39:33.771456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:33.771456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:39:33.771456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:33.771456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:39:33.771456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:33.771456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:39:33.771456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:33.771456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:39:33.771456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:33.771456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:39:33.771456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:39:33.772672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:33.772672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:39:33.772672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:33.772672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:39:33.772672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:39:33.772672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:33.772672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:39:33.772672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:33.772672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:39:33.772672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:33.772672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:39:33.772672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:39:33.772672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:39:33.772672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:39:33.773169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:39:33.773169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:39:33.773169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:39:33.773169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:39:33.773169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:33.773169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:39:33.773169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:33.773169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:39:33.773169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:33.773169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:39:33.773169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:33.773169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:39:33.773169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:33.773622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:39:33.773622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:39:33.773622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:33.773622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:39:33.773622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:33.773622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:39:33.773622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:39:33.773622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:33.773622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:39:33.773622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:33.773622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:39:33.773622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:33.774082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:39:33.774082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:39:33.774082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:39:33.774082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:39:33.774082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:39:33.774082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:39:33.774082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:39:33.774082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:39:33.774082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:39:33.774082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:39:33.774082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:39:33.774082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:39:33.774082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:39:33.774082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:39:33.774082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:39:33.774082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:39:33.774082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:39:33.774082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:39:33.774082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:39:33.774689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:33.774689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:39:33.774689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:33.774689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:39:33.774689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:33.774689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:39:33.774689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:33.774689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:39:33.774689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:33.774689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:39:33.774689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:39:33.774689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:33.774689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:39:33.774689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:33.774689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:39:33.774689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:39:33.774689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:33.774689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:39:33.774689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:33.774689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:39:33.775323 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:39:33.775323 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:39:33.775323 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:39:33.775323 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:39:33.775467 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:39:33.776220 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:39:33.776568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:33.777252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:39:33.777252 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:33.777252 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:39:33.777252 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:39:33.777252 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:33.777252 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:39:33.777252 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:33.777252 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:39:33.778127 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:33.778127 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:39:33.778127 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:39:33.778127 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:39:33.778127 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:39:33.778127 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:39:33.778127 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:39:33.778127 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:33.778127 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:39:33.778127 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:33.778127 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:39:33.778127 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:33.778497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:39:33.778497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:33.778497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:39:33.778497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:33.778497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:39:33.778497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:39:33.778497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:33.778497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:39:33.778497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:33.778497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:39:33.778497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:39:33.778497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:33.778497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:39:33.778982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:33.778982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:39:33.779197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:33.779197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:39:33.779197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:39:33.779197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:39:33.779197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:39:33.779197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:39:33.779197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:39:33.779197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:39:33.779197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:39:33.779197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:39:33.779197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:39:33.779197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:39:33.779197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:39:33.779197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:39:33.779197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:39:33.779197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:39:33.779633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:39:33.779633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:39:33.779633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:33.779633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:39:33.779633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:33.779633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:39:33.779633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:33.779633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:39:33.779633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:33.779633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:39:33.779633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:33.779633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:39:33.779633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:39:33.779633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:33.779633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:39:33.779633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:33.779633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:39:33.779633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:39:33.779633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:33.780178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:39:33.780178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:33.780178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:39:38.600438 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:39:38.601576 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:39:38.601576 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:39:38.602506 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:39:38.648020 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:39:38.648496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:38.650124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:39:38.650124 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:38.650124 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:39:38.650124 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:39:38.650124 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:38.650124 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:39:38.650124 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:38.650124 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:39:38.652039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.049 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:38.652039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:39:38.652039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:39:38.652039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:39:38.652039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:39:38.652039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:39:38.652039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:39:38.652039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:38.652039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:39:38.652039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:38.652039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:39:38.652039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:38.652509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:39:38.652509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:38.652509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:39:38.652509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:38.652509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:39:38.652509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:39:38.652509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:38.652509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:39:38.652509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:38.652509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:39:38.652509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:39:38.652509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:38.652509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:39:38.652909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:38.652909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:39:38.654878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:38.654878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:39:38.654878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:39:38.654878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:39:38.654878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:39:38.654878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:39:38.654878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:39:38.654878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:39:38.654878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:39:38.654878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:39:38.654878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:39:38.654878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:39:38.654878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:39:38.654878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:39:38.654878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:39:38.654878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:39:38.655438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:39:38.655438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:39:38.655438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:38.655438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:39:38.655438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:38.655438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:39:38.655438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:38.655438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:39:38.655438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:38.655438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:39:38.655438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:38.655438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:39:38.655438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:39:38.655438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:38.655438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:39:38.655438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:38.655438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:39:38.655438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:39:38.655438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:38.656107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:39:38.656107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:38.656107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:39:38.750659 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:39:38.751347 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:39:38.751487 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:39:38.752601 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:39:38.752875 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:39:38.753041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:38.753864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:39:38.753864 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:38.753864 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:39:38.753864 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:39:38.753864 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:38.753864 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:39:38.753864 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:38.753864 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:39:38.755349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:38.755349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:39:38.755349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:39:38.755349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:39:38.755349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:39:38.755349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:39:38.755349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:39:38.755349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:38.755349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:39:38.755349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:38.755349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:39:38.755349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:38.755652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:39:38.755652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:38.755652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:39:38.755652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:38.755652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:39:38.755652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:39:38.755652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:38.755652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:39:38.755652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:38.755652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:39:38.755652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:39:38.755652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:38.755652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:39:38.756089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:38.756089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:39:38.756827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:38.756827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:39:38.756827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:39:38.756827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:39:38.756827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:39:38.756827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:39:38.756827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:39:38.756827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:39:38.756827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:39:38.756827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:39:38.756827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:39:38.756827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:39:38.756827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:39:38.756827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:39:38.756827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:39:38.756827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:39:38.757177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:39:38.757177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:39:38.757177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:38.757177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:39:38.757177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:38.757177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:39:38.757177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:38.757177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:39:38.757177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:38.757177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:39:38.757177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:38.757177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:39:38.757177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:39:38.757177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:38.757177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:39:38.757177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:38.757177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:39:38.757177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:39:38.757177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:38.757572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:39:38.757572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:38.757572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:39:38.759578 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:39:38.759854 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:39:38.759966 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:39:38.760089 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:39:38.760678 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:39:38.761812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:38.763195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:39:38.763195 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:38.763195 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:39:38.763195 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:39:38.763195 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:38.763195 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:39:38.763195 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:38.763195 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:39:38.764106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:38.764106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:39:38.764106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:39:38.764106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:39:38.764106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:38.764106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:39:38.764106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:38.764106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:39:38.764106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:38.764106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:39:38.764106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:38.764106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:39:38.764106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:38.764106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:39:38.764106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:39:38.764634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:38.764634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:39:38.764634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:38.764634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:39:38.764634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:39:38.764634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:38.764634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:39:38.764634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:38.764634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:39:38.765019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:38.765019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:39:38.765019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:39:38.765019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:39:38.765019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:39:38.765019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:39:38.765019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:39:38.765019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:39:38.765019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:39:38.765019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:39:38.765019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:38.765019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:39:38.765525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:38.765525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:39:38.765525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:38.765525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:39:38.765525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:38.765525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:39:38.765525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:38.765525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:39:38.765525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:39:38.765525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:38.765525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:39:38.765525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:38.765525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:39:38.765525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:39:38.765973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:38.765973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:39:38.765973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:38.765973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:39:38.766242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:38.766242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:39:38.766242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:39:38.766242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:39:38.766242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:39:38.766242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:39:38.766242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:39:38.766242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:39:38.766242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:39:38.766242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:39:38.766242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:39:38.766242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:39:38.766242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:39:38.766242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:39:38.766242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:39:38.766242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:39:38.766731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:39:38.766731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:39:38.766731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:39:38.766731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:39:38.766731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:39:38.766731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:38.766731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:39:38.766731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:38.766731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:39:38.766731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:38.766731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:39:38.766731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:38.766731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:39:38.766731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:38.766731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:39:38.766731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:39:38.766731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:38.766731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:39:38.767248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:38.767248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:39:38.767248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:39:38.767248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:38.767248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:39:38.767248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:38.767248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:39:38.769065 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:39:38.769311 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:39:38.769422 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:39:38.769609 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:39:38.770131 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:39:38.770332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:38.770837 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:39:38.770837 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:38.770837 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:39:38.770837 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:39:38.770837 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:38.770837 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:39:38.770837 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:38.770837 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:39:38.771757 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:38.771757 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:39:38.771757 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:39:38.771757 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:39:38.771757 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:38.771757 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:39:38.771757 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:38.771757 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:39:38.771757 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:38.771757 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:39:38.771757 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:38.771757 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:39:38.771757 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:38.771757 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:39:38.771757 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:39:38.772184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:38.772184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:39:38.772184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:38.772184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:39:38.772184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:39:38.772184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:38.772184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:39:38.772184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:38.772184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:39:38.772657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:38.772657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:39:38.772657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:39:38.772657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:39:38.772657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:39:38.772657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:39:38.772657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:39:38.772657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:39:38.772657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:39:38.772657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:38.772657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:39:38.773004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:38.773004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:39:38.773004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:38.773004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:39:38.773004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:38.773004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:39:38.773004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:38.773004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:39:38.773004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:39:38.773004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:38.773004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:39:38.773004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:38.773004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:39:38.773004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:39:38.773435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:38.773435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:39:38.773435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:38.773435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:39:38.773797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:38.773797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:39:38.773797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:39:38.773797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:39:38.773797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:39:38.773797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:39:38.773797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:39:38.773797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:39:38.773797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:39:38.773797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:39:38.773797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:39:38.773797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:39:38.773797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:39:38.773797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:39:38.773797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:39:38.773797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:39:38.773797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:39:38.774402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:39:38.774402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:39:38.774402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:39:38.774402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:38.774402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:39:38.774402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:38.774402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:39:38.774402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:38.774402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:39:38.774402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:38.774402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:39:38.774402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:38.774402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:39:38.774402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:39:38.774402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:38.774402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:39:38.774402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:38.774402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:39:38.774402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:39:38.775024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:38.775024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:39:38.775024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:38.775024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:39:38.775024 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:39:38.775024 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:39:38.775024 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:39:38.775297 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:39:38.775682 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:39:38.775946 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:39:38.776146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:38.776734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:39:38.776734 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:38.776734 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:39:38.776734 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:39:38.776734 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:38.776734 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:39:38.776734 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:38.776734 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:39:38.777948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:38.777948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:39:38.777948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:39:38.777948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:39:38.777948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:39:38.777948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:39:38.777948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:39:38.777948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:38.777948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:39:38.777948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:38.777948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:39:38.777948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:38.778367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:39:38.778367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:38.778367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:39:38.778367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:38.778367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:39:38.778367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:39:38.778367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:38.778367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:39:38.778367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:38.778367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:39:38.778367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:39:38.778367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:38.778367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:39:38.778737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:38.778737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:39:38.779419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:38.779419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:39:38.779419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:39:38.779419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:39:38.779419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:39:38.779419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:39:38.779419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:39:38.779419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:39:38.779419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:39:38.779419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:39:38.779419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:39:38.779419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:39:38.779419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:39:38.779419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:39:38.779419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:39:38.779419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:39:38.779898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:39:38.779898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:39:38.779898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:38.779898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:39:38.779898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:38.779898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:39:38.779898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:38.779898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:39:38.779898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:38.779898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:39:38.779898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:38.779898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:39:38.779898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:39:38.779898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:38.779898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:39:38.779898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:38.779898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:39:38.779898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:39:38.779898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:38.780567 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:39:38.780567 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:38.780567 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:39:43.601203 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:39:43.601203 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:39:43.603675 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:39:43.603675 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:39:43.639174 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:39:43.639558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:43.640670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:39:43.640670 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:43.640670 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:39:43.640670 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:39:43.640670 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:43.640670 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:39:43.640670 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:43.640670 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:39:43.642210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:43.642210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:39:43.642210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:39:43.642210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:39:43.642210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:39:43.642210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:39:43.642210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:39:43.642210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:43.642210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:39:43.642210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:43.642210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:39:43.642210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:43.642529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:39:43.642529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:43.642529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:39:43.642529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:43.642529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:39:43.642529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:39:43.642529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:43.642529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:39:43.642529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:43.642529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:39:43.642529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:39:43.642529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:43.642529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:39:43.642836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:43.642836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:39:43.643799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:43.643799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:39:43.643799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:39:43.643799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:39:43.643799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:39:43.643799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:39:43.643799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:39:43.643799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:39:43.643799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:39:43.643799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:39:43.643799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:39:43.643799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:39:43.643799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:39:43.643799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:39:43.643799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:39:43.643799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:39:43.644236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:39:43.644236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:39:43.644236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:43.644236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:39:43.644236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:43.644236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:39:43.644236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:43.644236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:39:43.644236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:43.644236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:39:43.644236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:43.644236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:39:43.644236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:39:43.644236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:43.644236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:39:43.644236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:43.644236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:39:43.644236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:39:43.644236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:43.644660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:39:43.644660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:43.644660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:39:43.751802 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:39:43.752508 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:39:43.752975 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:39:43.753621 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:39:43.753939 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:39:43.754162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:43.754977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:39:43.754977 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:43.754977 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:39:43.754977 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:39:43.754977 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:43.754977 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:39:43.754977 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:43.754977 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:39:43.756451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:43.756451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:39:43.756451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:39:43.756451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:39:43.756451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:39:43.756451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:39:43.756451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:39:43.756451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:43.756451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:39:43.756451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:43.756451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:39:43.756451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:43.757146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:39:43.757146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:43.757146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:39:43.757146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:43.757146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:39:43.757146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:39:43.757146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:43.757146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:39:43.757146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:43.757146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:39:43.757146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:39:43.757146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:43.757146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:39:43.758608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:43.758608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:39:43.758608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:43.758608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:39:43.758608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:39:43.758608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:39:43.758608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:39:43.758608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:39:43.758608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:39:43.758608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:39:43.758608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:39:43.758608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:39:43.758608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:39:43.758608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:39:43.758608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:39:43.760217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:39:43.760217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:39:43.760217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:39:43.760217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:39:43.760217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:39:43.760217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:43.760217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:39:43.760217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:43.760217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:39:43.760217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:43.760217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:39:43.760217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:43.760217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:39:43.760217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:43.760217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:39:43.760217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:39:43.760217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:43.760217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:39:43.760217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:43.760732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:39:43.760732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:39:43.760732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:43.760732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:39:43.760732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:43.760732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:39:43.760732 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:39:43.760732 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:39:43.760732 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:39:43.760732 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:39:43.761362 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:39:43.761362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:43.761468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:39:43.761468 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:43.761468 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:39:43.761468 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:39:43.761468 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:43.761468 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:39:43.761468 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:43.761468 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:39:43.762234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:43.762234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:39:43.762234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:39:43.762234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:39:43.762234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:43.762234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:39:43.762234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:43.762234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:39:43.762234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:43.762234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:39:43.762234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:43.762234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:39:43.762234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:43.762234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:39:43.762234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:39:43.762584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:43.762584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:39:43.762584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:43.762584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:39:43.762584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:39:43.762584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:43.762584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:39:43.762584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:43.762584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:39:43.763107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:43.763107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:39:43.763107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:39:43.763107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:39:43.763107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:39:43.763107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:39:43.763107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:39:43.763107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:39:43.763107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:39:43.763107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:39:43.763107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:43.763107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:39:43.763390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:43.763390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:39:43.763390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:43.763390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:39:43.763390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:43.763390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:39:43.763390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:43.763390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:39:43.763390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:39:43.763390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:43.763390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:39:43.763390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:43.763390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:39:43.763390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:39:43.763664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:43.763664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:39:43.763664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:43.763664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:39:43.764187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:43.764187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:39:43.764187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:39:43.764187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:39:43.764187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:39:43.764187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:39:43.764187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:39:43.764187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:39:43.764187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:39:43.764187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:39:43.764187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:39:43.764187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:39:43.764187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:39:43.764187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:39:43.764187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:39:43.764187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:39:43.764502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:39:43.764502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:39:43.764502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:39:43.764502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:39:43.764502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:39:43.764502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:43.764502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:39:43.764502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:43.764502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:39:43.764502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:43.764502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:39:43.764502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:43.764502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:39:43.764502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:43.764502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:39:43.764502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:39:43.764502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:43.764502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:39:43.765036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:43.765036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:39:43.765036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:39:43.765036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:43.765036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:39:43.765036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:43.765036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:39:43.769649 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:39:43.769649 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:39:43.769649 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:39:43.769793 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:39:43.770048 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:39:43.770319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:43.770688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:39:43.770688 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:43.770688 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:39:43.770688 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:39:43.770688 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:43.770688 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:39:43.770688 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:43.770688 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:39:43.771447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:43.771447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:39:43.771447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:39:43.771447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:39:43.771447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:43.771447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:39:43.771447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:43.771447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:39:43.771447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:43.771447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:39:43.771447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:43.771447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:39:43.771447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:43.771447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:39:43.771447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:39:43.772687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:43.772687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:39:43.772687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:43.772687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:39:43.772687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:39:43.772687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:43.772687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:39:43.772687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:43.772687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:39:43.772687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:43.772687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:39:43.772687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:39:43.772687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:39:43.772687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:39:43.773391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:39:43.773391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:39:43.773391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:39:43.773391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:39:43.773391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:43.773391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:39:43.773391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:43.773391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:39:43.773391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:43.773391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:39:43.773391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:43.773391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:39:43.773391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:43.774136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:39:43.774136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:39:43.774136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:43.774136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:39:43.774136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:43.774136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:39:43.774136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:39:43.774136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:43.774136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:39:43.774136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:43.774136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:39:43.774136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:43.774724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:39:43.774724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:39:43.774724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:39:43.774724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:39:43.774724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:39:43.774724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:39:43.774724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:39:43.774724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:39:43.774724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:39:43.774724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:39:43.774724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:39:43.774724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:39:43.774724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:39:43.774724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:39:43.774724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:39:43.774724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:39:43.774724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:39:43.774724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:39:43.774724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:39:43.776731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:43.776731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:39:43.776731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:43.776731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:39:43.776731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:43.776731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:39:43.776731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:43.776731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:39:43.776731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:43.776731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:39:43.776731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:39:43.776731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:43.776731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:39:43.776731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:43.776731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:39:43.776731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:39:43.776731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:43.776731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:39:43.776731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:43.776731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:39:43.777274 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:39:43.777274 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:39:43.777274 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:39:43.777274 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:39:43.777274 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:39:43.777274 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:39:43.777490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:43.777490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:39:43.777490 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:43.777490 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:39:43.777490 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:39:43.777490 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:43.777490 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:39:43.777490 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:43.777490 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:39:43.777490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:43.777490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:39:43.777490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:39:43.777490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:39:43.777490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:39:43.777490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:39:43.777923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:39:43.777923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:43.777923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:39:43.777923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:43.777923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:39:43.777923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:43.777923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:39:43.777923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:43.777923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:39:43.777923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:43.777923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:39:43.777923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:39:43.777923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:43.777923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:39:43.778294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:43.778294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:39:43.778294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:39:43.778294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:43.778294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:39:43.778294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:43.778294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:39:43.778478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:43.778478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:39:43.778478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:39:43.778478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:39:43.778478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:39:43.778478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:39:43.778478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:39:43.778478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:39:43.778478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:39:43.778478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:39:43.778478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:39:43.778478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:39:43.778478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:39:43.778478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:39:43.778478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:39:43.778478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:39:43.778895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:39:43.778895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:39:43.778895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:43.778895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:39:43.778895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:43.778895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:39:43.778895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:43.778895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:39:43.778895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:43.778895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:39:43.778895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:43.778895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:39:43.778895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:39:43.778895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:43.778895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:39:43.778895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:43.778895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:39:43.778895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:39:43.778895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:43.779396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:39:43.779396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:43.779396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:39:47.983392 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_for_inconsistencies' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:39:47.984477 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_for_inconsistencies {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:39:47.984621 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Checking Neutron and OVN revision consistency. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:39:47.994375 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Maintenance task: Synchronizing Neutron and OVN databases started {{(pid=106314) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:440}}
May 09 09:39:47.994501 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Maintenance task: Number of inconsistencies found at create/update: networks=4, routers=1, router_ports=1 {{(pid=106314) _log /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:419}}
May 09 09:39:47.994602 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Maintenance task: Fixing resource f8a38fcb-e8a2-421a-a340-bf7f2f5cc72b (type: networks) at create/update {{(pid=106314) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 09 09:39:48.024735 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-f8a38fcb-e8a2-421a-a340-bf7f2f5cc72b, resource={'id': 'f8a38fcb-e8a2-421a-a340-bf7f2f5cc72b', 'name': 'private', 'tenant_id': 'f4d54dfe001d43fbb344093a1ccabd6a', 'project_id': 'f4d54dfe001d43fbb344093a1ccabd6a', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['96e72a03-97b8-412a-b2b9-1bf048ce1066', 'bbd6957a-953f-4fc1-8761-5d93128564d9'], 'standard_attr_id': 12, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-05-09T09:04:34Z', 'updated_at': '2026-05-09T09:04:37Z', 'revision_number': 3, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 3730}, resource_type=networks, if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:39:48.024894 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-f8a38fcb-e8a2-421a-a340-bf7f2f5cc72b, col_values=(('external_ids', {'neutron:network_name': 'private', 'neutron:mtu': '1372', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:39:48.025139 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-78a32c7c-7637-4b83-b0ab-62aee8bb3038, external_ids_update=None, columns={'tag': 3730}, if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:39:48.025253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:48.025648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:39:48.025648 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:48.025648 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:39:48.025648 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:39:48.025648 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:48.025648 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:39:48.025648 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:48.025648 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:39:48.026504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Maintenance task: Failed to fix resource f8a38fcb-e8a2-421a-a340-bf7f2f5cc72b (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:48.026504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:39:48.026504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 09 09:39:48.026504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 09 09:39:48.026504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 09 09:39:48.026504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 09 09:39:48.026504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2332, in update_network
May 09 09:39:48.026504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:39:48.026504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:48.026504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:39:48.026504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:48.026942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:39:48.026942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:48.026942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:39:48.026942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:48.026942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:39:48.026942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:48.026942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:39:48.026942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:39:48.026942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:48.026942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:39:48.026942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:48.026942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:39:48.026942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:39:48.026942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:48.027433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:39:48.027433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:48.027433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:39:48.027433 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Maintenance task: Fixing resource d18a9fac-2ebf-4d1e-a86c-0f33fba8eeda (type: networks) at create/update {{(pid=106314) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 09 09:39:48.054633 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-d18a9fac-2ebf-4d1e-a86c-0f33fba8eeda, resource={'id': 'd18a9fac-2ebf-4d1e-a86c-0f33fba8eeda', 'name': 'public', 'tenant_id': 'f4d54dfe001d43fbb344093a1ccabd6a', 'project_id': 'f4d54dfe001d43fbb344093a1ccabd6a', 'admin_state_up': True, 'mtu': 1430, 'status': 'ACTIVE', 'subnets': ['088bbd0f-7e81-4a76-8399-24d15c984009', 'bbd6a8cc-80e6-4336-a70b-1c962c561590'], 'standard_attr_id': 18, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': True, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'is_default': True, 'tags': [], 'created_at': '2026-05-09T09:04:40Z', 'updated_at': '2026-05-09T09:04:50Z', 'revision_number': 3, 'provider:network_type': 'flat', 'provider:physical_network': 'public', 'provider:segmentation_id': None}, resource_type=networks, if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:39:48.054911 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-d18a9fac-2ebf-4d1e-a86c-0f33fba8eeda, col_values=(('external_ids', {'neutron:network_name': 'public', 'neutron:mtu': '1430', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'flat', 'neutron:provnet-physical-network': 'public'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false', 'fdb_age_threshold': '300', 'broadcast-arps-to-all-routers': 'true'})), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:39:48.055252 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-1163f525-70cd-4b84-ae9b-7d56378305e9, external_ids_update=None, columns={'tag': []}, if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:39:48.055484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:48.055989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:39:48.055989 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:48.055989 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:39:48.055989 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:39:48.055989 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:48.055989 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:39:48.055989 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:48.055989 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:39:48.057064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Maintenance task: Failed to fix resource d18a9fac-2ebf-4d1e-a86c-0f33fba8eeda (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:48.057064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:39:48.057064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 09 09:39:48.057064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 09 09:39:48.057064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 09 09:39:48.057064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 09 09:39:48.057064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2332, in update_network
May 09 09:39:48.057064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:39:48.057064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:48.057064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:39:48.057064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:48.057664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:39:48.057664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:48.057664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:39:48.057664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:48.057664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:39:48.057664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:48.057664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:39:48.057664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:39:48.057664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:48.057664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:39:48.057664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:48.057664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:39:48.057664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:39:48.057664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:48.058669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:39:48.058669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:48.058669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:39:48.058669 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Maintenance task: Fixing resource 417da935-2882-452f-8ba7-9076d68fcd2b (type: networks) at create/update {{(pid=106314) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 09 09:39:48.084928 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-417da935-2882-452f-8ba7-9076d68fcd2b, resource={'id': '417da935-2882-452f-8ba7-9076d68fcd2b', 'name': 'admin_net', 'tenant_id': '38e536cfc9c94b6cbf529caf55cc7739', 'project_id': '38e536cfc9c94b6cbf529caf55cc7739', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['d3c53d14-f490-4615-a986-3715a315f827'], 'standard_attr_id': 42, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-05-09T09:10:38Z', 'updated_at': '2026-05-09T09:10:39Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 32823}, resource_type=networks, if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:39:48.085192 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-417da935-2882-452f-8ba7-9076d68fcd2b, col_values=(('external_ids', {'neutron:network_name': 'admin_net', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:39:48.085321 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-5738ab2a-b471-447b-9292-7f8c6bc97168, external_ids_update=None, columns={'tag': 32823}, if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:39:48.085446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:48.085845 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:39:48.085845 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:48.085845 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:39:48.085845 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:39:48.085845 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:48.085845 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:39:48.085845 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:48.085845 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:39:48.086715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Maintenance task: Failed to fix resource 417da935-2882-452f-8ba7-9076d68fcd2b (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:48.086715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:39:48.086715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 09 09:39:48.086715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 09 09:39:48.086715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 09 09:39:48.086715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 09 09:39:48.086715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2332, in update_network
May 09 09:39:48.086715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:39:48.086715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:48.086715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:39:48.086715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:48.087131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:39:48.087131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:48.087131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:39:48.087131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:48.087131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:39:48.087131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:48.087131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:39:48.087131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:39:48.087131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:48.087131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:39:48.087131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:48.087131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:39:48.087131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:39:48.087131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:48.087632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:39:48.087632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:48.087632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:39:48.087632 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Maintenance task: Fixing resource 433617e8-cdf8-414a-8cc6-63a7b5d0c678 (type: networks) at create/update {{(pid=106314) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 09 09:39:48.111275 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-433617e8-cdf8-414a-8cc6-63a7b5d0c678, resource={'id': '433617e8-cdf8-414a-8cc6-63a7b5d0c678', 'name': 'k8s-clusterapi-cluster-magnum-system-kube-whtb8', 'tenant_id': '3e013d45a44544c3b50eb0e83e4b1efc', 'project_id': '3e013d45a44544c3b50eb0e83e4b1efc', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['9187f065-9264-427a-8df8-672a6981191d'], 'standard_attr_id': 52, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-05-09T09:11:20Z', 'updated_at': '2026-05-09T09:11:20Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 32644}, resource_type=networks, if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:39:48.111466 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-433617e8-cdf8-414a-8cc6-63a7b5d0c678, col_values=(('external_ids', {'neutron:network_name': 'k8s-clusterapi-cluster-magnum-system-kube-whtb8', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:39:48.111608 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-8c3fe586-c182-47ba-9845-d7608113cb71, external_ids_update=None, columns={'tag': 32644}, if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:39:48.111712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:48.112284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:39:48.112284 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:48.112284 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:39:48.112284 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:39:48.112284 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:48.112284 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:39:48.112284 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:48.112284 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:39:48.113043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Maintenance task: Failed to fix resource 433617e8-cdf8-414a-8cc6-63a7b5d0c678 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:48.113043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:39:48.113043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 09 09:39:48.113043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 09 09:39:48.113043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 09 09:39:48.113043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 09 09:39:48.113043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2332, in update_network
May 09 09:39:48.113043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:39:48.113043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:48.113043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:39:48.113043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:48.113826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:39:48.113826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:48.113826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:39:48.113826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:48.113826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:39:48.113826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:48.113826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:39:48.113826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:39:48.113826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:48.113826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:39:48.113826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:48.113826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:39:48.113826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:39:48.113826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:48.114719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:39:48.114719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:48.114719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:39:48.114719 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Maintenance task: Fixing resource cab10384-14c5-492c-8bd0-061f5838310c (type: routers) at create/update {{(pid=106314) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 09 09:39:48.185936 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-cab10384-14c5-492c-8bd0-061f5838310c, resource={'id': 'cab10384-14c5-492c-8bd0-061f5838310c', 'name': 'k8s-clusterapi-cluster-magnum-system-kube-whtb8', 'project_id': '3e013d45a44544c3b50eb0e83e4b1efc', 'admin_state_up': True, 'status': 'ACTIVE', 'enable_snat': True, 'external_gateway_info': {'network_id': 'd18a9fac-2ebf-4d1e-a86c-0f33fba8eeda', 'external_fixed_ips': [{'subnet_id': 'bbd6a8cc-80e6-4336-a70b-1c962c561590', 'ip_address': '172.24.5.116'}, {'subnet_id': '088bbd0f-7e81-4a76-8399-24d15c984009', 'ip_address': '2001:db8::1b7'}], 'enable_snat': True}, 'gw_port_id': 'af118f11-dc01-429a-b20a-e97b77b8d5ec', 'description': 'Created by cluster-api-provider-openstack cluster magnum-system-kube-whtb8', 'availability_zones': [], 'distributed': False, 'ha': True, 'ha_vr_id': 0, 'availability_zone_hints': [], 'enable_default_route_ecmp': False, 'enable_default_route_bfd': False, 'external_gateways': [{'network_id': 'd18a9fac-2ebf-4d1e-a86c-0f33fba8eeda', 'external_fixed_ips': [{'ip_address': '172.24.5.116', 'subnet_id': 'bbd6a8cc-80e6-4336-a70b-1c962c561590'}, {'ip_address': '2001:db8::1b7', 'subnet_id': '088bbd0f-7e81-4a76-8399-24d15c984009'}]}], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2026-05-09T09:11:21Z', 'updated_at': '2026-05-09T09:11:24Z', 'revision_number': 3, 'tenant_id': '3e013d45a44544c3b50eb0e83e4b1efc'}, resource_type=routers, if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:39:48.186307 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterCommand(_result=None, name=neutron-cab10384-14c5-492c-8bd0-061f5838310c, columns={'external_ids': {'neutron:router_name': 'k8s-clusterapi-cluster-magnum-system-kube-whtb8', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': ''}, 'enabled': True}, if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:39:48.186307 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): QoSDelCommand(_result=None, switch=neutron-d18a9fac-2ebf-4d1e-a86c-0f33fba8eeda, conditions=[('direction', '=', 'from-lport'), ('priority', '=', 2002), ('match', '=', 'inport == "af118f11-dc01-429a-b20a-e97b77b8d5ec"')], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:39:48.186619 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): QoSDelCommand(_result=None, switch=neutron-d18a9fac-2ebf-4d1e-a86c-0f33fba8eeda, conditions=[('direction', '=', 'to-lport'), ('priority', '=', 2002), ('match', '=', 'outport == "af118f11-dc01-429a-b20a-e97b77b8d5ec"')], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:39:48.186619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:48.187255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:39:48.187255 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:48.187255 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:39:48.187255 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:39:48.187255 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:48.187255 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:39:48.187255 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:48.187255 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:39:48.187737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_client [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Unable to update router cab10384-14c5-492c-8bd0-061f5838310c. Error: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:48.188980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Maintenance task: Failed to fix resource cab10384-14c5-492c-8bd0-061f5838310c (type: routers): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:48.188980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:39:48.188980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 09 09:39:48.188980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 09 09:39:48.188980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 09 09:39:48.188980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 09 09:39:48.188980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1635, in update_router
May 09 09:39:48.188980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with excutils.save_and_reraise_exception():
May 09 09:39:48.188980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 271, in __exit__
May 09 09:39:48.188980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.force_reraise()
May 09 09:39:48.188980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 233, in force_reraise
May 09 09:39:48.189637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise self.value
May 09 09:39:48.189637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1569, in update_router
May 09 09:39:48.189637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:39:48.189637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:48.189637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:39:48.189637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:48.189637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:39:48.189637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:48.189637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:39:48.189637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:48.189637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:39:48.189637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:48.189637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:39:48.189637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:39:48.190571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:48.190571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:39:48.190571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:48.190571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:39:48.190571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:39:48.190571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:48.190571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:39:48.190571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:48.190571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:39:48.190571 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Maintenance task: Fixing resource af118f11-dc01-429a-b20a-e97b77b8d5ec (type: router_ports) at create/update {{(pid=106314) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 09 09:39:48.298918 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=lrp-af118f11-dc01-429a-b20a-e97b77b8d5ec, resource={'id': 'af118f11-dc01-429a-b20a-e97b77b8d5ec', 'name': '', 'network_id': 'd18a9fac-2ebf-4d1e-a86c-0f33fba8eeda', 'project_id': '', 'mac_address': 'fa:16:3e:51:04:94', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'cab10384-14c5-492c-8bd0-061f5838310c', 'device_owner': 'network:router_gateway', 'standard_attr_id': 57, 'fixed_ips': [{'subnet_id': 'bbd6a8cc-80e6-4336-a70b-1c962c561590', 'ip_address': '172.24.5.116'}, {'subnet_id': '088bbd0f-7e81-4a76-8399-24d15c984009', 'ip_address': '2001:db8::1b7'}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'np0000173033', 'binding:vif_type': 'ovs', 'binding:vif_details': {'port_filter': True, 'connectivity': 'l2', 'ovs_create_tap': False, 'bridge_name': 'br-int', 'datapath_type': 'system', 'bound_drivers': {'0': 'ovn'}}, 'port_security_enabled': False, 'tags': [], 'created_at': '2026-05-09T09:11:21Z', 'updated_at': '2026-05-09T09:11:24Z', 'revision_number': 5, 'tenant_id': ''}, resource_type=router_ports, if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:39:48.299818 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterPortCommand(_result=None, name=lrp-af118f11-dc01-429a-b20a-e97b77b8d5ec, columns={'external_ids': {'neutron:revision_number': '5', 'neutron:subnet_ids': 'bbd6a8cc-80e6-4336-a70b-1c962c561590 088bbd0f-7e81-4a76-8399-24d15c984009', 'neutron:network_name': 'neutron-d18a9fac-2ebf-4d1e-a86c-0f33fba8eeda', 'neutron:is_ext_gw': 'True', 'neutron:router_name': 'neutron-cab10384-14c5-492c-8bd0-061f5838310c'}, 'options': {'reside-on-redirect-chassis': 'true', 'gateway_mtu': '1372'}, 'networks': ['172.24.5.116/24', '2001:db8::1b7/64'], 'ipv6_ra_configs': {}}, if_exists=False) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:39:48.300357 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLRouterPortInLSwitchPortCommand(_result=None, lswitch_port=af118f11-dc01-429a-b20a-e97b77b8d5ec, lrouter_port=lrp-af118f11-dc01-429a-b20a-e97b77b8d5ec, is_gw_port=True, if_exists=True, lsp_address=router) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:39:48.300775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:48.301713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:39:48.301713 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:48.301713 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:39:48.301713 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:39:48.301713 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:48.301713 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:39:48.301713 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:48.301713 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:39:48.302969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Maintenance task: Failed to fix resource af118f11-dc01-429a-b20a-e97b77b8d5ec (type: router_ports): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:48.302969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:39:48.302969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 09 09:39:48.302969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 09 09:39:48.302969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 09 09:39:48.302969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 09 09:39:48.302969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1976, in update_router_port
May 09 09:39:48.302969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:39:48.302969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:48.302969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:39:48.302969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:48.303532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:39:48.303532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:48.303532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:39:48.303532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:48.303532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:39:48.303532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:48.303532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:39:48.303532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:39:48.303532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:48.303532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:39:48.303532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:48.303532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:39:48.303532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:39:48.303532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:48.304146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:39:48.304146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:48.304146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:39:48.304146 np0000173033 neutron-ovn-maintenance-worker[106314]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_for_inconsistencies finished in 0.318 seconds
May 09 09:39:48.601856 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:39:48.602326 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:39:48.602486 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:39:48.603384 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:39:48.675608 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:39:48.676283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:48.676665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:39:48.676665 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:48.676665 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:39:48.676665 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:39:48.676665 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:48.676665 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:39:48.676665 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:48.676665 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:39:48.677912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.074 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:48.677912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:39:48.677912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:39:48.677912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:39:48.677912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:39:48.677912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:39:48.677912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:39:48.677912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:48.677912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:39:48.677912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:48.677912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:39:48.677912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:48.678374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:39:48.678374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:48.678374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:39:48.678374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:48.678374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:39:48.678374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:39:48.678374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:48.678374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:39:48.678374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:48.678374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:39:48.678374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:39:48.678374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:48.678374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:39:48.678811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:48.678811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:39:48.679636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:48.679636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:39:48.679636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:39:48.679636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:39:48.679636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:39:48.679636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:39:48.679636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:39:48.679636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:39:48.679636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:39:48.679636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:39:48.679636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:39:48.679636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:39:48.679636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:39:48.679636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:39:48.679636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:39:48.679636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:39:48.680303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:39:48.680303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:39:48.680303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:48.680303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:39:48.680303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:48.680303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:39:48.680303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:48.680303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:39:48.680303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:48.680303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:39:48.680303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:48.680303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:39:48.680303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:39:48.680303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:48.680303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:39:48.680303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:48.680303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:39:48.680303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:39:48.680303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:48.680993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:39:48.680993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:48.680993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:39:48.752599 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:39:48.753717 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:39:48.753945 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:39:48.755097 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:39:48.755350 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:39:48.755508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:48.757019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:39:48.757019 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:48.757019 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:39:48.757019 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:39:48.757019 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:48.757019 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:39:48.757019 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:48.757019 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:39:48.758071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:48.758071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:39:48.758071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:39:48.758071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:39:48.758071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:39:48.758071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:39:48.758071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:39:48.758071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:48.758071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:39:48.758071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:48.758071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:39:48.758071 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:48.758666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:39:48.758666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:48.758666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:39:48.758666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:48.758666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:39:48.758666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:39:48.758666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:48.758666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:39:48.758666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:48.758666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:39:48.758666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:39:48.758666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:48.758666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:39:48.759361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:48.759361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:39:48.759361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:48.759361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:39:48.759361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:39:48.759361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:39:48.759361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:39:48.759361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:39:48.759361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:39:48.759361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:39:48.759361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:39:48.759361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:39:48.759361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:39:48.759361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:39:48.759361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:39:48.759951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:39:48.759951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:39:48.759951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:39:48.759951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:39:48.759951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:39:48.759951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:48.759951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:39:48.759951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:48.759951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:39:48.759951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:48.759951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:39:48.759951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:48.759951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:39:48.759951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:48.759951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:39:48.759951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:39:48.759951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:48.759951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:39:48.759951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:48.760475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:39:48.760475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:39:48.760475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:48.760475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:39:48.760475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:48.760475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:39:48.760475 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:39:48.760670 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:39:48.760713 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:39:48.760877 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:39:48.761239 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:39:48.761442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:48.761971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:39:48.761971 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:48.761971 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:39:48.761971 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:39:48.761971 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:48.761971 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:39:48.761971 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:48.761971 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:39:48.762698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:48.762698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:39:48.762698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:39:48.762698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:39:48.762698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:48.762698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:39:48.762698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:48.762698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:39:48.762698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:48.762698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:39:48.762698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:48.762698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:39:48.762698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:48.762698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:39:48.762698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:39:48.763200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:48.763200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:39:48.763200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:48.763200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:39:48.763200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:39:48.763200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:48.763200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:39:48.763200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:48.763200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:39:48.763608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:48.763608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:39:48.763608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:39:48.763608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:39:48.763608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:39:48.763608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:39:48.763608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:39:48.763608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:39:48.763608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:39:48.763608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:39:48.763608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:48.763608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:39:48.763999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:48.763999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:39:48.763999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:48.763999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:39:48.763999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:48.763999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:39:48.763999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:48.763999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:39:48.763999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:39:48.763999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:48.763999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:39:48.763999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:48.763999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:39:48.763999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:39:48.764387 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:48.764387 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:39:48.764387 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:48.764387 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:39:48.764619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:48.764619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:39:48.764619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:39:48.764619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:39:48.764619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:39:48.764619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:39:48.764619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:39:48.764619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:39:48.764619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:39:48.764619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:39:48.764619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:39:48.764619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:39:48.764619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:39:48.764619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:39:48.764619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:39:48.764619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:39:48.765056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:39:48.765056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:39:48.765056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:39:48.765056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:39:48.765056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:39:48.765056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:48.765056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:39:48.765056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:48.765056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:39:48.765056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:48.765056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:39:48.765056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:48.765056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:39:48.765056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:48.765056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:39:48.765056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:39:48.765056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:48.765056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:39:48.765626 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:48.765626 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:39:48.765626 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:39:48.765626 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:48.765626 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:39:48.765626 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:48.765626 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:39:48.769863 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:39:48.770212 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:39:48.770334 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:39:48.770459 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:39:48.770899 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:39:48.771123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:48.771655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:39:48.771655 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:48.771655 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:39:48.771655 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:39:48.771655 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:48.771655 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:39:48.771655 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:48.771655 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:39:48.772822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:48.772822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:39:48.772822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:39:48.772822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:39:48.772822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:48.772822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:39:48.772822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:48.772822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:39:48.772822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:48.772822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:39:48.772822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:48.772822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:39:48.772822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:48.772822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:39:48.772822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:39:48.773347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:48.773347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:39:48.773347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:48.773347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:39:48.773347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:39:48.773347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:48.773347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:39:48.773347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:48.773347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:39:48.774225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:48.774225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:39:48.774225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:39:48.774225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:39:48.774225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:39:48.774225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:39:48.774225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:39:48.774225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:39:48.774225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:39:48.774225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:48.774225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:39:48.774684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:48.774684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:39:48.774684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:48.774684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:39:48.774684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:48.774684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:39:48.774684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:48.774684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:39:48.774684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:39:48.774684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:48.774684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:39:48.774684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:48.774684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:39:48.774684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:39:48.775372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:48.775372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:39:48.775372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:48.775372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:39:48.775957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:48.775957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:39:48.775957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:39:48.775957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:39:48.775957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:39:48.775957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:39:48.775957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:39:48.775957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:39:48.775957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:39:48.775957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:39:48.775957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:39:48.775957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:39:48.775957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:39:48.775957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:39:48.775957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:39:48.775957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:39:48.775957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:39:48.776597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:39:48.776597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:39:48.776597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:39:48.776597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:48.776597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:39:48.776597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:48.776597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:39:48.776597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:48.776597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:39:48.776597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:48.776597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:39:48.776597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:48.776597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:39:48.776597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:39:48.776597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:48.776597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:39:48.776597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:48.776597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:39:48.776597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:39:48.777323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:48.777323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:39:48.777323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:48.777323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:39:48.777323 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:39:48.777323 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:39:48.777323 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:39:48.777323 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:39:48.777692 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:39:48.777692 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:39:48.777833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:48.778368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:39:48.778368 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:48.778368 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:39:48.778368 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:39:48.778368 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:48.778368 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:39:48.778368 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:48.778368 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:39:48.779655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:48.779655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:39:48.779655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:39:48.779655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:39:48.779655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:39:48.779655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:39:48.779655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:39:48.779655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:48.779655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:39:48.779655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:48.779655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:39:48.779655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:48.780134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:39:48.780134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:48.780134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:39:48.780134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:48.780134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:39:48.780134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:39:48.780134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:48.780134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:39:48.780134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:48.780134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:39:48.780134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:39:48.780134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:48.780134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:39:48.780617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:48.780617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:39:48.781216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:48.781216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:39:48.781216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:39:48.781216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:39:48.781216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:39:48.781216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:39:48.781216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:39:48.781216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:39:48.781216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:39:48.781216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:39:48.781216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:39:48.781216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:39:48.781216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:39:48.781216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:39:48.781216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:39:48.781216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:39:48.781845 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:39:48.781845 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:39:48.781845 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:48.781845 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:39:48.781845 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:48.781845 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:39:48.781845 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:48.781845 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:39:48.781845 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:48.781845 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:39:48.781845 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:48.781845 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:39:48.781845 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:39:48.781845 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:48.781845 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:39:48.781845 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:48.781845 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:39:48.781845 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:39:48.781845 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:48.782548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:39:48.782548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:48.782548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:39:53.602346 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:39:53.603358 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:39:53.603358 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:39:53.603935 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:39:53.638499 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:39:53.638836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:53.639435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:39:53.639435 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:53.639435 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:39:53.639435 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:39:53.639435 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:53.639435 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:39:53.639435 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:53.639435 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:39:53.640494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:53.640494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:39:53.640494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:39:53.640494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:39:53.640494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:39:53.640494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:39:53.640494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:39:53.640494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:53.640494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:39:53.640494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:53.640494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:39:53.640494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:53.641278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:39:53.641278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:53.641278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:39:53.641278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:53.641278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:39:53.641278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:39:53.641278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:53.641278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:39:53.641278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:53.641278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:39:53.641278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:39:53.641278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:53.641278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:39:53.641848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:53.641848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:39:53.641848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:53.641848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:39:53.641848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:39:53.641848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:39:53.641848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:39:53.641848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:39:53.641848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:39:53.641848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:39:53.641848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:39:53.641848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:39:53.641848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:39:53.641848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:39:53.641848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:39:53.643690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:39:53.643690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:39:53.643690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:39:53.643690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:39:53.643690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:39:53.643690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:53.643690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:39:53.643690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:53.643690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:39:53.643690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:53.643690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:39:53.643690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:53.643690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:39:53.643690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:53.643690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:39:53.643690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:39:53.643690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:53.643690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:39:53.643690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:53.644467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:39:53.644467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:39:53.644467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:53.644467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:39:53.644467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:53.644467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:39:53.753794 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:39:53.754481 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:39:53.754677 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:39:53.756014 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:39:53.756452 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:39:53.756756 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:53.757868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:39:53.757868 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:53.757868 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:39:53.757868 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:39:53.757868 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:53.757868 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:39:53.757868 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:53.757868 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:39:53.759523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:53.759523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:39:53.759523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:39:53.759523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:39:53.759523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:39:53.759523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:39:53.759523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:39:53.759523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:53.759523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:39:53.759523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:53.759523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:39:53.759523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:53.760103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:39:53.760103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:53.760103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:39:53.760103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:53.760103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:39:53.760103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:39:53.760103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:53.760103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:39:53.760103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:53.760103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:39:53.760103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:39:53.760103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:53.760103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:39:53.760634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:53.760634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:39:53.761266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:53.761266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:39:53.761266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:39:53.761266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:39:53.761266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:39:53.761266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:39:53.761266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:39:53.761266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:39:53.761266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:39:53.761266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:39:53.761266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:39:53.761266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:39:53.761266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:39:53.761266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:39:53.761266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:39:53.761266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:39:53.763580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:39:53.763580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:39:53.763580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:53.763580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:39:53.763580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:53.763580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:39:53.763580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:53.763580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:39:53.763580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:53.763580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:39:53.763580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:53.763580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:39:53.763580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:39:53.763580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:53.763580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:39:53.763580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:53.763580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:39:53.763580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:39:53.763580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:53.764711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:39:53.764711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:53.764711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:39:53.764711 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:39:53.764711 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:39:53.764711 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:39:53.764711 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:39:53.764711 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:39:53.765117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:53.765117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:39:53.765117 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:53.765117 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:39:53.765117 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:39:53.765117 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:53.765117 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:39:53.765117 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:53.765117 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:39:53.765117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:53.765117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:39:53.765117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:39:53.765117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:39:53.765117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:53.765117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:39:53.765117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:53.765117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:39:53.766186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:53.766186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:39:53.766186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:53.766186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:39:53.766186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:53.766186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:39:53.766186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:39:53.766186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:53.766186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:39:53.766186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:53.766186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:39:53.766186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:39:53.766186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:53.766186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:39:53.766186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:53.766186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:39:53.768184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:53.768184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:39:53.768184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:39:53.768184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:39:53.768184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:39:53.768184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:39:53.768184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:39:53.768184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:39:53.768184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:39:53.768184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:39:53.768184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:53.768184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:39:53.769344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:53.769344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:39:53.769344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:53.769344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:39:53.769344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:53.769344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:39:53.769344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:53.769344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:39:53.769344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:39:53.769344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:53.769344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:39:53.769344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:53.769344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:39:53.769344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:39:53.770275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:53.770275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:39:53.770275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:53.770275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:39:53.770275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:53.770275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:39:53.770275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:39:53.770275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:39:53.770275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:39:53.770275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:39:53.770275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:39:53.770275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:39:53.770275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:39:53.770275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:39:53.770790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:39:53.770790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:39:53.770790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:39:53.770790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:39:53.770790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:39:53.770790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:39:53.770790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:39:53.770790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:39:53.770790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:39:53.770790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:39:53.770790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:39:53.770790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:53.770790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:39:53.770790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:53.770790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:39:53.770790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:53.770790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:39:53.770790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:53.770790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:39:53.771122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:53.771122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:39:53.771122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:39:53.771122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:53.771122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:39:53.771122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:53.771122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:39:53.771122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:39:53.771122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:53.771122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:39:53.771122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:53.771122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:39:53.771122 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:39:53.771122 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:39:53.771372 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:39:53.771372 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:39:53.771372 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:39:53.771657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:53.772061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:39:53.772061 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:53.772061 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:39:53.772061 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:39:53.772061 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:53.772061 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:39:53.772061 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:53.772061 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:39:53.772917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:53.772917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:39:53.772917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:39:53.772917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:39:53.772917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:53.772917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:39:53.772917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:53.772917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:39:53.772917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:53.772917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:39:53.772917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:53.772917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:39:53.772917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:53.772917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:39:53.772917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:39:53.773646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:53.773646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:39:53.773646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:53.773646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:39:53.773646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:39:53.773646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:53.773646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:39:53.773646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:53.773646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:39:53.773974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:53.773974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:39:53.773974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:39:53.773974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:39:53.773974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:39:53.773974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:39:53.773974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:39:53.773974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:39:53.773974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:39:53.773974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:53.773974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:39:53.774283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:53.774283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:39:53.774283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:53.774283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:39:53.774283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:53.774283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:39:53.774283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:53.774283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:39:53.774283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:39:53.774283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:53.774283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:39:53.774283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:53.774283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:39:53.774283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:39:53.774609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:53.774609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:39:53.774609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:53.774609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:39:53.775418 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:53.775418 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:39:53.775418 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:39:53.775418 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:39:53.775418 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:39:53.775418 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:39:53.775418 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:39:53.775418 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:39:53.775418 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:39:53.775418 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:39:53.775418 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:39:53.775418 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:39:53.775418 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:39:53.775418 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:39:53.775418 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:39:53.775418 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:39:53.775418 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:39:53.775809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:39:53.775809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:39:53.775809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:39:53.775809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:53.775809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:39:53.775809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:53.775809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:39:53.775809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:53.775809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:39:53.775809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:53.775809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:39:53.775809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:53.775809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:39:53.775809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:39:53.775809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:53.775809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:39:53.775809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:53.775809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:39:53.775809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:39:53.776304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:53.776304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:39:53.776304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:53.776304 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:39:53.776724 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:39:53.777081 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:39:53.777309 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:39:53.777599 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:39:53.777825 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:39:53.778053 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:39:53.778192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:53.778735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:39:53.778735 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:53.778735 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:39:53.778735 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:39:53.778735 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:53.778735 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:39:53.778735 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:53.778735 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:39:53.779638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:53.779638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:39:53.779638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:39:53.779638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:39:53.779638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:39:53.779638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:39:53.779638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:39:53.779638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:53.779638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:39:53.779638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:53.779638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:39:53.779638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:53.780178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:39:53.780178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:53.780178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:39:53.780178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:53.780178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:39:53.780178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:39:53.780178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:53.780178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:39:53.780178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:53.780178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:39:53.780178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:39:53.780178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:53.780178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:39:53.780879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:53.780879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:39:53.780879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:53.780879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:39:53.780879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:39:53.780879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:39:53.780879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:39:53.780879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:39:53.780879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:39:53.780879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:39:53.780879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:39:53.780879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:39:53.780879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:39:53.780879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:39:53.780879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:39:53.781551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:39:53.781551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:39:53.781551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:39:53.781551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:39:53.781551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:39:53.781551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:53.781551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:39:53.781551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:53.781551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:39:53.781551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:53.781551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:39:53.781551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:53.781551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:39:53.781551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:53.781551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:39:53.781551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:39:53.781551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:53.781551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:39:53.781551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:53.783047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:39:53.783047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:39:53.783047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:53.783047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:39:53.783047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:53.783047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:39:58.603042 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:39:58.604254 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:39:58.604254 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:39:58.604834 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:39:58.606237 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG dbcounter [-] [106314] Writing DB stats neutron:SELECT=434 {{(pid=106314) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 09 09:39:58.645065 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:39:58.645551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:58.646616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:39:58.646616 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:58.646616 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:39:58.646616 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:39:58.646616 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:58.646616 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:39:58.646616 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:58.646616 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:39:58.647397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.043 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:58.647397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:39:58.647397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:39:58.647397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:39:58.647397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:39:58.647397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:39:58.647397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:39:58.647397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:58.647397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:39:58.647397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:58.647397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:39:58.647397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:58.648240 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:39:58.648240 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:58.648240 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:39:58.648240 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:58.648240 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:39:58.648240 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:39:58.648240 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:58.648240 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:39:58.648240 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:58.648240 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:39:58.648240 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:39:58.648240 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:58.648240 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:39:58.648996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:58.648996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:39:58.648996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:58.648996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:39:58.648996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:39:58.648996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:39:58.648996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:39:58.648996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:39:58.648996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:39:58.648996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:39:58.648996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:39:58.648996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:39:58.648996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:39:58.648996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:39:58.648996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:39:58.649722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:39:58.649722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:39:58.649722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:39:58.649722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:39:58.649722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:39:58.649722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:58.649722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:39:58.649722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:58.649722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:39:58.649722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:58.649722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:39:58.649722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:58.649722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:39:58.649722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:58.649722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:39:58.649722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:39:58.649722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:58.649722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:39:58.649722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:58.652118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:39:58.652118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:39:58.652118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:58.652118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:39:58.652118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:58.652118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:39:58.754704 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:39:58.755400 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:39:58.755520 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:39:58.756516 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:39:58.756732 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:39:58.756904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:58.757788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:39:58.757788 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:58.757788 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:39:58.757788 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:39:58.757788 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:58.757788 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:39:58.757788 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:58.757788 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:39:58.758919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:58.758919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:39:58.758919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:39:58.758919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:39:58.758919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:39:58.758919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:39:58.758919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:39:58.758919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:58.758919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:39:58.758919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:58.758919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:39:58.758919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:58.759800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:39:58.759800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:58.759800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:39:58.759800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:58.759800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:39:58.759800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:39:58.759800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:58.759800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:39:58.759800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:58.759800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:39:58.759800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:39:58.759800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:58.759800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:39:58.760704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:58.760704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:39:58.760704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:58.760704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:39:58.760704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:39:58.760704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:39:58.760704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:39:58.760704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:39:58.760704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:39:58.760704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:39:58.760704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:39:58.760704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:39:58.760704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:39:58.760704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:39:58.760704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:39:58.761322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:39:58.761322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:39:58.761322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:39:58.761322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:39:58.761322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:39:58.761322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:58.761322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:39:58.761322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:58.761322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:39:58.761322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:58.761322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:39:58.761322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:58.761322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:39:58.761322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:58.761322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:39:58.761322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:39:58.761322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:58.761322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:39:58.761322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:58.761874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:39:58.761874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:39:58.761874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:58.761874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:39:58.761874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:58.761874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:39:58.762064 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:39:58.762381 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:39:58.762491 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:39:58.762613 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:39:58.763041 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:39:58.763250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:58.763766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:39:58.763766 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:58.763766 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:39:58.763766 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:39:58.763766 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:58.763766 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:39:58.763766 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:58.763766 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:39:58.764529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:58.764529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:39:58.764529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:39:58.764529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:39:58.764529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:58.764529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:39:58.764529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:58.764529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:39:58.764529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:58.764529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:39:58.764529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:58.764529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:39:58.764529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:58.764529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:39:58.764529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:39:58.765561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:58.765561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:39:58.765561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:58.765561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:39:58.765561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:39:58.765561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:58.765561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:39:58.765561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:58.765561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:39:58.765561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:58.765561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:39:58.765561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:39:58.765561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:39:58.765561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:39:58.766223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:39:58.766223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:39:58.766223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:39:58.766223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:39:58.766223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:39:58.766223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:58.766223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:39:58.766223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:58.766223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:39:58.766223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:58.766223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:39:58.766223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:58.766223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:39:58.766606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:58.766606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:39:58.766606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:39:58.766606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:58.766606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:39:58.766606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:58.766606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:39:58.766606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:39:58.766606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:58.766606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:39:58.766606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:58.766606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:39:58.766925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:58.766925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:39:58.766925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:39:58.766925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:39:58.766925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:39:58.766925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:39:58.766925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:39:58.766925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:39:58.766925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:39:58.766925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:39:58.766925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:39:58.766925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:39:58.766925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:39:58.766925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:39:58.766925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:39:58.766925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:39:58.767343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:39:58.767343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:39:58.767343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:39:58.767343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:39:58.767343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:39:58.767343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:58.767343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:39:58.767343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:58.767343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:39:58.767343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:58.767343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:39:58.767343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:58.767343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:39:58.767343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:58.767343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:39:58.767343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:39:58.767343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:58.767343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:39:58.767976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:58.767976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:39:58.767976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:39:58.767976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:58.767976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:39:58.767976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:58.767976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:39:58.770871 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:39:58.771146 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:39:58.771248 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:39:58.771380 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:39:58.771820 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:39:58.772158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:58.773645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:39:58.773645 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:58.773645 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:39:58.773645 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:39:58.773645 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:58.773645 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:39:58.773645 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:58.773645 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:39:58.774431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:58.774431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:39:58.774431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:39:58.774431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:39:58.774431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:58.774431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:39:58.774431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:58.774431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:39:58.774431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:58.774431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:39:58.774431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:58.774431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:39:58.774431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:58.774431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:39:58.774431 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:39:58.775150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:58.775150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:39:58.775150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:58.775150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:39:58.775150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:39:58.775150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:58.775150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:39:58.775150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:58.775150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:39:58.775708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:58.775708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:39:58.775708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:39:58.775708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:39:58.775708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:39:58.775708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:39:58.775708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:39:58.775708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:39:58.775708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:39:58.775708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:58.775708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:39:58.776432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:58.776432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:39:58.776432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:58.776432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:39:58.776432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:58.776432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:39:58.776432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:58.776432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:39:58.776432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:39:58.776432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:58.776432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:39:58.776432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:58.776432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:39:58.776432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:39:58.777165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:58.777165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:39:58.777165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:58.777165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:39:58.777165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:58.777165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:39:58.777165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:39:58.777165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:39:58.777165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:39:58.777165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:39:58.777165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:39:58.777165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:39:58.777165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:39:58.777165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:39:58.778907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:39:58.778907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:39:58.778907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:39:58.778907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:39:58.778907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:39:58.778907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:39:58.778907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:39:58.778907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:39:58.778907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:39:58.778907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:39:58.778907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:58.778907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:39:58.778907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:58.778907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:39:58.778907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:58.778907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:39:58.778907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:58.778907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:39:58.778907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:58.781129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:39:58.781129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:39:58.781129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:58.781129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:39:58.781129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:58.781129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:39:58.781129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:39:58.781129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:58.781129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:39:58.781129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:58.781129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:39:58.781129 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:39:58.781129 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:39:58.782452 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:39:58.782452 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:39:58.782452 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:39:58.782452 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:39:58.782452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:58.782452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:39:58.782452 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:58.782452 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:39:58.782452 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:39:58.783391 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:58.783391 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:39:58.783391 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:58.783391 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:39:58.783391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:58.783391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:39:58.783391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:39:58.783391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:39:58.783391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:39:58.783391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:39:58.783391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:39:58.783391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:58.783391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:39:58.783391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:58.784781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:39:58.784781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:58.784781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:39:58.784781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:58.784781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:39:58.784781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:58.784781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:39:58.784781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:39:58.784781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:58.784781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:39:58.784781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:58.784781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:39:58.784781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:39:58.784781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:58.785128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:39:58.785128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:58.785128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:39:58.785128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:58.785128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:39:58.785128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:39:58.785128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:39:58.785128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:39:58.785128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:39:58.785128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:39:58.785128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:39:58.785128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:39:58.785128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:39:58.785128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:39:58.785128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:39:58.785128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:39:58.785426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:39:58.785426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:39:58.785426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:39:58.785426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:39:58.785426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:39:58.785426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:58.785426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:39:58.785426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:39:58.785426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:39:58.785426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:39:58.785426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:39:58.785426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:39:58.785426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:39:58.785426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:39:58.785426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:39:58.785426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:39:58.785426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:39:58.785426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:39:58.785426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:39:58.786025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:39:58.786025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:39:58.786025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:39:58.786025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:39:58.786025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:39:58.786025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:40:03.603630 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:40:03.604574 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:40:03.604574 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:40:03.606341 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:40:03.646022 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:40:03.646501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:03.647395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:40:03.647395 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:03.647395 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:40:03.647395 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:40:03.647395 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:03.647395 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:40:03.647395 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:03.647395 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:40:03.648114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.043 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:03.648114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:40:03.648114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:40:03.648114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:40:03.648114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:40:03.648114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:40:03.648114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:40:03.648114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:03.648114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:40:03.648114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:03.648114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:40:03.648114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:03.648881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:40:03.648881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:03.648881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:40:03.648881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:03.648881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:40:03.648881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:40:03.648881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:03.648881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:40:03.648881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:03.648881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:40:03.648881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:40:03.648881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:03.648881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:40:03.649677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:03.649677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:40:03.649677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:03.649677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:40:03.649677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:40:03.649677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:40:03.649677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:40:03.649677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:40:03.649677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:40:03.649677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:40:03.649677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:40:03.649677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:40:03.649677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:40:03.649677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:40:03.649677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:40:03.650883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:40:03.650883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:40:03.650883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:40:03.650883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:40:03.650883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:40:03.650883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:03.650883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:40:03.650883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:03.650883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:40:03.650883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:03.650883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:40:03.650883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:03.650883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:40:03.650883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:03.650883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:40:03.650883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:40:03.650883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:03.650883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:40:03.650883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:03.652051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:40:03.652051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:40:03.652051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:03.652051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:40:03.652051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:03.652051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:40:03.755518 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:40:03.756238 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:40:03.756465 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:40:03.757654 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:40:03.758063 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:40:03.758344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:03.759372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:40:03.759372 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:03.759372 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:40:03.759372 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:40:03.759372 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:03.759372 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:40:03.759372 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:03.759372 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:40:03.760983 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:03.760983 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:40:03.760983 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:40:03.760983 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:40:03.760983 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:40:03.760983 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:40:03.760983 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:40:03.760983 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:03.760983 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:40:03.760983 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:03.760983 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:40:03.760983 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:03.761602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:40:03.761602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:03.761602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:40:03.761602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:03.761602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:40:03.761602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:40:03.761602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:03.761602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:40:03.761602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:03.761602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:40:03.761602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:40:03.761602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:03.761602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:40:03.762218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:03.762218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:40:03.763202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:03.763202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:40:03.763202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:40:03.763202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:40:03.763202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:40:03.763202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:40:03.763202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:40:03.763202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:40:03.763202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:40:03.763202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:40:03.763202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:40:03.763202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:40:03.763202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:40:03.763202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:40:03.763202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:40:03.763202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:40:03.763968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:40:03.763968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:40:03.763968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:03.763968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:40:03.763968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:03.763968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:40:03.763968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:03.763968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:40:03.763968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:03.763968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:40:03.763968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:03.763968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:40:03.763968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:40:03.763968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:03.763968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:40:03.763968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:03.763968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:40:03.763968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:40:03.763968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:03.764819 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:40:03.764819 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:03.764819 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:40:03.764819 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:40:03.764819 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:40:03.764819 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:40:03.764819 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:40:03.765296 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:40:03.765598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:03.766384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:40:03.766384 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:03.766384 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:40:03.766384 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:40:03.766384 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:03.766384 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:40:03.766384 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:03.766384 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:40:03.767469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:03.767469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:40:03.767469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:40:03.767469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:40:03.767469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:03.767469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:40:03.767469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:03.767469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:40:03.767469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:03.767469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:40:03.767469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:03.767469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:40:03.767469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:03.767469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:40:03.767469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:40:03.768058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:03.768058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:40:03.768058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:03.768058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:40:03.768058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:40:03.768058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:03.768058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:40:03.768058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:03.768058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:40:03.768319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:03.768319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:40:03.768319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:40:03.768319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:40:03.768319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:40:03.768319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:40:03.768319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:40:03.768319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:40:03.768319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:40:03.768319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:40:03.768319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:03.768319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:40:03.768688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:03.768688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:40:03.768688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:03.768688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:40:03.768688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:03.768688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:40:03.768688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:03.768688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:40:03.768688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:40:03.768688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:03.768688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:40:03.768688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:03.768688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:40:03.768688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:40:03.769180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:03.769180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:40:03.769180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:03.769180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:40:03.769416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:03.769416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:40:03.769416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:40:03.769416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:40:03.769416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:40:03.769416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:40:03.769416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:40:03.769416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:40:03.769416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:40:03.769416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:40:03.769416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:40:03.769416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:40:03.769416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:40:03.769416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:40:03.769416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:40:03.769416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:40:03.769892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:40:03.769892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:40:03.769892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:40:03.769892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:40:03.769892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:40:03.769892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:03.769892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:40:03.769892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:03.769892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:40:03.769892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:03.769892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:40:03.769892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:03.769892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:40:03.769892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:03.769892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:40:03.769892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:40:03.769892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:03.769892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:40:03.770460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:03.770460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:40:03.770460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:40:03.770460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:03.770460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:40:03.770460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:03.770460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:40:03.771190 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:40:03.771465 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:40:03.771582 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:40:03.771715 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:40:03.772104 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:40:03.772361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:03.772912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:40:03.772912 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:03.772912 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:40:03.772912 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:40:03.772912 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:03.772912 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:40:03.772912 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:03.772912 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:40:03.773664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:03.773664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:40:03.773664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:40:03.773664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:40:03.773664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:03.773664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:40:03.773664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:03.773664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:40:03.773664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:03.773664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:40:03.773664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:03.773664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:40:03.773664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:03.773664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:40:03.773664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:40:03.774161 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:03.774161 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:40:03.774161 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:03.774161 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:40:03.774161 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:40:03.774161 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:03.774161 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:40:03.774161 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:03.774161 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:40:03.774639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:03.774639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:40:03.774639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:40:03.774639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:40:03.774639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:40:03.774639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:40:03.774639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:40:03.774639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:40:03.774639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:40:03.774639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:03.774639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:40:03.775031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:03.775031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:40:03.775031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:03.775031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:40:03.775031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:03.775031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:40:03.775031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:03.775031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:40:03.775031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:40:03.775031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:03.775031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:40:03.775031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:03.775031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:40:03.775031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:40:03.775396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:03.775396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:40:03.775396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:03.775396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:40:03.775966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:03.775966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:40:03.775966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:40:03.775966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:40:03.775966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:40:03.775966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:40:03.775966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:40:03.775966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:40:03.775966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:40:03.775966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:40:03.775966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:40:03.775966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:40:03.775966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:40:03.775966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:40:03.775966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:40:03.775966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:40:03.775966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:40:03.776465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:40:03.776465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:40:03.776465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:40:03.776465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:03.776465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:40:03.776465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:03.776465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:40:03.776465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:03.776465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:40:03.776465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:03.776465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:40:03.776465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:03.776465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:40:03.776465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:40:03.776465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:03.776465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:40:03.776465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:03.776465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:40:03.776465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:40:03.777004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:03.777004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:40:03.777004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:03.777004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:40:03.777449 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:40:03.777715 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:40:03.777918 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:40:03.778105 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:40:03.778485 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:40:03.778795 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:40:03.778970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:03.779512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:40:03.779512 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:03.779512 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:40:03.779512 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:40:03.779512 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:03.779512 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:40:03.779512 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:03.779512 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:40:03.780363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:03.780363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:40:03.780363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:40:03.780363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:40:03.780363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:40:03.780363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:40:03.780363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:40:03.780363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:03.780363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:40:03.780363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:03.780363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:40:03.780363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:03.780710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:40:03.780710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:03.780710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:40:03.780710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:03.780710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:40:03.780710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:40:03.780710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:03.780710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:40:03.780710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:03.780710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:40:03.780710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:40:03.780710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:03.780710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:40:03.781124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:03.781124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:40:03.781349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:03.781349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:40:03.781349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:40:03.781349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:40:03.781349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:40:03.781349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:40:03.781349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:40:03.781349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:40:03.781349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:40:03.781349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:40:03.781349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:40:03.781349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:40:03.781349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:40:03.781349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:40:03.781349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:40:03.781349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:40:03.781800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:40:03.781800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:40:03.781800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:03.781800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:40:03.781800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:03.781800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:40:03.781800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:03.781800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:40:03.781800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:03.781800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:40:03.781800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:03.781800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:40:03.781800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:40:03.781800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:03.781800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:40:03.781800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:03.781800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:40:03.781800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:40:03.781800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:03.782320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:40:03.782320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:03.782320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:40:08.604936 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:40:08.606098 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:40:08.606098 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:40:08.607205 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:40:08.642233 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:40:08.642466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:08.643370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:40:08.643370 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:08.643370 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:40:08.643370 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:40:08.643370 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:08.643370 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:40:08.643370 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:08.643370 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:40:08.644143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:08.644143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:40:08.644143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:40:08.644143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:40:08.644143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:40:08.644143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:40:08.644143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:40:08.644143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:08.644143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:40:08.644143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:08.644143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:40:08.644143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:08.644912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:40:08.644912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:08.644912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:40:08.644912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:08.644912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:40:08.644912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:40:08.644912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:08.644912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:40:08.644912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:08.644912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:40:08.644912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:40:08.644912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:08.644912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:40:08.645680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:08.645680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:40:08.645680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:08.645680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:40:08.645680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:40:08.645680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:40:08.645680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:40:08.645680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:40:08.645680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:40:08.645680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:40:08.645680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:40:08.645680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:40:08.645680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:40:08.645680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:40:08.645680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:40:08.646963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:40:08.646963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:40:08.646963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:40:08.646963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:40:08.646963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:40:08.646963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:08.646963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:40:08.646963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:08.646963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:40:08.646963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:08.646963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:40:08.646963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:08.646963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:40:08.646963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:08.646963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:40:08.646963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:40:08.646963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:08.646963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:40:08.646963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:08.648273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:40:08.648273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:40:08.648273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:08.648273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:40:08.648273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:08.648273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:40:08.756340 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:40:08.756979 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:40:08.757175 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:40:08.758473 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:40:08.758838 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:40:08.759075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:08.760117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:40:08.760117 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:08.760117 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:40:08.760117 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:40:08.760117 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:08.760117 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:40:08.760117 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:08.760117 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:40:08.761955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:08.761955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:40:08.761955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:40:08.761955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:40:08.761955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:40:08.761955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:40:08.761955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:40:08.761955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:08.761955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:40:08.761955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:08.761955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:40:08.761955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:08.762380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:40:08.762380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:08.762380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:40:08.762380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:08.762380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:40:08.762380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:40:08.762380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:08.762380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:40:08.762380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:08.762380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:40:08.762380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:40:08.762380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:08.762380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:40:08.762729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:08.762729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:40:08.763913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:08.763913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:40:08.763913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:40:08.763913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:40:08.763913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:40:08.763913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:40:08.763913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:40:08.763913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:40:08.763913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:40:08.763913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:40:08.763913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:40:08.763913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:40:08.763913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:40:08.763913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:40:08.763913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:40:08.763913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:40:08.764366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:40:08.764366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:40:08.764366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:08.764366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:40:08.764366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:08.764366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:40:08.764366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:08.764366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:40:08.764366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:08.764366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:40:08.764366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:08.764366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:40:08.764366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:40:08.764366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:08.764366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:40:08.764366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:08.764366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:40:08.764366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:40:08.764366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:08.764720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:40:08.764720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:08.764720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:40:08.764720 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:40:08.764720 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:40:08.764851 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:40:08.765027 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:40:08.765617 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:40:08.765935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:08.766647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:40:08.766647 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:08.766647 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:40:08.766647 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:40:08.766647 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:08.766647 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:40:08.766647 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:08.766647 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:40:08.767692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:08.767692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:40:08.767692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:40:08.767692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:40:08.767692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:08.767692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:40:08.767692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:08.767692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:40:08.767692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:08.767692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:40:08.767692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:08.767692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:40:08.767692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:08.767692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:40:08.767692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:40:08.768037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:08.768037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:40:08.768037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:08.768037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:40:08.768037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:40:08.768037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:08.768037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:40:08.768037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:08.768037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:40:08.768843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:08.768843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:40:08.768843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:40:08.768843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:40:08.768843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:40:08.768843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:40:08.768843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:40:08.768843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:40:08.768843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:40:08.768843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:40:08.768843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:08.768843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:40:08.769115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:08.769115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:40:08.769115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:08.769115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:40:08.769115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:08.769115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:40:08.769115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:08.769115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:40:08.769115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:40:08.769115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:08.769115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:40:08.769115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:08.769115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:40:08.769115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:40:08.769392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:08.769392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:40:08.769392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:08.769392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:40:08.770070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:08.770070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:40:08.770070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:40:08.770070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:40:08.770070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:40:08.770070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:40:08.770070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:40:08.770070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:40:08.770070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:40:08.770070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:40:08.770070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:40:08.770070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:40:08.770070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:40:08.770070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:40:08.770070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:40:08.770070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:40:08.770477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:40:08.770477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:40:08.770477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:40:08.770477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:40:08.770477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:40:08.770477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:08.770477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:40:08.770477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:08.770477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:40:08.770477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:08.770477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:40:08.770477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:08.770477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:40:08.770477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:08.770477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:40:08.770477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:40:08.770477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:08.770477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:40:08.770924 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:08.770924 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:40:08.770924 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:40:08.770924 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:08.770924 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:40:08.770924 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:08.770924 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:40:08.771626 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:40:08.771946 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:40:08.772054 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:40:08.772188 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:40:08.772606 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:40:08.772952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:08.773468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:40:08.773468 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:08.773468 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:40:08.773468 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:40:08.773468 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:08.773468 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:40:08.773468 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:08.773468 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:40:08.774387 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:08.774387 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:40:08.774387 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:40:08.774387 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:40:08.774387 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:08.774387 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:40:08.774387 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:08.774387 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:40:08.774387 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:08.774387 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:40:08.774387 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:08.774387 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:40:08.774387 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:08.774387 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:40:08.774387 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:40:08.774801 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:08.774801 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:40:08.774801 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:08.774801 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:40:08.774801 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:40:08.774801 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:08.774801 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:40:08.774801 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:08.774801 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:40:08.775044 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:08.775044 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:40:08.775044 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:40:08.775044 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:40:08.775044 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:40:08.775044 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:40:08.775044 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:40:08.775044 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:40:08.775044 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:40:08.775044 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:08.775044 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:40:08.775366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:08.775366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:40:08.775366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:08.775366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:40:08.775366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:08.775366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:40:08.775366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:08.775366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:40:08.775366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:40:08.775366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:08.775366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:40:08.775366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:08.775366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:40:08.775366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:40:08.775698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:08.775698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:40:08.775698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:08.775698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:40:08.776139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:08.776139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:40:08.776139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:40:08.776139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:40:08.776139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:40:08.776139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:40:08.776139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:40:08.776139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:40:08.776139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:40:08.776139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:40:08.776139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:40:08.776139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:40:08.776139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:40:08.776139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:40:08.776139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:40:08.776139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:40:08.776139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:40:08.776562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:40:08.776562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:40:08.776562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:40:08.776562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:08.776562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:40:08.776562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:08.776562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:40:08.776562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:08.776562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:40:08.776562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:08.776562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:40:08.776562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:08.776562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:40:08.776562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:40:08.776562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:08.776562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:40:08.776562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:08.776562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:40:08.776562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:40:08.777067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:08.777067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:40:08.777067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:08.777067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:40:08.778171 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:40:08.778247 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:40:08.778291 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:40:08.778469 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:40:08.778800 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:40:08.779037 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:40:08.779188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:08.779715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:40:08.779715 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:08.779715 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:40:08.779715 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:40:08.779715 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:08.779715 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:40:08.779715 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:08.779715 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:40:08.780537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:08.780537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:40:08.780537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:40:08.780537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:40:08.780537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:40:08.780537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:40:08.780537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:40:08.780537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:08.780537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:40:08.780537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:08.780537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:40:08.780537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:08.780886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:40:08.780886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:08.780886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:40:08.780886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:08.780886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:40:08.780886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:40:08.780886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:08.780886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:40:08.780886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:08.780886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:40:08.780886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:40:08.780886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:08.780886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:40:08.781216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:08.781216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:40:08.781538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:08.781538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:40:08.781538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:40:08.781538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:40:08.781538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:40:08.781538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:40:08.781538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:40:08.781538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:40:08.781538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:40:08.781538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:40:08.781538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:40:08.781538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:40:08.781538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:40:08.781538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:40:08.781538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:40:08.781538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:40:08.781947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:40:08.781947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:40:08.781947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:08.781947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:40:08.781947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:08.781947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:40:08.781947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:08.781947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:40:08.781947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:08.781947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:40:08.781947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:08.781947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:40:08.781947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:40:08.781947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:08.781947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:40:08.781947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:08.781947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:40:08.781947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:40:08.781947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:08.782421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:40:08.782421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:08.782421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:40:13.605698 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:40:13.606569 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:40:13.606569 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:40:13.607377 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:40:13.641183 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:40:13.641351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:13.641844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:40:13.641844 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:13.641844 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:40:13.641844 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:40:13.641844 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:13.641844 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:40:13.641844 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:13.641844 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:40:13.642630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:13.642630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:40:13.642630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:40:13.642630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:40:13.642630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:40:13.642630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:40:13.642630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:40:13.642630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:13.642630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:40:13.642630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:13.642630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:40:13.642630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:13.643107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:40:13.643107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:13.643107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:40:13.643107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:13.643107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:40:13.643107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:40:13.643107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:13.643107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:40:13.643107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:13.643107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:40:13.643107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:40:13.643107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:13.643107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:40:13.643513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:13.643513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:40:13.643727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:13.643727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:40:13.643727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:40:13.643727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:40:13.643727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:40:13.643727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:40:13.643727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:40:13.643727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:40:13.643727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:40:13.643727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:40:13.643727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:40:13.643727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:40:13.643727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:40:13.643727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:40:13.643727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:40:13.643727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:40:13.644366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:40:13.644366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:40:13.644366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:13.644366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:40:13.644366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:13.644366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:40:13.644366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:13.644366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:40:13.644366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:13.644366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:40:13.644366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:13.644366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:40:13.644366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:40:13.644366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:13.644366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:40:13.644366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:13.644366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:40:13.644366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:40:13.644366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:13.646090 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:40:13.646090 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:13.646090 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:40:13.757154 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:40:13.757577 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:40:13.757685 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:40:13.758614 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:40:13.758968 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:40:13.759221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:13.760101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:40:13.760101 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:13.760101 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:40:13.760101 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:40:13.760101 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:13.760101 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:40:13.760101 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:13.760101 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:40:13.761258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:13.761258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:40:13.761258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:40:13.761258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:40:13.761258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:40:13.761258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:40:13.761258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:40:13.761258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:13.761258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:40:13.761258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:13.761258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:40:13.761258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:13.761878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:40:13.761878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:13.761878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:40:13.761878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:13.761878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:40:13.761878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:40:13.761878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:13.761878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:40:13.761878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:13.761878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:40:13.761878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:40:13.761878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:13.761878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:40:13.762657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:13.762657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:40:13.762657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:13.762657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:40:13.762657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:40:13.762657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:40:13.762657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:40:13.762657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:40:13.762657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:40:13.762657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:40:13.762657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:40:13.762657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:40:13.762657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:40:13.762657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:40:13.762657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:40:13.763439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:40:13.763439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:40:13.763439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:40:13.763439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:40:13.763439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:40:13.763439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:13.763439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:40:13.763439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:13.763439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:40:13.763439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:13.763439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:40:13.763439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:13.763439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:40:13.763439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:13.763439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:40:13.763439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:40:13.763439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:13.763439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:40:13.763439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:13.765444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:40:13.765444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:40:13.765444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:13.765444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:40:13.765444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:13.765444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:40:13.765444 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:40:13.765444 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:40:13.766136 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:40:13.766136 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:40:13.766136 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:40:13.766136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:13.766640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:40:13.766640 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:13.766640 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:40:13.766640 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:40:13.766640 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:13.766640 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:40:13.766640 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:13.766640 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:40:13.767657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:13.767657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:40:13.767657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:40:13.767657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:40:13.767657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:13.767657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:40:13.767657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:13.767657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:40:13.767657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:13.767657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:40:13.767657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:13.767657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:40:13.767657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:13.767657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:40:13.767657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:40:13.768115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:13.768115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:40:13.768115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:13.768115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:40:13.768115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:40:13.768115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:13.768115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:40:13.768115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:13.768115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:40:13.768664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:13.768664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:40:13.768664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:40:13.768664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:40:13.768664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:40:13.768664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:40:13.768664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:40:13.768664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:40:13.768664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:40:13.768664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:40:13.768664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:13.768664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:40:13.768938 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:13.768938 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:40:13.768938 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:13.768938 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:40:13.768938 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:13.768938 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:40:13.768938 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:13.768938 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:40:13.768938 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:40:13.768938 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:13.768938 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:40:13.768938 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:13.768938 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:40:13.768938 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:40:13.769178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:13.769178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:40:13.769178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:13.769178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:40:13.769674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:13.769674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:40:13.769674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:40:13.769674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:40:13.769674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:40:13.769674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:40:13.769674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:40:13.769674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:40:13.769674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:40:13.769674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:40:13.769674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:40:13.769674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:40:13.769674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:40:13.769674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:40:13.769674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:40:13.769674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:40:13.770022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:40:13.770022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:40:13.770022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:40:13.770022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:40:13.770022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:40:13.770022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:13.770022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:40:13.770022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:13.770022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:40:13.770022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:13.770022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:40:13.770022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:13.770022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:40:13.770022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:13.770022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:40:13.770022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:40:13.770022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:13.770022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:40:13.770359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:13.770359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:40:13.770359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:40:13.770359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:13.770359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:40:13.770359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:13.770359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:40:13.772048 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:40:13.772276 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:40:13.772385 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:40:13.772513 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:40:13.772830 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:40:13.773045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:13.773497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:40:13.773497 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:13.773497 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:40:13.773497 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:40:13.773497 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:13.773497 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:40:13.773497 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:13.773497 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:40:13.774244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:13.774244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:40:13.774244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:40:13.774244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:40:13.774244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:13.774244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:40:13.774244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:13.774244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:40:13.774244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:13.774244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:40:13.774244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:13.774244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:40:13.774244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:13.774244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:40:13.774244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:40:13.774776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:13.774776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:40:13.774776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:13.774776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:40:13.774776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:40:13.774776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:13.774776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:40:13.774776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:13.774776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:40:13.775128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:13.775128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:40:13.775128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:40:13.775128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:40:13.775128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:40:13.775128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:40:13.775128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:40:13.775128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:40:13.775128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:40:13.775128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:13.775128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:40:13.775356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:13.775356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:40:13.775356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:13.775356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:40:13.775356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:13.775356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:40:13.775356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:13.775356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:40:13.775356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:40:13.775356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:13.775356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:40:13.775356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:13.775356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:40:13.775356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:40:13.775690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:13.775690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:40:13.775690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:13.775690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:40:13.776246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:13.776246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:40:13.776246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:40:13.776246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:40:13.776246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:40:13.776246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:40:13.776246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:40:13.776246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:40:13.776246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:40:13.776246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:40:13.776246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:40:13.776246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:40:13.776246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:40:13.776246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:40:13.776246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:40:13.776246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:40:13.776246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:40:13.776769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:40:13.776769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:40:13.776769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:40:13.776769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:13.776769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:40:13.776769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:13.776769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:40:13.776769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:13.776769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:40:13.776769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:13.776769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:40:13.776769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:13.776769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:40:13.776769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:40:13.776769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:13.776769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:40:13.776769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:13.776769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:40:13.776769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:40:13.777244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:13.777244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:40:13.777244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:13.777244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:40:13.778420 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:40:13.778605 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:40:13.778710 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:40:13.778929 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:40:13.779294 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:40:13.779653 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:40:13.779997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:13.780630 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:40:13.780630 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:13.780630 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:40:13.780630 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:40:13.780630 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:13.780630 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:40:13.780630 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:13.780630 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:40:13.781490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:13.781490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:40:13.781490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:40:13.781490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:40:13.781490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:40:13.781490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:40:13.781490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:40:13.781490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:13.781490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:40:13.781490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:13.781490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:40:13.781490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:13.781949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:40:13.781949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:13.781949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:40:13.781949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:13.781949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:40:13.781949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:40:13.781949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:13.781949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:40:13.781949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:13.781949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:40:13.781949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:40:13.781949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:13.781949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:40:13.782283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:13.782283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:40:13.782425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:13.782425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:40:13.782425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:40:13.782425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:40:13.782425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:40:13.782425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:40:13.782425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:40:13.782425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:40:13.782425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:40:13.782425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:40:13.782425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:40:13.782425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:40:13.782425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:40:13.782425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:40:13.782425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:40:13.782425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:40:13.783345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:40:13.783345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:40:13.783345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:13.783345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:40:13.783345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:13.783345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:40:13.783345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:13.783345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:40:13.783345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:13.783345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:40:13.783345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:13.783345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:40:13.783345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:40:13.783345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:13.783345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:40:13.783345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:13.783345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:40:13.783345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:40:13.783345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:13.784358 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:40:13.784358 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:13.784358 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:40:18.606025 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:40:18.606785 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:40:18.606785 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:40:18.607568 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:40:18.648991 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:40:18.648991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:18.649328 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:40:18.649328 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:18.649328 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:40:18.649328 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:40:18.649328 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:18.649328 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:40:18.649328 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:18.649328 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:40:18.650250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.043 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:18.650250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:40:18.650250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:40:18.650250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:40:18.650250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:40:18.650250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:40:18.650250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:40:18.650250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:18.650250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:40:18.650250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:18.650250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:40:18.650250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:18.651013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:40:18.651013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:18.651013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:40:18.651013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:18.651013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:40:18.651013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:40:18.651013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:18.651013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:40:18.651013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:18.651013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:40:18.651013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:40:18.651013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:18.651013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:40:18.651637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:18.651637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:40:18.651637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:18.651637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:40:18.651637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:40:18.651637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:40:18.651637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:40:18.651637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:40:18.651637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:40:18.651637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:40:18.651637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:40:18.651637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:40:18.651637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:40:18.651637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:40:18.651637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:40:18.652689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:40:18.652689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:40:18.652689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:40:18.652689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:40:18.652689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:40:18.652689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:18.652689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:40:18.652689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:18.652689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:40:18.652689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:18.652689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:40:18.652689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:18.652689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:40:18.652689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:18.652689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:40:18.652689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:40:18.652689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:18.652689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:40:18.652689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:18.653357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:40:18.653357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:40:18.653357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:18.653357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:40:18.653357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:18.653357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:40:18.757842 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:40:18.758512 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:40:18.758717 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:40:18.760212 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:40:18.760212 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:40:18.760458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:18.761165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:40:18.761165 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:18.761165 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:40:18.761165 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:40:18.761165 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:18.761165 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:40:18.761165 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:18.761165 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:40:18.762724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:18.762724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:40:18.762724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:40:18.762724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:40:18.762724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:40:18.762724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:40:18.762724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:40:18.762724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:18.762724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:40:18.762724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:18.762724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:40:18.762724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:18.763035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:40:18.763035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:18.763035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:40:18.763035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:18.763035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:40:18.763035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:40:18.763035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:18.763035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:40:18.763035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:18.763035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:40:18.763035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:40:18.763035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:18.763035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:40:18.763275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:18.763275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:40:18.764412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:18.764412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:40:18.764412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:40:18.764412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:40:18.764412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:40:18.764412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:40:18.764412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:40:18.764412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:40:18.764412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:40:18.764412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:40:18.764412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:40:18.764412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:40:18.764412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:40:18.764412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:40:18.764412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:40:18.764412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:40:18.764724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:40:18.764724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:40:18.764724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:18.764724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:40:18.764724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:18.764724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:40:18.764724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:18.764724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:40:18.764724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:18.764724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:40:18.764724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:18.764724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:40:18.764724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:40:18.764724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:18.764724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:40:18.764724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:18.764724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:40:18.764724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:40:18.764724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:18.765104 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:40:18.765104 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:18.765104 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:40:18.765261 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:40:18.765623 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:40:18.765839 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:40:18.766063 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:40:18.766814 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:40:18.766814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:18.767066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:40:18.767066 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:18.767066 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:40:18.767066 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:40:18.767066 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:18.767066 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:40:18.767066 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:18.767066 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:40:18.768034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:18.768034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:40:18.768034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:40:18.768034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:40:18.768034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:18.768034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:40:18.768034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:18.768034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:40:18.768034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:18.768034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:40:18.768034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:18.768034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:40:18.768034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:18.768034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:40:18.768034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:40:18.768387 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:18.768387 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:40:18.768387 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:18.768387 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:40:18.768387 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:40:18.768387 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:18.768387 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:40:18.768387 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:18.768387 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:40:18.768859 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:18.768859 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:40:18.768859 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:40:18.768859 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:40:18.768859 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:40:18.768859 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:40:18.768859 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:40:18.768859 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:40:18.768859 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:40:18.768859 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:40:18.768859 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:18.768859 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:40:18.769121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:18.769121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:40:18.769121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:18.769121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:40:18.769121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:18.769121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:40:18.769121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:18.769121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:40:18.769121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:40:18.769121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:18.769121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:40:18.769121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:18.769121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:40:18.769121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:40:18.769384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:18.769384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:40:18.769384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:18.769384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:40:18.769892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:18.769892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:40:18.769892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:40:18.769892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:40:18.769892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:40:18.769892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:40:18.769892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:40:18.769892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:40:18.769892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:40:18.769892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:40:18.769892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:40:18.769892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:40:18.769892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:40:18.769892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:40:18.769892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:40:18.769892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:40:18.770254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:40:18.770254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:40:18.770254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:40:18.770254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:40:18.770254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:40:18.770254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:18.770254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:40:18.770254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:18.770254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:40:18.770254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:18.770254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:40:18.770254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:18.770254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:40:18.770254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:18.770254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:40:18.770254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:40:18.770254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:18.770254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:40:18.770593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:18.770593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:40:18.770593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:40:18.770593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:18.770593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:40:18.770593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:18.770593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:40:18.772462 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:40:18.772773 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:40:18.772905 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:40:18.773033 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:40:18.773937 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:40:18.773937 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:18.774265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:40:18.774265 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:18.774265 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:40:18.774265 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:40:18.774265 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:18.774265 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:40:18.774265 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:18.774265 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:40:18.775487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:18.775487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:40:18.775487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:40:18.775487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:40:18.775487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:18.775487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:40:18.775487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:18.775487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:40:18.775487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:18.775487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:40:18.775487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:18.775487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:40:18.775487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:18.775487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:40:18.775487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:40:18.776325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:18.776325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:40:18.776325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:18.776325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:40:18.776325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:40:18.776325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:18.776325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:40:18.776325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:18.776325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:40:18.776325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:18.776325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:40:18.776325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:40:18.776325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:40:18.776325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:40:18.777099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:40:18.777099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:40:18.777099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:40:18.777099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:40:18.777099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:18.777099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:40:18.777099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:18.777099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:40:18.777099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:18.777099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:40:18.777099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:18.777099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:40:18.777099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:18.778137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:40:18.778137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:40:18.778137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:18.778137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:40:18.778137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:18.778137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:40:18.778137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:40:18.778137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:18.778137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:40:18.778137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:18.778137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:40:18.778137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:18.778708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:40:18.778708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:40:18.778708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:40:18.778708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:40:18.778708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:40:18.778708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:40:18.778708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:40:18.778708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:40:18.778708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:40:18.778708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:40:18.778708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:40:18.778708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:40:18.778708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:40:18.778708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:40:18.778708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:40:18.778708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:40:18.778708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:40:18.778708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:40:18.778708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:40:18.779671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:18.779671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:40:18.779671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:18.779671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:40:18.779671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:18.779671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:40:18.779671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:18.779671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:40:18.779671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:18.779671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:40:18.779671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:40:18.779671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:18.779671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:40:18.779671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:18.779671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:40:18.779671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:40:18.779671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:18.779671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:40:18.779671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:18.779671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:40:18.780657 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:40:18.780657 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:40:18.780657 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:40:18.780657 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:40:18.780657 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:40:18.780657 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:40:18.781110 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:18.781110 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:40:18.781110 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:18.781110 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:40:18.781110 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:40:18.781110 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:18.781110 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:40:18.781110 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:18.781110 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:40:18.782075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:18.782075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:40:18.782075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:40:18.782075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:40:18.782075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:40:18.782075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:40:18.782075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:40:18.782075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:18.782075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:40:18.782075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:18.782075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:40:18.782075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:18.782709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:40:18.782709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:18.782709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:40:18.782709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:18.782709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:40:18.782709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:40:18.782709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:18.782709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:40:18.782709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:18.782709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:40:18.782709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:40:18.782709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:18.782709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:40:18.782970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:18.782970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:40:18.783081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:18.783081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:40:18.783081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:40:18.783081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:40:18.783081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:40:18.783081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:40:18.783081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:40:18.783081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:40:18.783081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:40:18.783081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:40:18.783081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:40:18.783081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:40:18.783081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:40:18.783081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:40:18.783081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:40:18.783081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:40:18.783405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:40:18.783405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:40:18.783405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:18.783405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:40:18.783405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:18.783405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:40:18.783405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:18.783405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:40:18.783405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:18.783405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:40:18.783405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:18.783405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:40:18.783405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:40:18.783405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:18.783405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:40:18.783405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:18.783405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:40:18.783405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:40:18.783405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:18.783826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:40:18.783826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:18.783826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:40:23.606738 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:40:23.607518 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:40:23.607923 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:40:23.609399 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:40:23.652703 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:40:23.653040 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:23.653895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:40:23.653895 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:23.653895 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:40:23.653895 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:40:23.653895 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:23.653895 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:40:23.653895 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:23.653895 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:40:23.655308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.046 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:23.655308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:40:23.655308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:40:23.655308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:40:23.655308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:40:23.655308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:40:23.655308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:40:23.655308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:23.655308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:40:23.655308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:23.655308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:40:23.655308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:23.655605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:40:23.655605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:23.655605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:40:23.655605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:23.655605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:40:23.655605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:40:23.655605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:23.655605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:40:23.655605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:23.655605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:40:23.655605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:40:23.655605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:23.655605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:40:23.655954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:23.655954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:40:23.656267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:23.656267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:40:23.656267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:40:23.656267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:40:23.656267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:40:23.656267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:40:23.656267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:40:23.656267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:40:23.656267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:40:23.656267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:40:23.656267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:40:23.656267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:40:23.656267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:40:23.656267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:40:23.656267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:40:23.656267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:40:23.656744 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:40:23.656744 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:40:23.656744 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:23.656744 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:40:23.656744 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:23.656744 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:40:23.656744 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:23.656744 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:40:23.656744 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:23.656744 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:40:23.656744 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:23.656744 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:40:23.656744 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:40:23.656744 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:23.656744 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:40:23.656744 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:23.656744 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:40:23.656744 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:40:23.656744 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:23.657285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:40:23.657285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:23.657285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:40:23.758562 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:40:23.759440 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:40:23.759576 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:40:23.760735 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:40:23.761122 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:40:23.761374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:23.762396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:40:23.762396 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:23.762396 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:40:23.762396 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:40:23.762396 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:23.762396 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:40:23.762396 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:23.762396 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:40:23.764142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:23.764142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:40:23.764142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:40:23.764142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:40:23.764142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:40:23.764142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:40:23.764142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:40:23.764142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:23.764142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:40:23.764142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:23.764142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:40:23.764142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:23.764993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:40:23.764993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:23.764993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:40:23.764993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:23.764993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:40:23.764993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:40:23.764993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:23.764993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:40:23.764993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:23.764993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:40:23.764993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:40:23.764993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:23.764993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:40:23.765698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:23.765698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:40:23.766091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:23.766091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:40:23.766091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:40:23.766091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:40:23.766091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:40:23.766091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:40:23.766091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:40:23.766091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:40:23.766091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:40:23.766091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:40:23.766091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:40:23.766091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:40:23.766091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:40:23.766091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:40:23.766091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:40:23.766091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:40:23.767022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:40:23.767022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:40:23.767022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:23.767022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:40:23.767022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:23.767022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:40:23.767022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:23.767022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:40:23.767022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:23.767022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:40:23.767022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:23.767022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:40:23.767022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:40:23.767022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:23.767022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:40:23.767022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:23.767022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:40:23.767022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:40:23.767022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:23.768851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:40:23.768851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:23.768851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:40:23.768851 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:40:23.768851 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:40:23.768851 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:40:23.768851 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:40:23.768851 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:40:23.769255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:23.769447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:40:23.769447 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:23.769447 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:40:23.769447 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:40:23.769447 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:23.769447 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:40:23.769447 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:23.769447 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:40:23.770232 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:23.770232 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:40:23.770232 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:40:23.770232 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:40:23.770232 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:23.770232 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:40:23.770232 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:23.770232 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:40:23.770232 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:23.770232 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:40:23.770232 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:23.770232 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:40:23.770232 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:23.770232 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:40:23.770232 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:40:23.770838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:23.770838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:40:23.770838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:23.770838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:40:23.770838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:40:23.770838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:23.770838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:40:23.770838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:23.770838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:40:23.771218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:23.771218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:40:23.771218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:40:23.771218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:40:23.771218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:40:23.771218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:40:23.771218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:40:23.771218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:40:23.771218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:40:23.771218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:40:23.771218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:23.771218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:40:23.771671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:23.771671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:40:23.771671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:23.771671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:40:23.771671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:23.771671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:40:23.771671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:23.771671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:40:23.771671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:40:23.771671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:23.771671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:40:23.771671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:23.771671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:40:23.771671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:40:23.772124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:23.772124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:40:23.772124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:23.772124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:40:23.772124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:23.772124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:40:23.772124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:40:23.772124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:40:23.772124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:40:23.772124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:40:23.772124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:40:23.772124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:40:23.772124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:40:23.772124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:40:23.772556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:40:23.772556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:40:23.772556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:40:23.772556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:40:23.772556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:40:23.772556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:40:23.772556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:40:23.772556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:40:23.772556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:40:23.772556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:40:23.772556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:40:23.772556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:23.772556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:40:23.772556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:23.772556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:40:23.772556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:23.772556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:40:23.772556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:23.772556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:40:23.773171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:23.773171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:40:23.773171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:40:23.773171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:23.773171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:40:23.773171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:23.773171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:40:23.773171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:40:23.773171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:23.773171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:40:23.773171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:23.773171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:40:23.773171 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:40:23.773716 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:40:23.773716 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:40:23.773716 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:40:23.774112 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:40:23.774489 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:23.775221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:40:23.775221 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:23.775221 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:40:23.775221 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:40:23.775221 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:23.775221 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:40:23.775221 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:23.775221 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:40:23.776153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:23.776153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:40:23.776153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:40:23.776153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:40:23.776153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:23.776153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:40:23.776153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:23.776153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:40:23.776153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:23.776153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:40:23.776153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:23.776153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:40:23.776153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:23.776153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:40:23.776153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:40:23.776604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:23.776604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:40:23.776604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:23.776604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:40:23.776604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:40:23.776604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:23.776604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:40:23.776604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:23.776604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:40:23.777103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:23.777103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:40:23.777103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:40:23.777103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:40:23.777103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:40:23.777103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:40:23.777103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:40:23.777103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:40:23.777103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:40:23.777103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:23.777103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:40:23.777447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:23.777447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:40:23.777447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:23.777447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:40:23.777447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:23.777447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:40:23.777447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:23.777447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:40:23.777447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:40:23.777447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:23.777447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:40:23.777447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:23.777447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:40:23.777447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:40:23.777867 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:23.777867 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:40:23.777867 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:23.777867 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:40:23.778208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:23.778208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:40:23.778208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:40:23.778208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:40:23.778208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:40:23.778208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:40:23.778208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:40:23.778208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:40:23.778208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:40:23.778208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:40:23.778208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:40:23.778208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:40:23.778208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:40:23.778208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:40:23.778208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:40:23.778208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:40:23.778208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:40:23.778677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:40:23.778677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:40:23.778677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:40:23.778677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:23.778677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:40:23.778677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:23.778677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:40:23.778677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:23.778677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:40:23.778677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:23.778677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:40:23.778677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:23.778677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:40:23.778677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:40:23.778677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:23.778677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:40:23.778677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:23.778677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:40:23.778677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:40:23.779249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:23.779249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:40:23.779249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:23.779249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:40:23.779439 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:40:23.780074 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:40:23.780183 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:40:23.780346 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:40:23.780813 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:40:23.781146 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:40:23.781397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:23.782107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:40:23.782107 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:23.782107 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:40:23.782107 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:40:23.782107 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:23.782107 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:40:23.782107 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:23.782107 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:40:23.782951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:23.782951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:40:23.782951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:40:23.782951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:40:23.782951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:40:23.782951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:40:23.782951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:40:23.782951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:23.782951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:40:23.782951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:23.782951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:40:23.782951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:23.783334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:40:23.783334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:23.783334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:40:23.783334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:23.783334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:40:23.783334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:40:23.783334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:23.783334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:40:23.783334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:23.783334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:40:23.783334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:40:23.783334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:23.783334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:40:23.783722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:23.783722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:40:23.783980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:23.783980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:40:23.783980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:40:23.783980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:40:23.783980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:40:23.783980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:40:23.783980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:40:23.783980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:40:23.783980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:40:23.783980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:40:23.783980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:40:23.783980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:40:23.783980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:40:23.783980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:40:23.783980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:40:23.783980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:40:23.784413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:40:23.784413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:40:23.784413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:23.784413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:40:23.784413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:23.784413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:40:23.784413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:23.784413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:40:23.784413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:23.784413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:40:23.784413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:23.784413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:40:23.784413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:40:23.784413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:23.784413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:40:23.784413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:23.784413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:40:23.784413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:40:23.784413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:23.785030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:40:23.785030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:23.785030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:40:28.607460 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:40:28.608297 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:40:28.608388 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:40:28.609693 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:40:28.643706 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:40:28.644150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:28.645087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:40:28.645087 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:28.645087 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:40:28.645087 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:40:28.645087 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:28.645087 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:40:28.645087 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:28.645087 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:40:28.645860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:28.645860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:40:28.645860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:40:28.645860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:40:28.645860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:40:28.645860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:40:28.645860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:40:28.645860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:28.645860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:40:28.645860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:28.645860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:40:28.645860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:28.646260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:40:28.646260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:28.646260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:40:28.646260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:28.646260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:40:28.646260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:40:28.646260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:28.646260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:40:28.646260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:28.646260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:40:28.646260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:40:28.646260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:28.646260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:40:28.646605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:28.646605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:40:28.646605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:28.646605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:40:28.646605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:40:28.646605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:40:28.646605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:40:28.646605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:40:28.646605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:40:28.646605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:40:28.646605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:40:28.646605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:40:28.646605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:40:28.646605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:40:28.646605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:40:28.647050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:40:28.647050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:40:28.647050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:40:28.647050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:40:28.647050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:40:28.647050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:28.647050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:40:28.647050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:28.647050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:40:28.647050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:28.647050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:40:28.647050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:28.647050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:40:28.647050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:28.647050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:40:28.647050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:40:28.647050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:28.647050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:40:28.647050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:28.647564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:40:28.647564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:40:28.647564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:28.647564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:40:28.647564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:28.647564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:40:28.759578 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:40:28.760245 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:40:28.760409 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:40:28.761783 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:40:28.762216 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:40:28.762515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:28.763539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:40:28.763539 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:28.763539 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:40:28.763539 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:40:28.763539 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:28.763539 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:40:28.763539 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:28.763539 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:40:28.765050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:28.765050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:40:28.765050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:40:28.765050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:40:28.765050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:40:28.765050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:40:28.765050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:40:28.765050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:28.765050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:40:28.765050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:28.765050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:40:28.765050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:28.765533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:40:28.765533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:28.765533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:40:28.765533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:28.765533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:40:28.765533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:40:28.765533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:28.765533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:40:28.765533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:28.765533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:40:28.765533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:40:28.765533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:28.765533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:40:28.765909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:28.765909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:40:28.766690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:28.766690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:40:28.766690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:40:28.766690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:40:28.766690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:40:28.766690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:40:28.766690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:40:28.766690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:40:28.766690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:40:28.766690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:40:28.766690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:40:28.766690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:40:28.766690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:40:28.766690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:40:28.766690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:40:28.766690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:40:28.767110 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:40:28.767110 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:40:28.767110 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:28.767110 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:40:28.767110 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:28.767110 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:40:28.767110 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:28.767110 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:40:28.767110 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:28.767110 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:40:28.767110 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:28.767110 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:40:28.767110 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:40:28.767110 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:28.767110 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:40:28.767110 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:28.767110 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:40:28.767110 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:40:28.767110 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:28.767483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:40:28.767483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:28.767483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:40:28.767483 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:40:28.767571 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:40:28.767955 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:40:28.768093 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:40:28.768621 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:40:28.768843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:28.769536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:40:28.769536 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:28.769536 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:40:28.769536 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:40:28.769536 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:28.769536 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:40:28.769536 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:28.769536 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:40:28.770237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:28.770237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:40:28.770237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:40:28.770237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:40:28.770237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:28.770237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:40:28.770237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:28.770237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:40:28.770237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:28.770237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:40:28.770237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:28.770237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:40:28.770237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:28.770237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:40:28.770237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:40:28.770601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:28.770601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:40:28.770601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:28.770601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:40:28.770601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:40:28.770601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:28.770601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:40:28.770601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:28.770601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:40:28.770891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:28.770891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:40:28.770891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:40:28.770891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:40:28.770891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:40:28.770891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:40:28.770891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:40:28.770891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:40:28.770891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:40:28.770891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:40:28.770891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:28.770891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:40:28.771333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:28.771333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:40:28.771333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:28.771333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:40:28.771333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:28.771333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:40:28.771333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:28.771333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:40:28.771333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:40:28.771333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:28.771333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:40:28.771333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:28.771333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:40:28.771333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:40:28.771854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:28.771854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:40:28.771854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:28.771854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:40:28.771854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:28.771854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:40:28.771854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:40:28.771854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:40:28.771854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:40:28.771854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:40:28.771854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:40:28.771854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:40:28.771854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:40:28.771854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:40:28.772363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:40:28.772363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:40:28.772363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:40:28.772363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:40:28.772363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:40:28.772363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:40:28.772363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:40:28.772363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:40:28.772363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:40:28.772363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:40:28.772363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:40:28.772363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:28.772363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:40:28.772363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:28.772363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:40:28.772363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:28.772363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:40:28.772363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:28.772363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:40:28.772786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:28.772786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:40:28.772786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:40:28.772786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:28.772786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:40:28.772786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:28.772786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:40:28.772786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:40:28.772786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:28.772786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:40:28.772786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:28.772786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:40:28.773208 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:40:28.773420 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:40:28.773510 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:40:28.773621 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:40:28.773947 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:40:28.774241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:28.774858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:40:28.774858 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:28.774858 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:40:28.774858 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:40:28.774858 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:28.774858 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:40:28.774858 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:28.774858 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:40:28.775497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:28.775497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:40:28.775497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:40:28.775497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:40:28.775497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:28.775497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:40:28.775497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:28.775497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:40:28.775497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:28.775497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:40:28.775497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:28.775497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:40:28.775497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:28.775497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:40:28.775497 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:40:28.775982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:28.775982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:40:28.775982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:28.775982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:40:28.775982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:40:28.775982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:28.775982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:40:28.775982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:28.775982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:40:28.776353 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:28.776353 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:40:28.776353 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:40:28.776353 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:40:28.776353 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:40:28.776353 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:40:28.776353 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:40:28.776353 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:40:28.776353 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:40:28.776353 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:28.776353 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:40:28.776713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:28.776713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:40:28.776713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:28.776713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:40:28.776713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:28.776713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:40:28.776713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:28.776713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:40:28.776713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:40:28.776713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:28.776713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:40:28.776713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:28.776713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:40:28.776713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:40:28.777150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:28.777150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:40:28.777150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:28.777150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:40:28.777433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:28.777433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:40:28.777433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:40:28.777433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:40:28.777433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:40:28.777433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:40:28.777433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:40:28.777433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:40:28.777433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:40:28.777433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:40:28.777433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:40:28.777433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:40:28.777433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:40:28.777433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:40:28.777433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:40:28.777433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:40:28.777433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:40:28.777988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:40:28.777988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:40:28.777988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:40:28.777988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:28.777988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:40:28.777988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:28.777988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:40:28.777988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:28.777988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:40:28.777988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:28.777988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:40:28.777988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:28.777988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:40:28.777988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:40:28.777988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:28.777988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:40:28.777988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:28.777988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:40:28.777988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:40:28.778610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:28.778610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:40:28.778610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:28.778610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:40:28.780103 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:40:28.780332 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:40:28.780446 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:40:28.780605 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:40:28.781034 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:40:28.781311 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:40:28.781510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:28.782124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:40:28.782124 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:28.782124 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:40:28.782124 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:40:28.782124 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:28.782124 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:40:28.782124 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:28.782124 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:40:28.782971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:28.782971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:40:28.782971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:40:28.782971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:40:28.782971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:40:28.782971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:40:28.782971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:40:28.782971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:28.782971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:40:28.782971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:28.782971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:40:28.782971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:28.783352 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:40:28.783352 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:28.783352 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:40:28.783352 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:28.783352 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:40:28.783352 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:40:28.783352 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:28.783352 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:40:28.783352 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:28.783352 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:40:28.783352 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:40:28.783352 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:28.783352 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:40:28.783723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:28.783723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:40:28.783978 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:28.783978 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:40:28.783978 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:40:28.783978 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:40:28.783978 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:40:28.783978 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:40:28.783978 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:40:28.783978 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:40:28.783978 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:40:28.783978 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:40:28.783978 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:40:28.783978 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:40:28.783978 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:40:28.783978 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:40:28.783978 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:40:28.783978 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:40:28.784584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:40:28.784584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:40:28.784584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:28.784584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:40:28.784584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:28.784584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:40:28.784584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:28.784584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:40:28.784584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:28.784584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:40:28.784584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:28.784584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:40:28.784584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:40:28.784584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:28.784584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:40:28.784584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:28.784584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:40:28.784584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:40:28.784584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:28.785371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:40:28.785371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:28.785371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:40:33.608106 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:40:33.608675 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:40:33.608824 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:40:33.609635 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:40:33.648493 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:40:33.648735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:33.649636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:40:33.649636 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:33.649636 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:40:33.649636 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:40:33.649636 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:33.649636 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:40:33.649636 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:33.649636 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:40:33.650658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:33.650658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:40:33.650658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:40:33.650658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:40:33.650658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:40:33.650658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:40:33.650658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:40:33.650658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:33.650658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:40:33.650658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:33.650658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:40:33.650658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:33.651125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:40:33.651125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:33.651125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:40:33.651125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:33.651125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:40:33.651125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:40:33.651125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:33.651125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:40:33.651125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:33.651125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:40:33.651125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:40:33.651125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:33.651125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:40:33.651583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:33.651583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:40:33.651786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:33.651786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:40:33.651786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:40:33.651786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:40:33.651786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:40:33.651786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:40:33.651786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:40:33.651786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:40:33.651786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:40:33.651786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:40:33.651786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:40:33.651786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:40:33.651786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:40:33.651786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:40:33.651786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:40:33.651786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:40:33.652203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:40:33.652203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:40:33.652203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:33.652203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:40:33.652203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:33.652203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:40:33.652203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:33.652203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:40:33.652203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:33.652203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:40:33.652203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:33.652203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:40:33.652203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:40:33.652203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:33.652203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:40:33.652203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:33.652203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:40:33.652203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:40:33.652203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:33.652608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:40:33.652608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:33.652608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:40:33.760423 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:40:33.761293 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:40:33.761553 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:40:33.763070 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:40:33.763348 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:40:33.763512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:33.764375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:40:33.764375 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:33.764375 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:40:33.764375 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:40:33.764375 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:33.764375 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:40:33.764375 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:33.764375 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:40:33.766272 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:33.766272 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:40:33.766272 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:40:33.766272 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:40:33.766272 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:40:33.766272 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:40:33.766272 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:40:33.766272 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:33.766272 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:40:33.766272 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:33.766272 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:40:33.766272 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:33.766738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:40:33.766738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:33.766738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:40:33.766738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:33.766738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:40:33.766738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:40:33.766738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:33.766738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:40:33.766738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:33.766738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:40:33.766738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:40:33.766738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:33.766738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:40:33.767101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:33.767101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:40:33.767689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:33.767689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:40:33.767689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:40:33.767689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:40:33.767689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:40:33.767689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:40:33.767689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:40:33.767689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:40:33.767689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:40:33.767689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:40:33.767689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:40:33.767689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:40:33.767689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:40:33.767689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:40:33.767689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:40:33.767689 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:40:33.768289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:40:33.768289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:40:33.768289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:33.768289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:40:33.768289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:33.768289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:40:33.768289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:33.768289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:40:33.768289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:33.768289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:40:33.768289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:33.768289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:40:33.768289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:40:33.768289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:33.768289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:40:33.768289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:33.768289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:40:33.768289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:40:33.768289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:33.768911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:40:33.768911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:33.768911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:40:33.768911 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:40:33.768911 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:40:33.768911 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:40:33.768911 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:40:33.769708 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:40:33.769964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:33.770793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:40:33.770793 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:33.770793 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:40:33.770793 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:40:33.770793 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:33.770793 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:40:33.770793 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:33.770793 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:40:33.772424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:33.772424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:40:33.772424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:40:33.772424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:40:33.772424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:33.772424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:40:33.772424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:33.772424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:40:33.772424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:33.772424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:40:33.772424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:33.772424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:40:33.772424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:33.772424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:40:33.772424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:40:33.772971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:33.772971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:40:33.772971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:33.772971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:40:33.772971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:40:33.772971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:33.772971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:40:33.772971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:33.772971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:40:33.774325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:33.774325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:40:33.774325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:40:33.774325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:40:33.774325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:40:33.774325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:40:33.774325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:40:33.774325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:40:33.774325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:40:33.774325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:40:33.774325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:33.774325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:40:33.774869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:33.774869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:40:33.774869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:33.774869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:40:33.774869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:33.774869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:40:33.774869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:33.774869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:40:33.774869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:40:33.774869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:33.774869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:40:33.774869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:33.774869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:40:33.774869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:40:33.775381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:33.775381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:40:33.775381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:33.775381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:40:33.776349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:33.776349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:40:33.776349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:40:33.776349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:40:33.776349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:40:33.776349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:40:33.776349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:40:33.776349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:40:33.776349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:40:33.776349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:40:33.776349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:40:33.776349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:40:33.776349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:40:33.776349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:40:33.776349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:40:33.776349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:40:33.777017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:40:33.777017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:40:33.777017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:40:33.777017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:40:33.777017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:40:33.777017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:33.777017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:40:33.777017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:33.777017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:40:33.777017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:33.777017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:40:33.777017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:33.777017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:40:33.777017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:33.777017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:40:33.777017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:40:33.777017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:33.777017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:40:33.777665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:33.777665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:40:33.777665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:40:33.777665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:33.777665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:40:33.777665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:33.777665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:40:33.777665 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:40:33.777665 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:40:33.777665 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:40:33.777993 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:40:33.778296 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:40:33.778649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:33.779519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:40:33.779519 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:33.779519 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:40:33.779519 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:40:33.779519 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:33.779519 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:40:33.779519 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:33.779519 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:40:33.780140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:33.780140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:40:33.780140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:40:33.780140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:40:33.780140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:33.780140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:40:33.780140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:33.780140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:40:33.780140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:33.780140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:40:33.780140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:33.780140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:40:33.780140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:33.780140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:40:33.780140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:40:33.780582 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:33.780582 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:40:33.780582 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:33.780582 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:40:33.780582 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:40:33.780582 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:33.780582 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:40:33.780582 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:33.780582 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:40:33.781112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:33.781112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:40:33.781112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:40:33.781112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:40:33.781112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:40:33.781112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:40:33.781112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:40:33.781112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:40:33.781112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:40:33.781112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:33.781112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:40:33.781503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:33.781503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:40:33.781503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:33.781503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:40:33.781503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:33.781503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:40:33.781503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:33.781503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:40:33.781503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:40:33.781503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:33.781503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:40:33.781503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:33.781503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:40:33.781503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:40:33.781865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:33.781865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:40:33.781865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:33.781865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:40:33.781865 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:40:33.782433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:33.782433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:40:33.782433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:40:33.782433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:40:33.782433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:40:33.782433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:40:33.782433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:40:33.782433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:40:33.782433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:40:33.782433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:40:33.782433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:40:33.782433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:40:33.782433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:40:33.782433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:40:33.782433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:40:33.782433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:40:33.782433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:40:33.782929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:40:33.782929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:40:33.782929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:40:33.782929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:33.782929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:40:33.782929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:33.782929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:40:33.782929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:33.782929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:40:33.782929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:33.782929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:40:33.782929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:33.782929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:40:33.782929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:40:33.782929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:33.782929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:40:33.782929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:33.782929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:40:33.782929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:40:33.783406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:33.783406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:40:33.783406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:33.783406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:40:33.783406 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:40:33.783406 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:40:33.783406 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:40:33.783406 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:40:33.783757 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:40:33.784019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:33.784670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:40:33.784670 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:33.784670 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:40:33.784670 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:40:33.784670 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:33.784670 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:40:33.784670 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:33.784670 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:40:33.785332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:33.785332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:40:33.785332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:40:33.785332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:40:33.785332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:40:33.785332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:40:33.785332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:40:33.785332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:33.785332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:40:33.785332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:33.785332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:40:33.785332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:33.785732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:40:33.785732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:33.785732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:40:33.785732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:33.785732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:40:33.785732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:40:33.785732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:33.785732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:40:33.785732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:33.785732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:40:33.785732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:40:33.785732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:33.785732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:40:33.786077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:33.786077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:40:33.786146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:33.786146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:40:33.786146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:40:33.786146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:40:33.786146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:40:33.786146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:40:33.786146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:40:33.786146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:40:33.786146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:40:33.786146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:40:33.786146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:40:33.786146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:40:33.786146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:40:33.786146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:40:33.786146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:40:33.786146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:40:33.786598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:40:33.786598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:40:33.786598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:33.786598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:40:33.786598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:33.786598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:40:33.786598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:33.786598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:40:33.786598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:33.786598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:40:33.786598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:33.786598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:40:33.786598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:40:33.786598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:33.786598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:40:33.786598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:33.786598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:40:33.786598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:40:33.786598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:33.787152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:40:33.787152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:33.787152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:40:38.608855 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:40:38.609624 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:40:38.609789 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:40:38.611194 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:40:38.652597 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:40:38.653054 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:38.654042 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:40:38.654042 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:38.654042 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:40:38.654042 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:40:38.654042 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:38.654042 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:40:38.654042 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:38.654042 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:40:38.654998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.044 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:38.654998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:40:38.654998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:40:38.654998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:40:38.654998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:40:38.654998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:40:38.654998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:40:38.654998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:38.654998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:40:38.654998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:38.654998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:40:38.654998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:38.655446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:40:38.655446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:38.655446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:40:38.655446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:38.655446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:40:38.655446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:40:38.655446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:38.655446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:40:38.655446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:38.655446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:40:38.655446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:40:38.655446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:38.655446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:40:38.655931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:38.655931 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:40:38.656098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:38.656098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:40:38.656098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:40:38.656098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:40:38.656098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:40:38.656098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:40:38.656098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:40:38.656098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:40:38.656098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:40:38.656098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:40:38.656098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:40:38.656098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:40:38.656098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:40:38.656098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:40:38.656098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:40:38.656098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:40:38.656532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:40:38.656532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:40:38.656532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:38.656532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:40:38.656532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:38.656532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:40:38.656532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:38.656532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:40:38.656532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:38.656532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:40:38.656532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:38.656532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:40:38.656532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:40:38.656532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:38.656532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:40:38.656532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:38.656532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:40:38.656532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:40:38.656532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:38.657009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:40:38.657009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:38.657009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:40:38.761248 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:40:38.761919 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:40:38.762129 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:40:38.763201 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:40:38.763447 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:40:38.763622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:38.765256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:40:38.765256 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:38.765256 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:40:38.765256 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:40:38.765256 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:38.765256 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:40:38.765256 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:38.765256 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:40:38.766954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:38.766954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:40:38.766954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:40:38.766954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:40:38.766954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:40:38.766954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:40:38.766954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:40:38.766954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:38.766954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:40:38.766954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:38.766954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:40:38.766954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:38.767360 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:40:38.767360 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:38.767360 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:40:38.767360 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:38.767360 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:40:38.767360 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:40:38.767360 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:38.767360 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:40:38.767360 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:38.767360 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:40:38.767360 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:40:38.767360 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:38.767360 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:40:38.767786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:38.767786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:40:38.768730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:38.768730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:40:38.768730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:40:38.768730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:40:38.768730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:40:38.768730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:40:38.768730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:40:38.768730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:40:38.768730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:40:38.768730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:40:38.768730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:40:38.768730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:40:38.768730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:40:38.768730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:40:38.768730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:40:38.768730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:40:38.769249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:40:38.769249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:40:38.769249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:38.769249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:40:38.769249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:38.769249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:40:38.769249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:38.769249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:40:38.769249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:38.769249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:40:38.769249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:38.769249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:40:38.769249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:40:38.769249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:38.769249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:40:38.769249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:38.769249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:40:38.769249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:40:38.769249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:38.769868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:40:38.769868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:38.769868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:40:38.769868 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:40:38.769868 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:40:38.769868 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:40:38.769868 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:40:38.770208 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:40:38.770673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:38.771576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:40:38.771576 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:38.771576 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:40:38.771576 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:40:38.771576 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:38.771576 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:40:38.771576 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:38.771576 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:40:38.772418 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:38.772418 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:40:38.772418 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:40:38.772418 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:40:38.772418 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:38.772418 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:40:38.772418 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:38.772418 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:40:38.772418 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:38.772418 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:40:38.772418 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:38.772418 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:40:38.772418 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:38.772418 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:40:38.772418 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:40:38.774251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:38.774251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:40:38.774251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:38.774251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:40:38.774251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:40:38.774251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:38.774251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:40:38.774251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:38.774251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:40:38.774251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:38.774251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:40:38.774251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:40:38.774251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:40:38.774251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:40:38.775402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:40:38.775402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:40:38.775402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:40:38.775402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:40:38.775402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:40:38.775402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:38.775402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:40:38.775402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:38.775402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:40:38.775402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:38.775402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:40:38.775402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:38.775402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:40:38.776325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:38.776325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:40:38.776325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:40:38.776325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:38.776325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:40:38.776325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:38.776325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:40:38.776325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:40:38.776325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:38.776325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:40:38.776325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:38.776325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:40:38.777100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:38.777100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:40:38.777100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:40:38.777100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:40:38.777100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:40:38.777100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:40:38.777100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:40:38.777100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:40:38.777100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:40:38.777100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:40:38.777100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:40:38.777100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:40:38.777100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:40:38.777100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:40:38.777100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:40:38.777100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:40:38.778109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:40:38.778109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:40:38.778109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:40:38.778109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:40:38.778109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:40:38.778109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:38.778109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:40:38.778109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:38.778109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:40:38.778109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:38.778109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:40:38.778109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:38.778109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:40:38.778109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:38.778109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:40:38.778109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:40:38.778109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:38.778109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:40:38.779420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:38.779420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:40:38.779420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:40:38.779420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:38.779420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:40:38.779420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:38.779420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:40:38.779420 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:40:38.779420 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:40:38.779420 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:40:38.779718 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:40:38.779718 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:40:38.779718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:38.779718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:40:38.779718 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:38.779718 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:40:38.779718 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:40:38.779718 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:38.779718 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:40:38.779718 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:38.779718 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:40:38.780400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:38.780400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:40:38.780400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:40:38.780400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:40:38.780400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:38.780400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:40:38.780400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:38.780400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:40:38.780400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:38.780400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:40:38.780400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:38.780400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:40:38.780400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:38.780400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:40:38.780400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:40:38.780676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:38.780676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:40:38.780676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:38.780676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:40:38.780676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:40:38.780676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:38.780676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:40:38.780676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:38.780676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:40:38.781287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:38.781287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:40:38.781287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:40:38.781287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:40:38.781287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:40:38.781287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:40:38.781287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:40:38.781287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:40:38.781287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:40:38.781287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:38.781287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:40:38.781552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:38.781552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:40:38.781552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:38.781552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:40:38.781552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:38.781552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:40:38.781552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:38.781552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:40:38.781552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:40:38.781552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:38.781552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:40:38.781552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:38.781552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:40:38.781552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:40:38.781823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:38.781823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:40:38.781823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:38.781823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:40:38.782372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:38.782372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:40:38.782372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:40:38.782372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:40:38.782372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:40:38.782372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:40:38.782372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:40:38.782372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:40:38.782372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:40:38.782372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:40:38.782372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:40:38.782372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:40:38.782372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:40:38.782372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:40:38.782372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:40:38.782372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:40:38.782372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:40:38.782723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:40:38.782723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:40:38.782723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:40:38.782723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:38.782723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:40:38.782723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:38.782723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:40:38.782723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:38.782723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:40:38.782723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:38.782723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:40:38.782723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:38.782723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:40:38.782723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:40:38.782723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:38.782723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:40:38.782723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:38.782723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:40:38.782723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:40:38.783143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:38.783143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:40:38.783143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:38.783143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:40:38.783143 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:40:38.783143 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:40:38.783143 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:40:38.783288 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:40:38.783715 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:40:38.783989 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:40:38.784146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:38.785045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:40:38.785045 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:38.785045 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:40:38.785045 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:40:38.785045 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:38.785045 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:40:38.785045 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:38.785045 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:40:38.785874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:38.785874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:40:38.785874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:40:38.785874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:40:38.785874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:40:38.785874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:40:38.785874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:40:38.785874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:38.785874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:40:38.785874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:38.785874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:40:38.785874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:38.786112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:40:38.786112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:38.786112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:40:38.786112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:38.786112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:40:38.786112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:40:38.786112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:38.786112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:40:38.786112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:38.786112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:40:38.786112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:40:38.786112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:38.786112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:40:38.786341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:38.786341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:40:38.786908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:38.786908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:40:38.786908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:40:38.786908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:40:38.786908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:40:38.786908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:40:38.786908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:40:38.786908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:40:38.786908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:40:38.786908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:40:38.786908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:40:38.786908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:40:38.786908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:40:38.786908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:40:38.786908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:40:38.786908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:40:38.787208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:40:38.787208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:40:38.787208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:38.787208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:40:38.787208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:38.787208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:40:38.787208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:38.787208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:40:38.787208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:38.787208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:40:38.787208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:38.787208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:40:38.787208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:40:38.787208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:38.787208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:40:38.787208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:38.787208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:40:38.787208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:40:38.787208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:38.787645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:40:38.787645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:38.787645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:40:43.609693 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:40:43.610721 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:40:43.610721 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:40:43.612254 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:40:43.652480 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:40:43.652772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:43.653595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:40:43.653595 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:43.653595 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:40:43.653595 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:40:43.653595 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:43.653595 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:40:43.653595 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:43.653595 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:40:43.654489 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.043 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:43.654489 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:40:43.654489 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:40:43.654489 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:40:43.654489 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:40:43.654489 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:40:43.654489 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:40:43.654489 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:43.654489 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:40:43.654489 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:43.654489 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:40:43.654489 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:43.656183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:40:43.656183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:43.656183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:40:43.656183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:43.656183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:40:43.656183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:40:43.656183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:43.656183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:40:43.656183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:43.656183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:40:43.656183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:40:43.656183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:43.656183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:40:43.656590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:43.656590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:40:43.656590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:43.656590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:40:43.656590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:40:43.656590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:40:43.656590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:40:43.656590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:40:43.656590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:40:43.656590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:40:43.656590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:40:43.656590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:40:43.656590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:40:43.656590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:40:43.656590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:40:43.657151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:40:43.657151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:40:43.657151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:40:43.657151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:40:43.657151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:40:43.657151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:43.657151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:40:43.657151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:43.657151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:40:43.657151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:43.657151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:40:43.657151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:43.657151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:40:43.657151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:43.657151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:40:43.657151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:40:43.657151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:43.657151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:40:43.657151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:43.657701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:40:43.657701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:40:43.657701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:43.657701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:40:43.657701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:43.657701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:40:43.761977 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:40:43.762659 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:40:43.763295 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:40:43.764214 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:40:43.765162 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:40:43.765162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:43.765517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:40:43.765517 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:43.765517 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:40:43.765517 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:40:43.765517 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:43.765517 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:40:43.765517 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:43.765517 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:40:43.767109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:43.767109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:40:43.767109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:40:43.767109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:40:43.767109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:40:43.767109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:40:43.767109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:40:43.767109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:43.767109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:40:43.767109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:43.767109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:40:43.767109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:43.767379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:40:43.767379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:43.767379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:40:43.767379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:43.767379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:40:43.767379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:40:43.767379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:43.767379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:40:43.767379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:43.767379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:40:43.767379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:40:43.767379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:43.767379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:40:43.767631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:43.767631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:40:43.769410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:43.769410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:40:43.769410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:40:43.769410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:40:43.769410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:40:43.769410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:40:43.769410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:40:43.769410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:40:43.769410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:40:43.769410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:40:43.769410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:40:43.769410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:40:43.769410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:40:43.769410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:40:43.769410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:40:43.769410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:40:43.769788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:40:43.769788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:40:43.769788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:43.769788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:40:43.769788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:43.769788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:40:43.769788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:43.769788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:40:43.769788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:43.769788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:40:43.769788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:43.769788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:40:43.769788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:40:43.769788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:43.769788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:40:43.769788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:43.769788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:40:43.769788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:40:43.769788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:43.770169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:40:43.770169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:43.770169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:40:43.770169 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:40:43.770169 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:40:43.770393 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:40:43.770534 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:40:43.770858 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:40:43.771447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:43.771893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:40:43.771893 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:43.771893 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:40:43.771893 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:40:43.771893 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:43.771893 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:40:43.771893 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:43.771893 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:40:43.772986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:43.772986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:40:43.772986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:40:43.772986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:40:43.772986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:43.772986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:40:43.772986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:43.772986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:40:43.772986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:43.772986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:40:43.772986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:43.772986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:40:43.772986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:43.772986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:40:43.772986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:40:43.773412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:43.773412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:40:43.773412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:43.773412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:40:43.773412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:40:43.773412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:43.773412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:40:43.773412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:43.773412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:40:43.773985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:43.773985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:40:43.773985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:40:43.773985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:40:43.773985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:40:43.773985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:40:43.773985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:40:43.773985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:40:43.773985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:40:43.773985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:40:43.773985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:43.773985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:40:43.774293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:43.774293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:40:43.774293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:43.774293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:40:43.774293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:43.774293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:40:43.774293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:43.774293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:40:43.774293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:40:43.774293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:43.774293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:40:43.774293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:43.774293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:40:43.774293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:40:43.774611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:43.774611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:40:43.774611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:43.774611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:40:43.775045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:43.775045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:40:43.775045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:40:43.775045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:40:43.775045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:40:43.775045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:40:43.775045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:40:43.775045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:40:43.775045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:40:43.775045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:40:43.775045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:40:43.775045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:40:43.775045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:40:43.775045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:40:43.775045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:40:43.775045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:40:43.775448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:40:43.775448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:40:43.775448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:40:43.775448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:40:43.775448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:40:43.775448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:43.775448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:40:43.775448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:43.775448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:40:43.775448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:43.775448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:40:43.775448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:43.775448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:40:43.775448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:43.775448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:40:43.775448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:40:43.775448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:43.775448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:40:43.775817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:43.775817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:40:43.775817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:40:43.775817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:43.775817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:40:43.775817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:43.775817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:40:43.778023 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:40:43.778360 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:40:43.778472 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:40:43.778596 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:40:43.778968 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:40:43.779670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:43.779894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:40:43.779894 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:43.779894 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:40:43.779894 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:40:43.779894 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:43.779894 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:40:43.779894 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:43.779894 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:40:43.780637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:43.780637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:40:43.780637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:40:43.780637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:40:43.780637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:43.780637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:40:43.780637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:43.780637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:40:43.780637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:43.780637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:40:43.780637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:43.780637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:40:43.780637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:43.780637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:40:43.780637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:40:43.780943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:43.780943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:40:43.780943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:43.780943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:40:43.780943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:40:43.780943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:43.780943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:40:43.780943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:43.780943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:40:43.781685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:43.781685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:40:43.781685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:40:43.781685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:40:43.781685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:40:43.781685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:40:43.781685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:40:43.781685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:40:43.781685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:40:43.781685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:43.781685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:40:43.781964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:43.781964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:40:43.781964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:43.781964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:40:43.781964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:43.781964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:40:43.781964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:43.781964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:40:43.781964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:40:43.781964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:43.781964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:40:43.781964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:43.781964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:40:43.781964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:40:43.782294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:43.782294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:40:43.782294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:43.782294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:40:43.782760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:43.782760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:40:43.782760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:40:43.782760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:40:43.782760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:40:43.782760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:40:43.782760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:40:43.782760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:40:43.782760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:40:43.782760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:40:43.782760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:40:43.782760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:40:43.782760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:40:43.782760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:40:43.782760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:40:43.782760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:40:43.782760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:40:43.783417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:40:43.783417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:40:43.783417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:40:43.783417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:43.783417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:40:43.783417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:43.783417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:40:43.783417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:43.783417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:40:43.783417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:43.783417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:40:43.783417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:43.783417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:40:43.783417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:40:43.783417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:43.783417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:40:43.783417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:43.783417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:40:43.783417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:40:43.784107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:43.784107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:40:43.784107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:43.784107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:40:43.784107 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:40:43.784107 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:40:43.784107 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:40:43.784107 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:40:43.785867 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:40:43.785867 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:40:43.785867 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:43.786049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:40:43.786049 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:43.786049 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:40:43.786049 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:40:43.786049 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:43.786049 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:40:43.786049 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:43.786049 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:40:43.786965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:43.786965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:40:43.786965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:40:43.786965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:40:43.786965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:40:43.786965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:40:43.786965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:40:43.786965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:43.786965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:40:43.786965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:43.786965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:40:43.786965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:43.787357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:40:43.787357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:43.787357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:40:43.787357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:43.787357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:40:43.787357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:40:43.787357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:43.787357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:40:43.787357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:43.787357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:40:43.787357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:40:43.787357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:43.787357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:40:43.787736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:43.787736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:40:43.787969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:43.787969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:40:43.787969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:40:43.787969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:40:43.787969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:40:43.787969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:40:43.787969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:40:43.787969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:40:43.787969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:40:43.787969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:40:43.787969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:40:43.787969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:40:43.787969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:40:43.787969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:40:43.787969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:40:43.787969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:40:43.788450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:40:43.788450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:40:43.788450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:43.788450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:40:43.788450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:43.788450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:40:43.788450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:43.788450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:40:43.788450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:43.788450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:40:43.788450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:43.788450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:40:43.788450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:40:43.788450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:43.788450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:40:43.788450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:43.788450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:40:43.788450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:40:43.788450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:43.789075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:40:43.789075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:43.789075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:40:48.610865 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:40:48.611698 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:40:48.611698 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:40:48.612798 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:40:48.654089 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:40:48.654285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:48.655001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:40:48.655001 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:48.655001 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:40:48.655001 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:40:48.655001 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:48.655001 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:40:48.655001 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:48.655001 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:40:48.656130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.044 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:48.656130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:40:48.656130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:40:48.656130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:40:48.656130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:40:48.656130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:40:48.656130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:40:48.656130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:48.656130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:40:48.656130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:48.656130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:40:48.656130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:48.656605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:40:48.656605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:48.656605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:40:48.656605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:48.656605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:40:48.656605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:40:48.656605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:48.656605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:40:48.656605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:48.656605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:40:48.656605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:40:48.656605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:48.656605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:40:48.657142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:48.657142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:40:48.657579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:48.657579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:40:48.657579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:40:48.657579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:40:48.657579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:40:48.657579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:40:48.657579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:40:48.657579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:40:48.657579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:40:48.657579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:40:48.657579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:40:48.657579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:40:48.657579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:40:48.657579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:40:48.657579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:40:48.657579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:40:48.658113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:40:48.658113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:40:48.658113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:48.658113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:40:48.658113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:48.658113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:40:48.658113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:48.658113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:40:48.658113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:48.658113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:40:48.658113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:48.658113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:40:48.658113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:40:48.658113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:48.658113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:40:48.658113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:48.658113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:40:48.658113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:40:48.658113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:48.658664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:40:48.658664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:48.658664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:40:48.762869 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:40:48.763253 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:40:48.763381 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:40:48.764618 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:40:48.765212 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:40:48.765571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:48.766324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:40:48.766324 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:48.766324 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:40:48.766324 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:40:48.766324 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:48.766324 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:40:48.766324 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:48.766324 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:40:48.767367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:48.767367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:40:48.767367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:40:48.767367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:40:48.767367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:40:48.767367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:40:48.767367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:40:48.767367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:48.767367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:40:48.767367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:48.767367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:40:48.767367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:48.767809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:40:48.767809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:48.767809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:40:48.767809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:48.767809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:40:48.767809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:40:48.767809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:48.767809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:40:48.767809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:48.767809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:40:48.767809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:40:48.767809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:48.767809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:40:48.768205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:48.768205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:40:48.768510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:48.768510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:40:48.768510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:40:48.768510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:40:48.768510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:40:48.768510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:40:48.768510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:40:48.768510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:40:48.768510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:40:48.768510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:40:48.768510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:40:48.768510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:40:48.768510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:40:48.768510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:40:48.768510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:40:48.768510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:40:48.769172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:40:48.769172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:40:48.769172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:48.769172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:40:48.769172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:48.769172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:40:48.769172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:48.769172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:40:48.769172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:48.769172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:40:48.769172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:48.769172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:40:48.769172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:40:48.769172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:48.769172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:40:48.769172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:48.769172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:40:48.769172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:40:48.769172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:48.769682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:40:48.769682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:48.769682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:40:48.770252 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:40:48.770510 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:40:48.770629 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:40:48.770756 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:40:48.771285 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:40:48.771604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:48.772349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:40:48.772349 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:48.772349 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:40:48.772349 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:40:48.772349 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:48.772349 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:40:48.772349 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:48.772349 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:40:48.773176 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:48.773176 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:40:48.773176 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:40:48.773176 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:40:48.773176 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:48.773176 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:40:48.773176 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:48.773176 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:40:48.773176 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:48.773176 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:40:48.773176 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:48.773176 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:40:48.773176 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:48.773176 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:40:48.773176 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:40:48.773613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:48.773613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:40:48.773613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:48.773613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:40:48.773613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:40:48.773613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:48.773613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:40:48.773613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:48.773613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:40:48.774044 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:48.774044 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:40:48.774044 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:40:48.774044 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:40:48.774044 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:40:48.774044 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:40:48.774044 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:40:48.774044 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:40:48.774044 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:40:48.774044 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:40:48.774044 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:48.774044 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:40:48.774467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:48.774467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:40:48.774467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:48.774467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:40:48.774467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:48.774467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:40:48.774467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:48.774467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:40:48.774467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:40:48.774467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:48.774467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:40:48.774467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:48.774467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:40:48.774467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:40:48.774849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:48.774849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:40:48.774849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:48.774849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:40:48.775072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:48.775072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:40:48.775072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:40:48.775072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:40:48.775072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:40:48.775072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:40:48.775072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:40:48.775072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:40:48.775072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:40:48.775072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:40:48.775072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:40:48.775072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:40:48.775072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:40:48.775072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:40:48.775072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:40:48.775072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:40:48.775540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:40:48.775540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:40:48.775540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:40:48.775540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:40:48.775540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:40:48.775540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:48.775540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:40:48.775540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:48.775540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:40:48.775540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:48.775540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:40:48.775540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:48.775540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:40:48.775540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:48.775540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:40:48.775540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:40:48.775540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:48.775540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:40:48.776124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:48.776124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:40:48.776124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:40:48.776124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:48.776124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:40:48.776124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:48.776124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:40:48.778483 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:40:48.778966 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:40:48.779242 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:40:48.779469 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:40:48.779888 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:40:48.780123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:48.780730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:40:48.780730 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:48.780730 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:40:48.780730 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:40:48.780730 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:48.780730 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:40:48.780730 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:48.780730 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:40:48.782154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:48.782154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:40:48.782154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:40:48.782154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:40:48.782154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:48.782154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:40:48.782154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:48.782154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:40:48.782154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:48.782154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:40:48.782154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:48.782154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:40:48.782154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:48.782154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:40:48.782154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:40:48.782592 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:48.782592 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:40:48.782592 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:48.782592 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:40:48.782592 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:40:48.782592 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:48.782592 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:40:48.782592 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:48.782592 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:40:48.783863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:48.783863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:40:48.783863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:40:48.783863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:40:48.783863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:40:48.783863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:40:48.783863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:40:48.783863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:40:48.783863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:40:48.783863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:48.783863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:40:48.784279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:48.784279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:40:48.784279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:48.784279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:40:48.784279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:48.784279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:40:48.784279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:48.784279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:40:48.784279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:40:48.784279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:48.784279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:40:48.784279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:48.784279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:40:48.784279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:40:48.784673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:48.784673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:40:48.784673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:48.784673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:40:48.785637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:48.785637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:40:48.785637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:40:48.785637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:40:48.785637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:40:48.785637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:40:48.785637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:40:48.785637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:40:48.785637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:40:48.785637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:40:48.785637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:40:48.785637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:40:48.785637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:40:48.785637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:40:48.785637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:40:48.785637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:40:48.785637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:40:48.786167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:40:48.786167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:40:48.786167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:40:48.786167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:48.786167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:40:48.786167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:48.786167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:40:48.786167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:48.786167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:40:48.786167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:48.786167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:40:48.786167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:48.786167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:40:48.786167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:40:48.786167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:48.786167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:40:48.786167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:48.786167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:40:48.786167 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:40:48.786738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:48.786738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:40:48.786738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:48.786738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:40:48.786738 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:40:48.786738 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:40:48.786738 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:40:48.786738 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:40:48.787075 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:40:48.787928 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:40:48.788221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:48.789348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:40:48.789348 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:48.789348 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:40:48.789348 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:40:48.789348 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:48.789348 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:40:48.789348 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:48.789348 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:40:48.791405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:48.791405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:40:48.791405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:40:48.791405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:40:48.791405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:40:48.791405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:40:48.791405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:40:48.791405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:48.791405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:40:48.791405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:48.791405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:40:48.791405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:48.791953 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:40:48.791953 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:48.791953 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:40:48.791953 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:48.791953 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:40:48.791953 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:40:48.791953 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:48.791953 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:40:48.791953 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:48.791953 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:40:48.791953 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:40:48.791953 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:48.791953 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:40:48.792465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:48.792465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:40:48.793639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:48.793639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:40:48.793639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:40:48.793639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:40:48.793639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:40:48.793639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:40:48.793639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:40:48.793639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:40:48.793639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:40:48.793639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:40:48.793639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:40:48.793639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:40:48.793639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:40:48.793639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:40:48.793639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:40:48.793639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:40:48.794172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:40:48.794172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:40:48.794172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:48.794172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:40:48.794172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:48.794172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:40:48.794172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:48.794172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:40:48.794172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:48.794172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:40:48.794172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:48.794172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:40:48.794172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:40:48.794172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:48.794172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:40:48.794172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:48.794172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:40:48.794172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:40:48.794172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:48.794683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:40:48.794683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:48.794683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:40:53.611313 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:40:53.612060 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:40:53.612060 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:40:53.612877 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:40:53.647408 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:40:53.647677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:53.649202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:40:53.649202 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:53.649202 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:40:53.649202 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:40:53.649202 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:53.649202 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:40:53.649202 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:53.649202 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:40:53.650236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:53.650236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:40:53.650236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:40:53.650236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:40:53.650236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:40:53.650236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:40:53.650236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:40:53.650236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:53.650236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:40:53.650236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:53.650236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:40:53.650236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:53.650865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:40:53.650865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:53.650865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:40:53.650865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:53.650865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:40:53.650865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:40:53.650865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:53.650865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:40:53.650865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:53.650865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:40:53.650865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:40:53.650865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:53.650865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:40:53.651453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:53.651453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:40:53.651453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:53.651453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:40:53.651453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:40:53.651453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:40:53.651453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:40:53.651453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:40:53.651453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:40:53.651453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:40:53.651453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:40:53.651453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:40:53.651453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:40:53.651453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:40:53.651453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:40:53.652419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:40:53.652419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:40:53.652419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:40:53.652419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:40:53.652419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:40:53.652419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:53.652419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:40:53.652419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:53.652419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:40:53.652419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:53.652419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:40:53.652419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:53.652419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:40:53.652419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:53.652419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:40:53.652419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:40:53.652419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:53.652419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:40:53.652419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:53.653819 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:40:53.653819 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:40:53.653819 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:53.653819 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:40:53.653819 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:53.653819 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:40:53.763707 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:40:53.764447 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:40:53.764625 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:40:53.765824 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:40:53.766228 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:40:53.766492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:53.767370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:40:53.767370 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:53.767370 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:40:53.767370 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:40:53.767370 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:53.767370 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:40:53.767370 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:53.767370 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:40:53.768429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:53.768429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:40:53.768429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:40:53.768429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:40:53.768429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:40:53.768429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:40:53.768429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:40:53.768429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:53.768429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:40:53.768429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:53.768429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:40:53.768429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:53.769087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:40:53.769087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:53.769087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:40:53.769087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:53.769087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:40:53.769087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:40:53.769087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:53.769087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:40:53.769087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:53.769087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:40:53.769087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:40:53.769087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:53.769087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:40:53.769729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:53.769729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:40:53.769729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:53.769729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:40:53.769729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:40:53.769729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:40:53.769729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:40:53.769729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:40:53.769729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:40:53.769729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:40:53.769729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:40:53.769729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:40:53.769729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:40:53.769729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:40:53.769729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:40:53.770494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:40:53.770494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:40:53.770494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:40:53.770494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:40:53.770494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:40:53.770494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:53.770494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:40:53.770494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:53.770494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:40:53.770494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:53.770494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:40:53.770494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:53.770494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:40:53.770494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:53.770494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:40:53.770494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:40:53.770494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:53.770494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:40:53.770494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:53.771469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:40:53.771469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:40:53.771469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:53.771469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:40:53.771469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:53.771469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:40:53.771469 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:40:53.771469 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:40:53.771469 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:40:53.771469 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:40:53.772215 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:40:53.772215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:53.772215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:40:53.772215 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:53.772215 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:40:53.772215 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:40:53.772215 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:53.772215 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:40:53.772215 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:53.772215 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:40:53.773016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:53.773016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:40:53.773016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:40:53.773016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:40:53.773016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:53.773016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:40:53.773016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:53.773016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:40:53.773016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:53.773016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:40:53.773016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:53.773016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:40:53.773016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:53.773016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:40:53.773016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:40:53.773524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:53.773524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:40:53.773524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:53.773524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:40:53.773524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:40:53.773524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:53.773524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:40:53.773524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:53.773524 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:40:53.773904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:53.773904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:40:53.773904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:40:53.773904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:40:53.773904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:40:53.773904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:40:53.773904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:40:53.773904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:40:53.773904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:40:53.773904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:40:53.773904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:53.773904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:40:53.774275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:53.774275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:40:53.774275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:53.774275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:40:53.774275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:53.774275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:40:53.774275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:53.774275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:40:53.774275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:40:53.774275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:53.774275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:40:53.774275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:53.774275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:40:53.774275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:40:53.774666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:53.774666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:40:53.774666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:53.774666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:40:53.774855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:53.774855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:40:53.774855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:40:53.774855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:40:53.774855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:40:53.774855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:40:53.774855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:40:53.774855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:40:53.774855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:40:53.774855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:40:53.774855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:40:53.774855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:40:53.774855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:40:53.774855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:40:53.774855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:40:53.774855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:40:53.775299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:40:53.775299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:40:53.775299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:40:53.775299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:40:53.775299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:40:53.775299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:53.775299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:40:53.775299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:53.775299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:40:53.775299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:53.775299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:40:53.775299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:53.775299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:40:53.775299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:53.775299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:40:53.775299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:40:53.775299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:53.775299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:40:53.775915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:53.775915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:40:53.775915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:40:53.775915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:53.775915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:40:53.775915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:53.775915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:40:53.778950 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:40:53.779235 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:40:53.779349 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:40:53.779476 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:40:53.779813 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:40:53.779981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:53.780387 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:40:53.780387 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:53.780387 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:40:53.780387 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:40:53.780387 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:53.780387 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:40:53.780387 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:53.780387 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:40:53.781119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:53.781119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:40:53.781119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:40:53.781119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:40:53.781119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:53.781119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:40:53.781119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:53.781119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:40:53.781119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:53.781119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:40:53.781119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:53.781119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:40:53.781119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:53.781119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:40:53.781119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:40:53.781917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:53.781917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:40:53.781917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:53.781917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:40:53.781917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:40:53.781917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:53.781917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:40:53.781917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:53.781917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:40:53.782267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:53.782267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:40:53.782267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:40:53.782267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:40:53.782267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:40:53.782267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:40:53.782267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:40:53.782267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:40:53.782267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:40:53.782267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:53.782267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:40:53.782715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:53.782715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:40:53.782715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:53.782715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:40:53.782715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:53.782715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:40:53.782715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:53.782715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:40:53.782715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:40:53.782715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:53.782715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:40:53.782715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:53.782715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:40:53.782715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:40:53.783117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:53.783117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:40:53.783117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:53.783117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:40:53.783117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:53.783117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:40:53.783117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:40:53.783117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:40:53.783117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:40:53.783117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:40:53.783117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:40:53.783117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:40:53.783117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:40:53.783117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:40:53.783998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:40:53.783998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:40:53.783998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:40:53.783998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:40:53.783998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:40:53.783998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:40:53.783998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:40:53.783998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:40:53.783998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:40:53.783998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:40:53.783998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:53.783998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:40:53.783998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:53.783998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:40:53.783998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:53.783998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:40:53.783998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:53.783998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:40:53.783998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:53.784574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:40:53.784574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:40:53.784574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:53.784574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:40:53.784574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:53.784574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:40:53.784574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:40:53.784574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:53.784574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:40:53.784574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:53.784574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:40:53.786277 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:40:53.786569 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:40:53.786683 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:40:53.786859 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:40:53.787181 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:40:53.787465 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:40:53.787641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:53.788274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:40:53.788274 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:53.788274 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:40:53.788274 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:40:53.788274 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:53.788274 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:40:53.788274 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:53.788274 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:40:53.789910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:53.789910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:40:53.789910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:40:53.789910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:40:53.789910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:40:53.789910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:40:53.789910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:40:53.789910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:53.789910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:40:53.789910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:53.789910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:40:53.789910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:53.790330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:40:53.790330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:53.790330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:40:53.790330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:53.790330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:40:53.790330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:40:53.790330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:53.790330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:40:53.790330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:53.790330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:40:53.790330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:40:53.790330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:53.790330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:40:53.790665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:53.790665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:40:53.790665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:53.790665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:40:53.790665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:40:53.790665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:40:53.790665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:40:53.790665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:40:53.790665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:40:53.790665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:40:53.790665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:40:53.790665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:40:53.790665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:40:53.790665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:40:53.790665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:40:53.791098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:40:53.791098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:40:53.791098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:40:53.791098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:40:53.791098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:40:53.791098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:53.791098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:40:53.791098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:53.791098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:40:53.791098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:53.791098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:40:53.791098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:53.791098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:40:53.791098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:53.791098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:40:53.791098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:40:53.791098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:53.791098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:40:53.791098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:53.791615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:40:53.791615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:40:53.791615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:53.791615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:40:53.791615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:53.791615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:40:58.612012 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:40:58.612625 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:40:58.612625 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:40:58.614058 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:40:58.617268 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG dbcounter [-] [106314] Writing DB stats neutron:SELECT=240 {{(pid=106314) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 09 09:40:58.651115 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:40:58.651432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:58.652111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:40:58.652111 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:58.652111 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:40:58.652111 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:40:58.652111 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:58.652111 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:40:58.652111 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:58.652111 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:40:58.653062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:58.653062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:40:58.653062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:40:58.653062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:40:58.653062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:40:58.653062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:40:58.653062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:40:58.653062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:58.653062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:40:58.653062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:58.653062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:40:58.653062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:58.653880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:40:58.653880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:58.653880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:40:58.653880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:58.653880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:40:58.653880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:40:58.653880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:58.653880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:40:58.653880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:58.653880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:40:58.653880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:40:58.653880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:58.653880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:40:58.654653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:58.654653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:40:58.654653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:58.654653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:40:58.654653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:40:58.654653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:40:58.654653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:40:58.654653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:40:58.654653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:40:58.654653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:40:58.654653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:40:58.654653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:40:58.654653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:40:58.654653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:40:58.654653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:40:58.655272 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:40:58.655272 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:40:58.655272 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:40:58.655272 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:40:58.655272 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:40:58.655272 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:58.655272 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:40:58.655272 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:58.655272 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:40:58.655272 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:58.655272 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:40:58.655272 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:58.655272 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:40:58.655272 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:58.655272 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:40:58.655272 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:40:58.655272 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:58.655272 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:40:58.655272 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:58.655873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:40:58.655873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:40:58.655873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:58.655873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:40:58.655873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:58.655873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:40:58.764524 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:40:58.765229 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:40:58.765437 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:40:58.766640 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:40:58.767112 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:40:58.767474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:58.768435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:40:58.768435 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:58.768435 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:40:58.768435 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:40:58.768435 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:58.768435 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:40:58.768435 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:58.768435 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:40:58.769617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:58.769617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:40:58.769617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:40:58.769617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:40:58.769617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:40:58.769617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:40:58.769617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:40:58.769617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:58.769617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:40:58.769617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:58.769617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:40:58.769617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:58.770295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:40:58.770295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:58.770295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:40:58.770295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:58.770295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:40:58.770295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:40:58.770295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:58.770295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:40:58.770295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:58.770295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:40:58.770295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:40:58.770295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:58.770295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:40:58.770879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:58.770879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:40:58.771187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:58.771187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:40:58.771187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:40:58.771187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:40:58.771187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:40:58.771187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:40:58.771187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:40:58.771187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:40:58.771187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:40:58.771187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:40:58.771187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:40:58.771187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:40:58.771187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:40:58.771187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:40:58.771187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:40:58.771187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:40:58.772087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:40:58.772087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:40:58.772087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:58.772087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:40:58.772087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:58.772087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:40:58.772087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:58.772087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:40:58.772087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:58.772087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:40:58.772087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:58.772087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:40:58.772087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:40:58.772087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:58.772087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:40:58.772087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:58.772087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:40:58.772087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:40:58.772087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:58.772769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:40:58.772769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:58.772769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:40:58.772769 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:40:58.772769 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:40:58.772769 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:40:58.772769 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:40:58.772993 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:40:58.773158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:58.773760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:40:58.773760 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:58.773760 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:40:58.773760 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:40:58.773760 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:58.773760 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:40:58.773760 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:58.773760 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:40:58.775398 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:58.775398 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:40:58.775398 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:40:58.775398 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:40:58.775398 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:58.775398 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:40:58.775398 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:58.775398 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:40:58.775398 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:58.775398 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:40:58.775398 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:58.775398 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:40:58.775398 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:58.775398 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:40:58.775398 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:40:58.776235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:58.776235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:40:58.776235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:58.776235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:40:58.776235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:40:58.776235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:58.776235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:40:58.776235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:58.776235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:40:58.776235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:58.776235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:40:58.776235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:40:58.776235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:40:58.776235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:40:58.776888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:40:58.776888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:40:58.776888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:40:58.776888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:40:58.776888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:40:58.776888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:58.776888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:40:58.776888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:58.776888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:40:58.776888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:58.776888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:40:58.776888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:58.776888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:40:58.777981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:58.777981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:40:58.777981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:40:58.777981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:58.777981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:40:58.777981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:58.777981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:40:58.777981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:40:58.777981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:58.777981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:40:58.777981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:58.777981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:40:58.778477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:58.778477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:40:58.778477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:40:58.778477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:40:58.778477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:40:58.778477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:40:58.778477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:40:58.778477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:40:58.778477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:40:58.778477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:40:58.778477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:40:58.778477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:40:58.778477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:40:58.778477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:40:58.778477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:40:58.778477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:40:58.779130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:40:58.779130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:40:58.779130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:40:58.779130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:40:58.779130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:40:58.779130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:58.779130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:40:58.779130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:58.779130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:40:58.779130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:58.779130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:40:58.779130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:58.779130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:40:58.779130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:58.779130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:40:58.779130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:40:58.779130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:58.779130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:40:58.779615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:58.779615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:40:58.779615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:40:58.779615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:58.779615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:40:58.779615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:58.779615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:40:58.779615 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:40:58.779615 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:40:58.779615 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:40:58.779913 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:40:58.780088 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:40:58.780322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:58.780848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:40:58.780848 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:58.780848 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:40:58.780848 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:40:58.780848 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:58.780848 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:40:58.780848 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:58.780848 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:40:58.781397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:58.781397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:40:58.781397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:40:58.781397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:40:58.781397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:58.781397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:40:58.781397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:58.781397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:40:58.781397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:58.781397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:40:58.781397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:58.781397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:40:58.781397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:58.781397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:40:58.781397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:40:58.782010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:58.782010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:40:58.782010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:58.782010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:40:58.782010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:40:58.782010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:58.782010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:40:58.782010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:58.782010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:40:58.782323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:58.782323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:40:58.782323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:40:58.782323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:40:58.782323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:40:58.782323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:40:58.782323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:40:58.782323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:40:58.782323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:40:58.782323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:58.782323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:40:58.782660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:58.782660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:40:58.782660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:58.782660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:40:58.782660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:58.782660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:40:58.782660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:58.782660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:40:58.782660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:40:58.782660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:58.782660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:40:58.782660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:58.782660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:40:58.782660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:40:58.783085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:58.783085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:40:58.783085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:58.783085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:40:58.783085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:58.783085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:40:58.783085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:40:58.783085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:40:58.783085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:40:58.783085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:40:58.783085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:40:58.783085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:40:58.783085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:40:58.783085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:40:58.783573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:40:58.783573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:40:58.783573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:40:58.783573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:40:58.783573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:40:58.783573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:40:58.783573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:40:58.783573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:40:58.783573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:40:58.783573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:40:58.783573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:58.783573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:40:58.783573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:58.783573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:40:58.783573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:58.783573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:40:58.783573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:58.783573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:40:58.783573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:58.784261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:40:58.784261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:40:58.784261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:58.784261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:40:58.784261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:58.784261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:40:58.784261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:40:58.784261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:58.784261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:40:58.784261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:58.784261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:40:58.786634 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:40:58.786828 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:40:58.786943 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:40:58.787073 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:40:58.787373 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:40:58.787610 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:40:58.787860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:58.788395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:40:58.788395 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:58.788395 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:40:58.788395 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:40:58.788395 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:58.788395 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:40:58.788395 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:58.788395 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:40:58.789053 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:58.789053 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:40:58.789053 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:40:58.789053 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:40:58.789053 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:40:58.789053 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:40:58.789053 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:40:58.789053 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:58.789053 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:40:58.789053 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:58.789053 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:40:58.789053 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:58.789619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:40:58.789619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:58.789619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:40:58.789619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:58.789619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:40:58.789619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:40:58.789619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:58.789619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:40:58.789619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:58.789619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:40:58.789619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:40:58.789619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:58.789619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:40:58.790124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:58.790124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:40:58.790124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:58.790124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:40:58.790124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:40:58.790124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:40:58.790124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:40:58.790124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:40:58.790124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:40:58.790124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:40:58.790124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:40:58.790124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:40:58.790124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:40:58.790124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:40:58.790124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:40:58.790955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:40:58.790955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:40:58.790955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:40:58.790955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:40:58.790955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:40:58.790955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:58.790955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:40:58.790955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:40:58.790955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:40:58.790955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:40:58.790955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:40:58.790955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:40:58.790955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:40:58.790955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:40:58.790955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:40:58.790955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:40:58.790955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:40:58.790955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:40:58.790955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:40:58.791677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:40:58.791677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:40:58.791677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:40:58.791677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:40:58.791677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:40:58.791677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:41:03.612894 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:41:03.613698 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:41:03.613698 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:41:03.614512 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:41:03.656438 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:41:03.656825 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:03.657530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:41:03.657530 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:03.657530 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:41:03.657530 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:41:03.657530 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:03.657530 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:41:03.657530 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:03.657530 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:41:03.658522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.044 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:03.658522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:41:03.658522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:41:03.658522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:41:03.658522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:41:03.658522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:41:03.658522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:41:03.658522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:03.658522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:41:03.658522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:03.658522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:41:03.658522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:03.659222 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:41:03.659222 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:03.659222 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:41:03.659222 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:03.659222 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:41:03.659222 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:41:03.659222 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:03.659222 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:41:03.659222 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:03.659222 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:41:03.659222 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:41:03.659222 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:03.659222 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:41:03.659892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:03.659892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:41:03.659892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:03.659892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:41:03.659892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:41:03.659892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:41:03.659892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:41:03.659892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:41:03.659892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:41:03.659892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:41:03.659892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:41:03.659892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:41:03.659892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:41:03.659892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:41:03.659892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:41:03.660648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:41:03.660648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:41:03.660648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:41:03.660648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:41:03.660648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:41:03.660648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:03.660648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:41:03.660648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:03.660648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:41:03.660648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:03.660648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:41:03.660648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:03.660648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:41:03.660648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:03.660648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:41:03.660648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:41:03.660648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:03.660648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:41:03.660648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:03.661736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:41:03.661736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:41:03.661736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:03.661736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:41:03.661736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:03.661736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:41:03.765187 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:41:03.765721 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:41:03.765869 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:41:03.766623 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:41:03.766889 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:41:03.767070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:03.767648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:41:03.767648 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:03.767648 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:41:03.767648 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:41:03.767648 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:03.767648 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:41:03.767648 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:03.767648 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:41:03.768474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:03.768474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:41:03.768474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:41:03.768474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:41:03.768474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:41:03.768474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:41:03.768474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:41:03.768474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:03.768474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:41:03.768474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:03.768474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:41:03.768474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:03.768792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:41:03.768792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:03.768792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:41:03.768792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:03.768792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:41:03.768792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:41:03.768792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:03.768792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:41:03.768792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:03.768792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:41:03.768792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:41:03.768792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:03.768792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:41:03.769105 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:03.769105 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:41:03.769359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:03.769359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:41:03.769359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:41:03.769359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:41:03.769359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:41:03.769359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:41:03.769359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:41:03.769359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:41:03.769359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:41:03.769359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:41:03.769359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:41:03.769359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:41:03.769359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:41:03.769359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:41:03.769359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:41:03.769359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:41:03.769700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:41:03.769700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:41:03.769700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:03.769700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:41:03.769700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:03.769700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:41:03.769700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:03.769700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:41:03.769700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:03.769700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:41:03.769700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:03.769700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:41:03.769700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:41:03.769700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:03.769700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:41:03.769700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:03.769700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:41:03.769700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:41:03.769700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:03.770106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:41:03.770106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:03.770106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:41:03.771962 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:41:03.772216 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:41:03.772288 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:41:03.772360 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:41:03.772674 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:41:03.772921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:03.773395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:41:03.773395 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:03.773395 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:41:03.773395 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:41:03.773395 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:03.773395 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:41:03.773395 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:03.773395 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:41:03.774114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:03.774114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:41:03.774114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:41:03.774114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:41:03.774114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:03.774114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:41:03.774114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:03.774114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:41:03.774114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:03.774114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:41:03.774114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:03.774114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:41:03.774114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:03.774114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:41:03.774114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:41:03.774661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:03.774661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:41:03.774661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:03.774661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:41:03.774661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:41:03.774661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:03.774661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:41:03.774661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:03.774661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:41:03.774986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:03.774986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:41:03.774986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:41:03.774986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:41:03.774986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:41:03.774986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:41:03.774986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:41:03.774986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:41:03.774986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:41:03.774986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:41:03.774986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:03.774986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:41:03.775319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:03.775319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:41:03.775319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:03.775319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:41:03.775319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:03.775319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:41:03.775319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:03.775319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:41:03.775319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:41:03.775319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:03.775319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:41:03.775319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:03.775319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:41:03.775319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:41:03.775671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:03.775671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:41:03.775671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:03.775671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:41:03.775671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:03.775671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:41:03.775671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:41:03.775671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:41:03.775671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:41:03.775671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:41:03.775671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:41:03.775671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:41:03.775671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:41:03.775671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:41:03.776087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:41:03.776087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:41:03.776087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:41:03.776087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:41:03.776087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:41:03.776087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:41:03.776087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:41:03.776087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:41:03.776087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:41:03.776087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:41:03.776087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:41:03.776087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:03.776087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:41:03.776087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:03.776087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:41:03.776087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:03.776087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:41:03.776087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:03.776087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:41:03.776726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:03.776726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:41:03.776726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:41:03.776726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:03.776726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:41:03.776726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:03.776726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:41:03.776726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:41:03.776726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:03.776726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:41:03.776726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:03.776726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:41:03.779677 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:41:03.780081 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:41:03.780202 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:41:03.780350 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:41:03.780840 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:41:03.781227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:03.783009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:41:03.783009 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:03.783009 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:41:03.783009 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:41:03.783009 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:03.783009 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:41:03.783009 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:03.783009 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:41:03.784379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:03.784379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:41:03.784379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:41:03.784379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:41:03.784379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:03.784379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:41:03.784379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:03.784379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:41:03.784379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:03.784379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:41:03.784379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:03.784379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:41:03.784379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:03.784379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:41:03.784379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:41:03.785073 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:03.785073 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:41:03.785073 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:03.785073 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:41:03.785073 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:41:03.785073 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:03.785073 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:41:03.785073 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:03.785073 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:41:03.785073 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:03.785073 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:41:03.785073 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:41:03.785073 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:41:03.785073 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:41:03.785675 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:41:03.785675 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:41:03.785675 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:41:03.785675 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:41:03.785675 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:03.785675 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:41:03.785675 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:03.785675 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:41:03.785675 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:03.785675 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:41:03.785675 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:03.785675 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:41:03.785675 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:03.786148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:41:03.786148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:41:03.786148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:03.786148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:41:03.786148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:03.786148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:41:03.786148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:41:03.786148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:03.786148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:41:03.786148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:03.786148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:41:03.786460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:03.786460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:41:03.786460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:41:03.786460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:41:03.786460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:41:03.786460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:41:03.786460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:41:03.786460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:41:03.786460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:41:03.786460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:41:03.786460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:41:03.786460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:41:03.786460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:41:03.786460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:41:03.786460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:41:03.786460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:41:03.786460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:41:03.787558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:41:03.787558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:41:03.787558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:41:03.787558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:03.787558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:41:03.787558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:03.787558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:41:03.787558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:03.787558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:41:03.787558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:03.787558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:41:03.787558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:03.787558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:41:03.787558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:41:03.787558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:03.787558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:41:03.787558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:03.787558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:41:03.787558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:41:03.789030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:03.789030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:41:03.789030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:03.789030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:41:03.789030 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:41:03.789030 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:41:03.789030 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:41:03.789030 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:41:03.789634 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:41:03.789634 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:41:03.789634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:03.789634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:41:03.789634 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:03.789634 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:41:03.789634 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:41:03.789634 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:03.789634 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:41:03.789634 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:03.789634 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:41:03.790242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:03.790242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:41:03.790242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:41:03.790242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:41:03.790242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:41:03.790242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:41:03.790242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:41:03.790242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:03.790242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:41:03.790242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:03.790242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:41:03.790242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:03.791065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:41:03.791065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:03.791065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:41:03.791065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:03.791065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:41:03.791065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:41:03.791065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:03.791065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:41:03.791065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:03.791065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:41:03.791065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:41:03.791065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:03.791065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:41:03.791708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:03.791708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:41:03.791708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:03.791708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:41:03.791708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:41:03.791708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:41:03.791708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:41:03.791708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:41:03.791708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:41:03.791708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:41:03.791708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:41:03.791708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:41:03.791708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:41:03.791708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:41:03.791708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:41:03.792793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:41:03.792793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:41:03.792793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:41:03.792793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:41:03.792793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:41:03.792793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:03.792793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:41:03.792793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:03.792793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:41:03.792793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:03.792793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:41:03.792793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:03.792793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:41:03.792793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:03.792793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:41:03.792793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:41:03.792793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:03.792793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:41:03.792793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:03.793414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:41:03.793414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:41:03.793414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:03.793414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:41:03.793414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:03.793414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:41:08.613461 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:41:08.614041 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:41:08.614041 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:41:08.614662 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:41:08.643172 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:41:08.643483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:08.644659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:41:08.644659 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:08.644659 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:41:08.644659 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:41:08.644659 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:08.644659 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:41:08.644659 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:08.644659 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:41:08.645091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.030 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:08.645091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:41:08.645091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:41:08.645091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:41:08.645091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:41:08.645091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:41:08.645091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:41:08.645091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:08.645091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:41:08.645091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:08.645091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:41:08.645091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:08.645773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:41:08.645773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:08.645773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:41:08.645773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:08.645773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:41:08.645773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:41:08.645773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:08.645773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:41:08.645773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:08.645773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:41:08.645773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:41:08.645773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:08.645773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:41:08.646355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:08.646355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:41:08.646355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:08.646355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:41:08.646355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:41:08.646355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:41:08.646355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:41:08.646355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:41:08.646355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:41:08.646355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:41:08.646355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:41:08.646355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:41:08.646355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:41:08.646355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:41:08.646355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:41:08.646864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:41:08.646864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:41:08.646864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:41:08.646864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:41:08.646864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:41:08.646864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:08.646864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:41:08.646864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:08.646864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:41:08.646864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:08.646864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:41:08.646864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:08.646864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:41:08.646864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:08.646864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:41:08.646864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:41:08.646864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:08.646864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:41:08.646864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:08.647470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:41:08.647470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:41:08.647470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:08.647470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:41:08.647470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:08.647470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:41:08.765823 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:41:08.766378 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:41:08.766571 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:41:08.767626 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:41:08.767861 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:41:08.767994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:08.768852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:41:08.768852 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:08.768852 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:41:08.768852 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:41:08.768852 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:08.768852 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:41:08.768852 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:08.768852 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:41:08.770525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:08.770525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:41:08.770525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:41:08.770525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:41:08.770525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:41:08.770525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:41:08.770525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:41:08.770525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:08.770525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:41:08.770525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:08.770525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:41:08.770525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:08.770844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:41:08.770844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:08.770844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:41:08.770844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:08.770844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:41:08.770844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:41:08.770844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:08.770844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:41:08.770844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:08.770844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:41:08.770844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:41:08.770844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:08.770844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:41:08.771117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:08.771117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:41:08.772554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:08.772554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:41:08.772554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:41:08.772554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:41:08.772554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:41:08.772554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:41:08.772554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:41:08.772554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:41:08.772554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:41:08.772554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:41:08.772554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:41:08.772554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:41:08.772554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:41:08.772554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:41:08.772554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:41:08.772554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:41:08.772995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:41:08.772995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:41:08.772995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:08.772995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:41:08.772995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:08.772995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:41:08.772995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:08.772995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:41:08.772995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:08.772995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:41:08.772995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:08.772995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:41:08.772995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:41:08.772995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:08.772995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:41:08.772995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:08.772995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:41:08.772995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:41:08.772995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:08.773493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:41:08.773493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:08.773493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:41:08.773493 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:41:08.773493 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:41:08.773493 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:41:08.773684 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:41:08.774390 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:41:08.774715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:08.775549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:41:08.775549 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:08.775549 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:41:08.775549 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:41:08.775549 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:08.775549 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:41:08.775549 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:08.775549 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:41:08.776346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:08.776346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:41:08.776346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:41:08.776346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:41:08.776346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:08.776346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:41:08.776346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:08.776346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:41:08.776346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:08.776346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:41:08.776346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:08.776346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:41:08.776346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:08.776346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:41:08.776346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:41:08.776806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:08.776806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:41:08.776806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:08.776806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:41:08.776806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:41:08.776806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:08.776806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:41:08.776806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:08.776806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:41:08.777199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:08.777199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:41:08.777199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:41:08.777199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:41:08.777199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:41:08.777199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:41:08.777199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:41:08.777199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:41:08.777199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:41:08.777199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:41:08.777199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:08.777199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:41:08.777500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:08.777500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:41:08.777500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:08.777500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:41:08.777500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:08.777500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:41:08.777500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:08.777500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:41:08.777500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:41:08.777500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:08.777500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:41:08.777500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:08.777500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:41:08.777500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:41:08.777761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:08.777761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:41:08.777761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:08.777761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:41:08.778218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:08.778218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:41:08.778218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:41:08.778218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:41:08.778218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:41:08.778218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:41:08.778218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:41:08.778218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:41:08.778218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:41:08.778218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:41:08.778218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:41:08.778218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:41:08.778218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:41:08.778218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:41:08.778218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:41:08.778218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:41:08.778547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:41:08.778547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:41:08.778547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:41:08.778547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:41:08.778547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:41:08.778547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:08.778547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:41:08.778547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:08.778547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:41:08.778547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:08.778547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:41:08.778547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:08.778547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:41:08.778547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:08.778547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:41:08.778547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:41:08.778547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:08.778547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:41:08.778919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:08.778919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:41:08.778919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:41:08.778919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:08.778919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:41:08.778919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:08.778919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:41:08.780252 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:41:08.780481 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:41:08.780572 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:41:08.780697 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:41:08.781372 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:41:08.781668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:08.782266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:41:08.782266 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:08.782266 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:41:08.782266 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:41:08.782266 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:08.782266 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:41:08.782266 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:08.782266 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:41:08.782819 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:08.782819 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:41:08.782819 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:41:08.782819 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:41:08.782819 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:08.782819 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:41:08.782819 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:08.782819 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:41:08.782819 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:08.782819 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:41:08.782819 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:08.782819 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:41:08.782819 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:08.782819 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:41:08.782819 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:41:08.783139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:08.783139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:41:08.783139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:08.783139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:41:08.783139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:41:08.783139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:08.783139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:41:08.783139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:08.783139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:41:08.783432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:08.783432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:41:08.783432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:41:08.783432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:41:08.783432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:41:08.783432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:41:08.783432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:41:08.783432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:41:08.783432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:41:08.783432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:08.783432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:41:08.783814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:08.783814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:41:08.783814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:08.783814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:41:08.783814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:08.783814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:41:08.783814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:08.783814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:41:08.783814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:41:08.783814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:08.783814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:41:08.783814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:08.783814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:41:08.783814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:41:08.784096 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:08.784096 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:41:08.784096 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:08.784096 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:41:08.784282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:08.784282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:41:08.784282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:41:08.784282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:41:08.784282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:41:08.784282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:41:08.784282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:41:08.784282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:41:08.784282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:41:08.784282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:41:08.784282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:41:08.784282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:41:08.784282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:41:08.784282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:41:08.784282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:41:08.784282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:41:08.784282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:41:08.784791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:41:08.784791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:41:08.784791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:41:08.784791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:08.784791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:41:08.784791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:08.784791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:41:08.784791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:08.784791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:41:08.784791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:08.784791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:41:08.784791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:08.784791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:41:08.784791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:41:08.784791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:08.784791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:41:08.784791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:08.784791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:41:08.784791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:41:08.785262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:08.785262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:41:08.785262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:08.785262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:41:08.787538 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:41:08.787783 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:41:08.787862 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:41:08.787985 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:41:08.788707 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:41:08.789069 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:41:08.789237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:08.789884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:41:08.789884 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:08.789884 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:41:08.789884 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:41:08.789884 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:08.789884 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:41:08.789884 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:08.789884 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:41:08.790797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:08.790797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:41:08.790797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:41:08.790797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:41:08.790797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:41:08.790797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:41:08.790797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:41:08.790797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:08.790797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:41:08.790797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:08.790797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:41:08.790797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:08.791178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:41:08.791178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:08.791178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:41:08.791178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:08.791178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:41:08.791178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:41:08.791178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:08.791178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:41:08.791178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:08.791178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:41:08.791178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:41:08.791178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:08.791178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:41:08.791471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:08.791471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:41:08.792868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:08.792868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:41:08.792868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:41:08.792868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:41:08.792868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:41:08.792868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:41:08.792868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:41:08.792868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:41:08.792868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:41:08.792868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:41:08.792868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:41:08.792868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:41:08.792868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:41:08.792868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:41:08.792868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:41:08.792868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:41:08.793498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:41:08.793498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:41:08.793498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:08.793498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:41:08.793498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:08.793498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:41:08.793498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:08.793498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:41:08.793498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:08.793498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:41:08.793498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:08.793498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:41:08.793498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:41:08.793498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:08.793498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:41:08.793498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:08.793498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:41:08.793498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:41:08.793498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:08.794074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:41:08.794074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:08.794074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:41:13.614087 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:41:13.615052 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:41:13.615052 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:41:13.615409 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:41:13.654608 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:41:13.654844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:13.655270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:41:13.655270 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:13.655270 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:41:13.655270 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:41:13.655270 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:13.655270 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:41:13.655270 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:13.655270 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:41:13.656151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:13.656151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:41:13.656151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:41:13.656151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:41:13.656151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:41:13.656151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:41:13.656151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:41:13.656151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:13.656151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:41:13.656151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:13.656151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:41:13.656151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:13.656851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:41:13.656851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:13.656851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:41:13.656851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:13.656851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:41:13.656851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:41:13.656851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:13.656851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:41:13.656851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:13.656851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:41:13.656851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:41:13.656851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:13.656851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:41:13.657571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:13.657571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:41:13.657571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:13.657571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:41:13.657571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:41:13.657571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:41:13.657571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:41:13.657571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:41:13.657571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:41:13.657571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:41:13.657571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:41:13.657571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:41:13.657571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:41:13.657571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:41:13.657571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:41:13.658439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:41:13.658439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:41:13.658439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:41:13.658439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:41:13.658439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:41:13.658439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:13.658439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:41:13.658439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:13.658439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:41:13.658439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:13.658439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:41:13.658439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:13.658439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:41:13.658439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:13.658439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:41:13.658439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:41:13.658439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:13.658439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:41:13.658439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:13.659338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:41:13.659338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:41:13.659338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:13.659338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:41:13.659338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:13.659338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:41:13.766550 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:41:13.767123 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:41:13.767301 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:41:13.768204 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:41:13.768423 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:41:13.768566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:13.769187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:41:13.769187 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:13.769187 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:41:13.769187 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:41:13.769187 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:13.769187 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:41:13.769187 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:13.769187 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:41:13.770656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:13.770656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:41:13.770656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:41:13.770656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:41:13.770656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:41:13.770656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:41:13.770656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:41:13.770656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:13.770656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:41:13.770656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:13.770656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:41:13.770656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:13.771314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:41:13.771314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:13.771314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:41:13.771314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:13.771314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:41:13.771314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:41:13.771314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:13.771314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:41:13.771314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:13.771314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:41:13.771314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:41:13.771314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:13.771314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:41:13.771953 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:13.771953 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:41:13.772058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:13.772058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:41:13.772058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:41:13.772058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:41:13.772058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:41:13.772058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:41:13.772058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:41:13.772058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:41:13.772058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:41:13.772058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:41:13.772058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:41:13.772058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:41:13.772058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:41:13.772058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:41:13.772058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:41:13.772058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:41:13.772955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:41:13.772955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:41:13.772955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:13.772955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:41:13.772955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:13.772955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:41:13.772955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:13.772955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:41:13.772955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:13.772955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:41:13.772955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:13.772955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:41:13.772955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:41:13.772955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:13.772955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:41:13.772955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:13.772955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:41:13.772955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:41:13.772955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:13.773945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:41:13.773945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:13.773945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:41:13.773945 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:41:13.773945 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:41:13.773945 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:41:13.773945 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:41:13.774414 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:41:13.774561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:13.776377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:41:13.776377 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:13.776377 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:41:13.776377 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:41:13.776377 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:13.776377 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:41:13.776377 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:13.776377 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:41:13.777451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:13.777451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:41:13.777451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:41:13.777451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:41:13.777451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:13.777451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:41:13.777451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:13.777451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:41:13.777451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:13.777451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:41:13.777451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:13.777451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:41:13.777451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:13.777451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:41:13.777451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:41:13.778263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:13.778263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:41:13.778263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:13.778263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:41:13.778263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:41:13.778263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:13.778263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:41:13.778263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:13.778263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:41:13.779165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:13.779165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:41:13.779165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:41:13.779165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:41:13.779165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:41:13.779165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:41:13.779165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:41:13.779165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:41:13.779165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:41:13.779165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:41:13.779165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:13.779165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:41:13.779607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:13.779607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:41:13.779607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:13.779607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:41:13.779607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:13.779607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:41:13.779607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:13.779607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:41:13.779607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:41:13.779607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:13.779607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:41:13.779607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:13.779607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:41:13.779607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:41:13.780036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:13.780036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:41:13.780036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:13.780036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:41:13.781015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:13.781015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:41:13.781015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:41:13.781015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:41:13.781015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:41:13.781015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:41:13.781015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:41:13.781015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:41:13.781015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:41:13.781015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:41:13.781015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:41:13.781015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:41:13.781015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:41:13.781015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:41:13.781015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:41:13.781015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:41:13.781530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:41:13.781530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:41:13.781530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:41:13.781530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:41:13.781530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:41:13.781530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:13.781530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:41:13.781530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:13.781530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:41:13.781530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:13.781530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:41:13.781530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:13.781530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:41:13.781530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:13.781530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:41:13.781530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:41:13.781530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:13.781530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:41:13.782341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:13.782341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:41:13.782341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:41:13.782341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:13.782341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:41:13.782341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:13.782341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:41:13.782341 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:41:13.782341 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:41:13.782341 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:41:13.782694 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:41:13.782694 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:41:13.782694 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:13.782694 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:41:13.782694 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:13.782694 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:41:13.782694 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:41:13.782694 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:13.782694 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:41:13.782694 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:13.782694 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:41:13.783221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:13.783221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:41:13.783221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:41:13.783221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:41:13.783221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:13.783221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:41:13.783221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:13.783221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:41:13.783221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:13.783221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:41:13.783221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:13.783221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:41:13.783221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:13.783221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:41:13.783221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:41:13.783648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:13.783648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:41:13.783648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:13.783648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:41:13.783648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:41:13.783648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:13.783648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:41:13.783648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:13.783648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:41:13.783978 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:13.783978 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:41:13.783978 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:41:13.783978 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:41:13.783978 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:41:13.783978 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:41:13.783978 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:41:13.783978 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:41:13.783978 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:41:13.783978 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:13.783978 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:41:13.784313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:13.784313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:41:13.784313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:13.784313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:41:13.784313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:13.784313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:41:13.784313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:13.784313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:41:13.784313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:41:13.784313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:13.784313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:41:13.784313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:13.784313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:41:13.784313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:41:13.784720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:13.784720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:41:13.784720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:13.784720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:41:13.785172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:13.785172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:41:13.785172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:41:13.785172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:41:13.785172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:41:13.785172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:41:13.785172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:41:13.785172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:41:13.785172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:41:13.785172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:41:13.785172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:41:13.785172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:41:13.785172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:41:13.785172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:41:13.785172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:41:13.785172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:41:13.785172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:41:13.785695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:41:13.785695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:41:13.785695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:41:13.785695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:13.785695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:41:13.785695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:13.785695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:41:13.785695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:13.785695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:41:13.785695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:13.785695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:41:13.785695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:13.785695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:41:13.785695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:41:13.785695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:13.785695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:41:13.785695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:13.785695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:41:13.785695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:41:13.786229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:13.786229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:41:13.786229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:13.786229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:41:13.787895 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:41:13.788234 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:41:13.788360 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:41:13.788529 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:41:13.788906 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:41:13.789247 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:41:13.789501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:13.790253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:41:13.790253 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:13.790253 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:41:13.790253 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:41:13.790253 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:13.790253 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:41:13.790253 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:13.790253 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:41:13.791165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:13.791165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:41:13.791165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:41:13.791165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:41:13.791165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:41:13.791165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:41:13.791165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:41:13.791165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:13.791165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:41:13.791165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:13.791165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:41:13.791165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:13.791532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:41:13.791532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:13.791532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:41:13.791532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:13.791532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:41:13.791532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:41:13.791532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:13.791532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:41:13.791532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:13.791532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:41:13.791532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:41:13.791532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:13.791532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:41:13.791943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:13.791943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:41:13.792165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:13.792165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:41:13.792165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:41:13.792165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:41:13.792165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:41:13.792165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:41:13.792165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:41:13.792165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:41:13.792165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:41:13.792165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:41:13.792165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:41:13.792165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:41:13.792165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:41:13.792165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:41:13.792165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:41:13.792165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:41:13.792700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:41:13.792700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:41:13.792700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:13.792700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:41:13.792700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:13.792700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:41:13.792700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:13.792700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:41:13.792700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:13.792700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:41:13.792700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:13.792700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:41:13.792700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:41:13.792700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:13.792700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:41:13.792700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:13.792700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:41:13.792700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:41:13.792700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:13.793264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:41:13.793264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:13.793264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:41:18.614901 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:41:18.615958 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:41:18.615958 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:41:18.617060 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:41:18.653945 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:41:18.653945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:18.654438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:41:18.654438 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:18.654438 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:41:18.654438 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:41:18.654438 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:18.654438 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:41:18.654438 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:18.654438 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:41:18.655542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:18.655542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:41:18.655542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:41:18.655542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:41:18.655542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:41:18.655542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:41:18.655542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:41:18.655542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:18.655542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:41:18.655542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:18.655542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:41:18.655542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:18.656100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:41:18.656100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:18.656100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:41:18.656100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:18.656100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:41:18.656100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:41:18.656100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:18.656100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:41:18.656100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:18.656100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:41:18.656100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:41:18.656100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:18.656100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:41:18.656641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:18.656641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:41:18.656976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:18.656976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:41:18.656976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:41:18.656976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:41:18.656976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:41:18.656976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:41:18.656976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:41:18.656976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:41:18.656976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:41:18.656976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:41:18.656976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:41:18.656976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:41:18.656976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:41:18.656976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:41:18.656976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:41:18.656976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:41:18.657546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:41:18.657546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:41:18.657546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:18.657546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:41:18.657546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:18.657546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:41:18.657546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:18.657546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:41:18.657546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:18.657546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:41:18.657546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:18.657546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:41:18.657546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:41:18.657546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:18.657546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:41:18.657546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:18.657546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:41:18.657546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:41:18.657546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:18.658129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:41:18.658129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:18.658129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:41:18.767320 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:41:18.768075 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:41:18.768360 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:41:18.769517 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:41:18.769769 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:41:18.769964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:18.770671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:41:18.770671 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:18.770671 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:41:18.770671 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:41:18.770671 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:18.770671 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:41:18.770671 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:18.770671 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:41:18.772506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:18.772506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:41:18.772506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:41:18.772506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:41:18.772506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:41:18.772506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:41:18.772506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:41:18.772506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:18.772506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:41:18.772506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:18.772506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:41:18.772506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:18.773377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:41:18.773377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:18.773377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:41:18.773377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:18.773377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:41:18.773377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:41:18.773377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:18.773377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:41:18.773377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:18.773377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:41:18.773377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:41:18.773377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:18.773377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:41:18.774255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:18.774255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:41:18.774899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:18.774899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:41:18.774899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:41:18.774899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:41:18.774899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:41:18.774899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:41:18.774899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:41:18.774899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:41:18.774899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:41:18.774899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:41:18.774899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:41:18.774899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:41:18.774899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:41:18.774899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:41:18.774899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:41:18.774899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:41:18.775874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:41:18.775874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:41:18.775874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:18.775874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:41:18.775874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:18.775874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:41:18.775874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:18.775874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:41:18.775874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:18.775874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:41:18.775874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:18.775874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:41:18.775874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:41:18.775874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:18.775874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:41:18.775874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:18.775874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:41:18.775874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:41:18.775874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:18.776993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:41:18.776993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:18.776993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:41:18.776993 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:41:18.776993 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:41:18.776993 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:41:18.776993 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:41:18.776993 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:41:18.777406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:18.777703 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:41:18.777703 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:18.777703 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:41:18.777703 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:41:18.777703 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:18.777703 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:41:18.777703 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:18.777703 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:41:18.778654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:18.778654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:41:18.778654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:41:18.778654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:41:18.778654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:18.778654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:41:18.778654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:18.778654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:41:18.778654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:18.778654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:41:18.778654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:18.778654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:41:18.778654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:18.778654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:41:18.778654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:41:18.779323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:18.779323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:41:18.779323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:18.779323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:41:18.779323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:41:18.779323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:18.779323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:41:18.779323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:18.779323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:41:18.779926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:18.779926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:41:18.779926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:41:18.779926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:41:18.779926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:41:18.779926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:41:18.779926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:41:18.779926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:41:18.779926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:41:18.779926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:41:18.779926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:18.779926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:41:18.780520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:18.780520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:41:18.780520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:18.780520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:41:18.780520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:18.780520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:41:18.780520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:18.780520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:41:18.780520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:41:18.780520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:18.780520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:41:18.780520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:18.780520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:41:18.780520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:41:18.781198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:18.781198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:41:18.781198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:18.781198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:41:18.781536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:18.781536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:41:18.781536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:41:18.781536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:41:18.781536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:41:18.781536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:41:18.781536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:41:18.781536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:41:18.781536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:41:18.781536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:41:18.781536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:41:18.781536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:41:18.781536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:41:18.781536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:41:18.781536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:41:18.781536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:41:18.782119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:41:18.782119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:41:18.782119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:41:18.782119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:41:18.782119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:41:18.782119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:18.782119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:41:18.782119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:18.782119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:41:18.782119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:18.782119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:41:18.782119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:18.782119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:41:18.782119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:18.782119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:41:18.782119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:41:18.782119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:18.782119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:41:18.782844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:18.782844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:41:18.782844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:41:18.782844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:18.782844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:41:18.782844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:18.782844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:41:18.782844 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:41:18.782844 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:41:18.782844 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:41:18.783141 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:41:18.783141 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:41:18.783141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:18.783488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:41:18.783488 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:18.783488 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:41:18.783488 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:41:18.783488 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:18.783488 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:41:18.783488 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:18.783488 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:41:18.784417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:18.784417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:41:18.784417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:41:18.784417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:41:18.784417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:18.784417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:41:18.784417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:18.784417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:41:18.784417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:18.784417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:41:18.784417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:18.784417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:41:18.784417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:18.784417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:41:18.784417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:41:18.785128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:18.785128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:41:18.785128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:18.785128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:41:18.785128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:41:18.785128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:18.785128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:41:18.785128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:18.785128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:41:18.785765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:18.785765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:41:18.785765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:41:18.785765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:41:18.785765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:41:18.785765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:41:18.785765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:41:18.785765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:41:18.785765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:41:18.785765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:18.785765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:41:18.786279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:18.786279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:41:18.786279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:18.786279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:41:18.786279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:18.786279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:41:18.786279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:18.786279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:41:18.786279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:41:18.786279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:18.786279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:41:18.786279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:18.786279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:41:18.786279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:41:18.786891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:18.786891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:41:18.786891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:18.786891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:41:18.787350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:18.787350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:41:18.787350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:41:18.787350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:41:18.787350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:41:18.787350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:41:18.787350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:41:18.787350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:41:18.787350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:41:18.787350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:41:18.787350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:41:18.787350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:41:18.787350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:41:18.787350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:41:18.787350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:41:18.787350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:41:18.787350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:41:18.787851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:41:18.787851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:41:18.787851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:41:18.787851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:18.787851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:41:18.787851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:18.787851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:41:18.787851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:18.787851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:41:18.787851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:18.787851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:41:18.787851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:18.787851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:41:18.787851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:41:18.787851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:18.787851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:41:18.787851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:18.787851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:41:18.787851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:41:18.788224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:18.788224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:41:18.788224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:18.788224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:41:18.788659 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:41:18.788968 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:41:18.789063 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:41:18.789216 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:41:18.789481 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:41:18.789646 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:41:18.789787 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:18.790392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:41:18.790392 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:18.790392 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:41:18.790392 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:41:18.790392 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:18.790392 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:41:18.790392 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:18.790392 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:41:18.791089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:18.791089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:41:18.791089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:41:18.791089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:41:18.791089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:41:18.791089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:41:18.791089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:41:18.791089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:18.791089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:41:18.791089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:18.791089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:41:18.791089 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:18.791467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:41:18.791467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:18.791467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:41:18.791467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:18.791467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:41:18.791467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:41:18.791467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:18.791467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:41:18.791467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:18.791467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:41:18.791467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:41:18.791467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:18.791467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:41:18.791877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:18.791877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:41:18.792085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:18.792085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:41:18.792085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:41:18.792085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:41:18.792085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:41:18.792085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:41:18.792085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:41:18.792085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:41:18.792085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:41:18.792085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:41:18.792085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:41:18.792085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:41:18.792085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:41:18.792085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:41:18.792085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:41:18.792085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:41:18.792414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:41:18.792414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:41:18.792414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:18.792414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:41:18.792414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:18.792414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:41:18.792414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:18.792414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:41:18.792414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:18.792414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:41:18.792414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:18.792414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:41:18.792414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:41:18.792414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:18.792414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:41:18.792414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:18.792414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:41:18.792414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:41:18.792414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:18.792856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:41:18.792856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:18.792856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:41:23.615726 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:41:23.616403 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:41:23.616560 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:41:23.618247 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:41:23.663669 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:41:23.663892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:23.664695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:41:23.664695 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:23.664695 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:41:23.664695 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:41:23.664695 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:23.664695 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:41:23.664695 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:23.664695 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:41:23.666162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.048 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:23.666162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:41:23.666162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:41:23.666162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:41:23.666162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:41:23.666162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:41:23.666162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:41:23.666162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:23.666162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:41:23.666162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:23.666162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:41:23.666162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:23.666610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:41:23.666610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:23.666610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:41:23.666610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:23.666610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:41:23.666610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:41:23.666610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:23.666610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:41:23.666610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:23.666610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:41:23.666610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:41:23.666610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:23.666610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:41:23.667162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:23.667162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:41:23.667718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:23.667718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:41:23.667718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:41:23.667718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:41:23.667718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:41:23.667718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:41:23.667718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:41:23.667718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:41:23.667718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:41:23.667718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:41:23.667718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:41:23.667718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:41:23.667718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:41:23.667718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:41:23.667718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:41:23.667718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:41:23.669439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:41:23.669439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:41:23.669439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:23.669439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:41:23.669439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:23.669439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:41:23.669439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:23.669439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:41:23.669439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:23.669439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:41:23.669439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:23.669439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:41:23.669439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:41:23.669439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:23.669439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:41:23.669439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:23.669439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:41:23.669439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:41:23.669439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:23.670045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:41:23.670045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:23.670045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:41:23.768120 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:41:23.768641 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:41:23.768813 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:41:23.770052 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:41:23.770310 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:41:23.770453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:23.772131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:41:23.772131 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:23.772131 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:41:23.772131 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:41:23.772131 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:23.772131 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:41:23.772131 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:23.772131 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:41:23.773898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:23.773898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:41:23.773898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:41:23.773898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:41:23.773898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:41:23.773898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:41:23.773898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:41:23.773898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:23.773898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:41:23.773898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:23.773898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:41:23.773898 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:23.774521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:41:23.774521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:23.774521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:41:23.774521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:23.774521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:41:23.774521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:41:23.774521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:23.774521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:41:23.774521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:23.774521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:41:23.774521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:41:23.774521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:23.774521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:41:23.775905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:23.775905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:41:23.776407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:23.776407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:41:23.776407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:41:23.776407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:41:23.776407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:41:23.776407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:41:23.776407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:41:23.776407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:41:23.776407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:41:23.776407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:41:23.776407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:41:23.776407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:41:23.776407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:41:23.776407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:41:23.776407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:41:23.776407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:41:23.778010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:41:23.778010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:41:23.778010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:23.778010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:41:23.778010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:23.778010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:41:23.778010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:23.778010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:41:23.778010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:23.778010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:41:23.778010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:23.778010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:41:23.778010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:41:23.778010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:23.778010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:41:23.778010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:23.778010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:41:23.778010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:41:23.778010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:23.778803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:41:23.778803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:23.778803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:41:23.778803 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:41:23.778803 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:41:23.778803 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:41:23.778803 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:41:23.778803 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:41:23.779246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:23.779246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:41:23.779246 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:23.779246 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:41:23.779246 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:41:23.779246 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:23.779246 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:41:23.779246 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:23.779246 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:41:23.780119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:23.780119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:41:23.780119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:41:23.780119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:41:23.780119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:23.780119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:41:23.780119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:23.780119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:41:23.780119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:23.780119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:41:23.780119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:23.780119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:41:23.780119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:23.780119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:41:23.780119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:41:23.780485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:23.780485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:41:23.780485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:23.780485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:41:23.780485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:41:23.780485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:23.780485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:41:23.780485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:23.780485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:41:23.780485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:23.780485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:41:23.780485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:41:23.780485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:41:23.780485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:41:23.781092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:41:23.781092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:41:23.781092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:41:23.781092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:41:23.781092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:41:23.781092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:23.781092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:41:23.781092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:23.781092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:41:23.781092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:23.781092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:41:23.781092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:23.781092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:41:23.781624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:23.781624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:41:23.781624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:41:23.781624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:23.781624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:41:23.781624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:23.781624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:41:23.781624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:41:23.781624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:23.781624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:41:23.781624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:23.781624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:41:23.782114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:23.782114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:41:23.782114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:41:23.782114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:41:23.782114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:41:23.782114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:41:23.782114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:41:23.782114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:41:23.782114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:41:23.782114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:41:23.782114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:41:23.782114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:41:23.782114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:41:23.782114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:41:23.782114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:41:23.782114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:41:23.782725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:41:23.782725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:41:23.782725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:41:23.782725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:41:23.782725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:41:23.782725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:23.782725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:41:23.782725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:23.782725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:41:23.782725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:23.782725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:41:23.782725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:23.782725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:41:23.782725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:23.782725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:41:23.782725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:41:23.782725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:23.782725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:41:23.783466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:23.783466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:41:23.783466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:41:23.783466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:23.783466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:41:23.783466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:23.783466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:41:23.783466 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:41:23.783466 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:41:23.783466 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:41:23.783907 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:41:23.783907 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:41:23.783907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:23.784271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:41:23.784271 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:23.784271 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:41:23.784271 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:41:23.784271 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:23.784271 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:41:23.784271 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:23.784271 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:41:23.784839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:23.784839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:41:23.784839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:41:23.784839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:41:23.784839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:23.784839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:41:23.784839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:23.784839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:41:23.784839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:23.784839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:41:23.784839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:23.784839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:41:23.784839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:23.784839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:41:23.784839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:41:23.785426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:23.785426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:41:23.785426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:23.785426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:41:23.785426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:41:23.785426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:23.785426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:41:23.785426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:23.785426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:41:23.785768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:23.785768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:41:23.785768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:41:23.785768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:41:23.785768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:41:23.785768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:41:23.785768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:41:23.785768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:41:23.785768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:41:23.785768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:23.785768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:41:23.785993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:23.785993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:41:23.785993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:23.785993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:41:23.785993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:23.785993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:41:23.785993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:23.785993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:41:23.785993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:41:23.785993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:23.785993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:41:23.785993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:23.785993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:41:23.785993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:41:23.786239 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:23.786239 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:41:23.786239 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:23.786239 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:41:23.786928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:23.786928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:41:23.786928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:41:23.786928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:41:23.786928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:41:23.786928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:41:23.786928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:41:23.786928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:41:23.786928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:41:23.786928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:41:23.786928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:41:23.786928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:41:23.786928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:41:23.786928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:41:23.786928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:41:23.786928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:41:23.786928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:41:23.787252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:41:23.787252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:41:23.787252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:41:23.787252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:23.787252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:41:23.787252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:23.787252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:41:23.787252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:23.787252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:41:23.787252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:23.787252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:41:23.787252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:23.787252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:41:23.787252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:41:23.787252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:23.787252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:41:23.787252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:23.787252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:41:23.787252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:41:23.787609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:23.787609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:41:23.787609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:23.787609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:41:23.789060 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:41:23.789325 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:41:23.789439 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:41:23.789628 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:41:23.790248 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:41:23.790554 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:41:23.790854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:23.791526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:41:23.791526 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:23.791526 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:41:23.791526 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:41:23.791526 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:23.791526 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:41:23.791526 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:23.791526 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:41:23.792691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:23.792691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:41:23.792691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:41:23.792691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:41:23.792691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:41:23.792691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:41:23.792691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:41:23.792691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:23.792691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:41:23.792691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:23.792691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:41:23.792691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:23.793679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:41:23.793679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:23.793679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:41:23.793679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:23.793679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:41:23.793679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:41:23.793679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:23.793679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:41:23.793679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:23.793679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:41:23.793679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:41:23.793679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:23.793679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:41:23.794555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:23.794555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:41:23.794555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:23.794555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:41:23.794555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:41:23.794555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:41:23.794555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:41:23.794555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:41:23.794555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:41:23.794555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:41:23.794555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:41:23.794555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:41:23.794555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:41:23.794555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:41:23.794555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:41:23.795275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:41:23.795275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:41:23.795275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:41:23.795275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:41:23.795275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:41:23.795275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:23.795275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:41:23.795275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:23.795275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:41:23.795275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:23.795275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:41:23.795275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:23.795275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:41:23.795275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:23.795275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:41:23.795275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:41:23.795275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:23.795275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:41:23.795275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:23.796641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:41:23.796641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:41:23.796641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:23.796641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:41:23.796641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:23.796641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:41:28.616401 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:41:28.617162 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:41:28.617473 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:41:28.618931 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:41:28.659780 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:41:28.659956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:28.660453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:41:28.660453 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:28.660453 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:41:28.660453 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:41:28.660453 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:28.660453 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:41:28.660453 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:28.660453 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:41:28.661371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.043 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:28.661371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:41:28.661371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:41:28.661371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:41:28.661371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:41:28.661371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:41:28.661371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:41:28.661371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:28.661371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:41:28.661371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:28.661371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:41:28.661371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:28.661802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:41:28.661802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:28.661802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:41:28.661802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:28.661802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:41:28.661802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:41:28.661802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:28.661802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:41:28.661802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:28.661802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:41:28.661802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:41:28.661802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:28.661802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:41:28.662269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:28.662269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:41:28.662518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:28.662518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:41:28.662518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:41:28.662518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:41:28.662518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:41:28.662518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:41:28.662518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:41:28.662518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:41:28.662518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:41:28.662518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:41:28.662518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:41:28.662518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:41:28.662518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:41:28.662518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:41:28.662518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:41:28.662518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:41:28.663125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:41:28.663125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:41:28.663125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:28.663125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:41:28.663125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:28.663125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:41:28.663125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:28.663125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:41:28.663125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:28.663125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:41:28.663125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:28.663125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:41:28.663125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:41:28.663125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:28.663125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:41:28.663125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:28.663125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:41:28.663125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:41:28.663125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:28.663694 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:41:28.663694 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:28.663694 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:41:28.770688 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:41:28.770688 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:41:28.770688 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:41:28.770688 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:41:28.771134 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:41:28.771134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:28.771578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:41:28.771578 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:28.771578 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:41:28.771578 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:41:28.771578 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:28.771578 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:41:28.771578 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:28.771578 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:41:28.772831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:28.772831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:41:28.772831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:41:28.772831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:41:28.772831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:41:28.772831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:41:28.772831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:41:28.772831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:28.772831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:41:28.772831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:28.772831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:41:28.772831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:28.773121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:41:28.773121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:28.773121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:41:28.773121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:28.773121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:41:28.773121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:41:28.773121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:28.773121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:41:28.773121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:28.773121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:41:28.773121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:41:28.773121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:28.773121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:41:28.773369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:28.773369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:41:28.774019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:28.774019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:41:28.774019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:41:28.774019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:41:28.774019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:41:28.774019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:41:28.774019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:41:28.774019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:41:28.774019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:41:28.774019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:41:28.774019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:41:28.774019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:41:28.774019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:41:28.774019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:41:28.774019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:41:28.774019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:41:28.774402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:41:28.774402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:41:28.774402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:28.774402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:41:28.774402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:28.774402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:41:28.774402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:28.774402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:41:28.774402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:28.774402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:41:28.774402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:28.774402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:41:28.774402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:41:28.774402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:28.774402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:41:28.774402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:28.774402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:41:28.774402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:41:28.774402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:28.774927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:41:28.774927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:28.774927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:41:28.777580 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:41:28.778019 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:41:28.778197 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:41:28.778381 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:41:28.778878 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:41:28.779074 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:28.779700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:41:28.779700 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:28.779700 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:41:28.779700 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:41:28.779700 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:28.779700 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:41:28.779700 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:28.779700 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:41:28.780653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:28.780653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:41:28.780653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:41:28.780653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:41:28.780653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:28.780653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:41:28.780653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:28.780653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:41:28.780653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:28.780653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:41:28.780653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:28.780653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:41:28.780653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:28.780653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:41:28.780653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:41:28.781103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:28.781103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:41:28.781103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:28.781103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:41:28.781103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:41:28.781103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:28.781103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:41:28.781103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:28.781103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:41:28.781938 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:28.781938 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:41:28.781938 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:41:28.781938 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:41:28.781938 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:41:28.781938 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:41:28.781938 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:41:28.781938 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:41:28.781938 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:41:28.781938 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:41:28.781938 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:28.781938 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:41:28.782913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:28.782913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:41:28.782913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:28.782913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:41:28.782913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:28.782913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:41:28.782913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:28.782913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:41:28.782913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:41:28.782913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:28.782913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:41:28.782913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:28.782913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:41:28.782913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:41:28.783260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:28.783260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:41:28.783260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:28.783260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:41:28.783260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:28.783260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:41:28.783260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:41:28.783260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:41:28.783260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:41:28.783260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:41:28.783260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:41:28.783260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:41:28.783260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:41:28.783260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:41:28.783631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:41:28.783631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:41:28.783631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:41:28.783631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:41:28.783631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:41:28.783631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:41:28.783631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:41:28.783631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:41:28.783631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:41:28.783631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:41:28.783631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:41:28.783631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:28.783631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:41:28.783631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:28.783631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:41:28.783631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:28.783631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:41:28.783631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:28.783631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:41:28.784149 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:28.784149 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:41:28.784149 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:41:28.784149 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:28.784149 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:41:28.784149 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:28.784149 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:41:28.784149 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:41:28.784149 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:28.784149 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:41:28.784149 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:28.784149 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:41:28.784149 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:41:28.784149 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:41:28.784522 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:41:28.784522 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:41:28.784522 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:41:28.784625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:28.785718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:41:28.785718 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:28.785718 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:41:28.785718 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:41:28.785718 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:28.785718 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:41:28.785718 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:28.785718 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:41:28.786346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:28.786346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:41:28.786346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:41:28.786346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:41:28.786346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:28.786346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:41:28.786346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:28.786346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:41:28.786346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:28.786346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:41:28.786346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:28.786346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:41:28.786346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:28.786346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:41:28.786346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:41:28.787318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:28.787318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:41:28.787318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:28.787318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:41:28.787318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:41:28.787318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:28.787318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:41:28.787318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:28.787318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:41:28.787788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:28.787788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:41:28.787788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:41:28.787788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:41:28.787788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:41:28.787788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:41:28.787788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:41:28.787788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:41:28.787788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:41:28.787788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:28.787788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:41:28.788281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:28.788281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:41:28.788281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:28.788281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:41:28.788281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:28.788281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:41:28.788281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:28.788281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:41:28.788281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:41:28.788281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:28.788281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:41:28.788281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:28.788281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:41:28.788281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:41:28.788939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:28.788939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:41:28.788939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:28.788939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:41:28.789046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:28.789046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:41:28.789046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:41:28.789046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:41:28.789046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:41:28.789046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:41:28.789046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:41:28.789046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:41:28.789046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:41:28.789046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:41:28.789046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:41:28.789046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:41:28.789046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:41:28.789046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:41:28.789046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:41:28.789046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:41:28.789046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:41:28.789473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:41:28.789473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:41:28.789473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:41:28.789473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:28.789473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:41:28.789473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:28.789473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:41:28.789473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:28.789473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:41:28.789473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:28.789473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:41:28.789473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:28.789473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:41:28.789473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:41:28.789473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:28.789473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:41:28.789473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:28.789473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:41:28.789473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:41:28.789951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:28.789951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:41:28.789951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:28.789951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:41:28.789951 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:41:28.789951 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:41:28.789951 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:41:28.790209 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:41:28.792021 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:41:28.792243 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:41:28.792350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:28.792711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:41:28.792711 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:28.792711 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:41:28.792711 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:41:28.792711 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:28.792711 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:41:28.792711 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:28.792711 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:41:28.793490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:28.793490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:41:28.793490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:41:28.793490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:41:28.793490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:41:28.793490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:41:28.793490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:41:28.793490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:28.793490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:41:28.793490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:28.793490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:41:28.793490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:28.793772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:41:28.793772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:28.793772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:41:28.793772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:28.793772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:41:28.793772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:41:28.793772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:28.793772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:41:28.793772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:28.793772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:41:28.793772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:41:28.793772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:28.793772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:41:28.794018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:28.794018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:41:28.794474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:28.794474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:41:28.794474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:41:28.794474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:41:28.794474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:41:28.794474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:41:28.794474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:41:28.794474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:41:28.794474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:41:28.794474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:41:28.794474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:41:28.794474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:41:28.794474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:41:28.794474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:41:28.794474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:41:28.794474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:41:28.794856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:41:28.794856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:41:28.794856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:28.794856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:41:28.794856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:28.794856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:41:28.794856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:28.794856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:41:28.794856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:28.794856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:41:28.794856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:28.794856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:41:28.794856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:41:28.794856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:28.794856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:41:28.794856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:28.794856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:41:28.794856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:41:28.794856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:28.795210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:41:28.795210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:28.795210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:41:33.618444 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:41:33.619022 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:41:33.619230 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:41:33.619818 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:41:33.656398 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:41:33.656398 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:33.657178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:41:33.657178 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:33.657178 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:41:33.657178 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:41:33.657178 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:33.657178 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:41:33.657178 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:33.657178 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:41:33.658078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:33.658078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:41:33.658078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:41:33.658078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:41:33.658078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:41:33.658078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:41:33.658078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:41:33.658078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:33.658078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:41:33.658078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:33.658078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:41:33.658078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:33.658477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:41:33.658477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:33.658477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:41:33.658477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:33.658477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:41:33.658477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:41:33.658477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:33.658477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:41:33.658477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:33.658477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:41:33.658477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:41:33.658477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:33.658477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:41:33.659281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:33.659281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:41:33.659281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:33.659281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:41:33.659281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:41:33.659281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:41:33.659281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:41:33.659281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:41:33.659281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:41:33.659281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:41:33.659281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:41:33.659281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:41:33.659281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:41:33.659281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:41:33.659281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:41:33.660237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:41:33.660237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:41:33.660237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:41:33.660237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:41:33.660237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:41:33.660237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:33.660237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:41:33.660237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:33.660237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:41:33.660237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:33.660237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:41:33.660237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:33.660237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:41:33.660237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:33.660237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:41:33.660237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:41:33.660237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:33.660237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:41:33.660237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:33.661159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:41:33.661159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:41:33.661159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:33.661159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:41:33.661159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:33.661159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:41:33.769682 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:41:33.770290 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:41:33.770586 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:41:33.771880 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:41:33.772224 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:41:33.772551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:33.773656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:41:33.773656 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:33.773656 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:41:33.773656 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:41:33.773656 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:33.773656 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:41:33.773656 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:33.773656 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:41:33.775354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:33.775354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:41:33.775354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:41:33.775354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:41:33.775354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:41:33.775354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:41:33.775354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:41:33.775354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:33.775354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:41:33.775354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:33.775354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:41:33.775354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:33.776060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:41:33.776060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:33.776060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:41:33.776060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:33.776060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:41:33.776060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:41:33.776060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:33.776060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:41:33.776060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:33.776060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:41:33.776060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:41:33.776060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:33.776060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:41:33.776797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:33.776797 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:41:33.777192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:33.777192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:41:33.777192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:41:33.777192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:41:33.777192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:41:33.777192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:41:33.777192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:41:33.777192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:41:33.777192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:41:33.777192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:41:33.777192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:41:33.777192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:41:33.777192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:41:33.777192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:41:33.777192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:41:33.777192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:41:33.777640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:41:33.777640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:41:33.777640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:33.777640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:41:33.777640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:33.777640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:41:33.777640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:33.777640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:41:33.777640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:33.777640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:41:33.777640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:33.777640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:41:33.777640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:41:33.777640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:33.777640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:41:33.777640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:33.777640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:41:33.777640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:41:33.777640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:33.778170 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:41:33.778170 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:33.778170 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:41:33.778170 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:41:33.778366 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:41:33.778481 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:41:33.778621 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:41:33.779173 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:41:33.779503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:33.780263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:41:33.780263 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:33.780263 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:41:33.780263 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:41:33.780263 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:33.780263 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:41:33.780263 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:33.780263 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:41:33.781116 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:33.781116 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:41:33.781116 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:41:33.781116 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:41:33.781116 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:33.781116 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:41:33.781116 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:33.781116 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:41:33.781116 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:33.781116 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:41:33.781116 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:33.781116 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:41:33.781116 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:33.781116 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:41:33.781116 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:41:33.781609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:33.781609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:41:33.781609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:33.781609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:41:33.781609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:41:33.781609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:33.781609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:41:33.781609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:33.781609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:41:33.782032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:33.782032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:41:33.782032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:41:33.782032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:41:33.782032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:41:33.782032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:41:33.782032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:41:33.782032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:41:33.782032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:41:33.782032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:41:33.782032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:33.782032 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:41:33.782509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:33.782509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:41:33.782509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:33.782509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:41:33.782509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:33.782509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:41:33.782509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:33.782509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:41:33.782509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:41:33.782509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:33.782509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:41:33.782509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:33.782509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:41:33.782509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:41:33.782912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:33.782912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:41:33.782912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:33.782912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:41:33.783052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:33.783052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:41:33.783052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:41:33.783052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:41:33.783052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:41:33.783052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:41:33.783052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:41:33.783052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:41:33.783052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:41:33.783052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:41:33.783052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:41:33.783052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:41:33.783052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:41:33.783052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:41:33.783052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:41:33.783052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:41:33.783491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:41:33.783491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:41:33.783491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:41:33.783491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:41:33.783491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:41:33.783491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:33.783491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:41:33.783491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:33.783491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:41:33.783491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:33.783491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:41:33.783491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:33.783491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:41:33.783491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:33.783491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:41:33.783491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:41:33.783491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:33.783491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:41:33.784261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:33.784261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:41:33.784261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:41:33.784261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:33.784261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:41:33.784261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:33.784261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:41:33.784261 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:41:33.784261 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:41:33.784903 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:41:33.784903 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:41:33.784989 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:41:33.785228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:33.785727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:41:33.785727 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:33.785727 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:41:33.785727 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:41:33.785727 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:33.785727 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:41:33.785727 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:33.785727 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:41:33.786469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:33.786469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:41:33.786469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:41:33.786469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:41:33.786469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:33.786469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:41:33.786469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:33.786469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:41:33.786469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:33.786469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:41:33.786469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:33.786469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:41:33.786469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:33.786469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:41:33.786469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:41:33.786887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:33.786887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:41:33.786887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:33.786887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:41:33.786887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:41:33.786887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:33.786887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:41:33.786887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:33.786887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:41:33.787341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:33.787341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:41:33.787341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:41:33.787341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:41:33.787341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:41:33.787341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:41:33.787341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:41:33.787341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:41:33.787341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:41:33.787341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:33.787341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:41:33.787647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:33.787647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:41:33.787647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:33.787647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:41:33.787647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:33.787647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:41:33.787647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:33.787647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:41:33.787647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:41:33.787647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:33.787647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:41:33.787647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:33.787647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:41:33.787647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:41:33.788178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:33.788178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:41:33.788178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:33.788178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:41:33.788519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:33.788519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:41:33.788519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:41:33.788519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:41:33.788519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:41:33.788519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:41:33.788519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:41:33.788519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:41:33.788519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:41:33.788519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:41:33.788519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:41:33.788519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:41:33.788519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:41:33.788519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:41:33.788519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:41:33.788519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:41:33.788519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:41:33.789098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:41:33.789098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:41:33.789098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:41:33.789098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:33.789098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:41:33.789098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:33.789098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:41:33.789098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:33.789098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:41:33.789098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:33.789098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:41:33.789098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:33.789098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:41:33.789098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:41:33.789098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:33.789098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:41:33.789098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:33.789098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:41:33.789098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:41:33.789620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:33.789620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:41:33.789620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:33.789620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:41:33.789836 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:41:33.790181 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:41:33.790323 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:41:33.790406 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:41:33.790971 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:41:33.791488 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:41:33.791939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:33.792916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:41:33.792916 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:33.792916 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:41:33.792916 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:41:33.792916 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:33.792916 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:41:33.792916 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:33.792916 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:41:33.794010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:33.794010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:41:33.794010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:41:33.794010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:41:33.794010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:41:33.794010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:41:33.794010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:41:33.794010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:33.794010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:41:33.794010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:33.794010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:41:33.794010 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:33.795016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:41:33.795016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:33.795016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:41:33.795016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:33.795016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:41:33.795016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:41:33.795016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:33.795016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:41:33.795016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:33.795016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:41:33.795016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:41:33.795016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:33.795016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:41:33.795773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:33.795773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:41:33.795773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:33.795773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:41:33.795773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:41:33.795773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:41:33.795773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:41:33.795773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:41:33.795773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:41:33.795773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:41:33.795773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:41:33.795773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:41:33.795773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:41:33.795773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:41:33.795773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:41:33.796567 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:41:33.796567 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:41:33.796567 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:41:33.796567 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:41:33.796567 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:41:33.796567 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:33.796567 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:41:33.796567 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:33.796567 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:41:33.796567 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:33.796567 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:41:33.796567 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:33.796567 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:41:33.796567 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:33.796567 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:41:33.796567 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:41:33.796567 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:33.796567 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:41:33.796567 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:33.797735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:41:33.797735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:41:33.797735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:33.797735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:41:33.797735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:33.797735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:41:38.619373 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:41:38.620997 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:41:38.621063 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:41:38.622189 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:41:38.664140 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:41:38.664457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:38.665915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:41:38.665915 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:38.665915 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:41:38.665915 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:41:38.665915 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:38.665915 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:41:38.665915 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:38.665915 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:41:38.666289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.044 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:38.666289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:41:38.666289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:41:38.666289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:41:38.666289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:41:38.666289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:41:38.666289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:41:38.666289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:38.666289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:41:38.666289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:38.666289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:41:38.666289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:38.666660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:41:38.666660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:38.666660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:41:38.666660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:38.666660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:41:38.666660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:41:38.666660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:38.666660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:41:38.666660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:38.666660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:41:38.666660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:41:38.666660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:38.666660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:41:38.667072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:38.667072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:41:38.668848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:38.668848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:41:38.668848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:41:38.668848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:41:38.668848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:41:38.668848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:41:38.668848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:41:38.668848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:41:38.668848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:41:38.668848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:41:38.668848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:41:38.668848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:41:38.668848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:41:38.668848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:41:38.668848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:41:38.668848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:41:38.669226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:41:38.669226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:41:38.669226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:38.669226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:41:38.669226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:38.669226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:41:38.669226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:38.669226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:41:38.669226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:38.669226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:41:38.669226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:38.669226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:41:38.669226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:41:38.669226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:38.669226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:41:38.669226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:38.669226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:41:38.669226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:41:38.669226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:38.669674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:41:38.669674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:38.669674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:41:38.770802 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:41:38.771534 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:41:38.771659 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:41:38.772874 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:41:38.773232 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:41:38.773483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:38.774474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:41:38.774474 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:38.774474 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:41:38.774474 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:41:38.774474 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:38.774474 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:41:38.774474 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:38.774474 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:41:38.776097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:38.776097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:41:38.776097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:41:38.776097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:41:38.776097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:41:38.776097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:41:38.776097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:41:38.776097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:38.776097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:41:38.776097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:38.776097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:41:38.776097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:38.776501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:41:38.776501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:38.776501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:41:38.776501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:38.776501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:41:38.776501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:41:38.776501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:38.776501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:41:38.776501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:38.776501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:41:38.776501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:41:38.776501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:38.776501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:41:38.776923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:38.776923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:41:38.777400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:38.777400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:41:38.777400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:41:38.777400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:41:38.777400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:41:38.777400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:41:38.777400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:41:38.777400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:41:38.777400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:41:38.777400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:41:38.777400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:41:38.777400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:41:38.777400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:41:38.777400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:41:38.777400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:41:38.777400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:41:38.777921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:41:38.777921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:41:38.777921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:38.777921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:41:38.777921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:38.777921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:41:38.777921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:38.777921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:41:38.777921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:38.777921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:41:38.777921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:38.777921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:41:38.777921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:41:38.777921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:38.777921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:41:38.777921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:38.777921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:41:38.777921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:41:38.777921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:38.778492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:41:38.778492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:38.778492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:41:38.778590 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:41:38.778892 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:41:38.779015 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:41:38.779141 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:41:38.779838 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:41:38.780139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:38.780813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:41:38.780813 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:38.780813 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:41:38.780813 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:41:38.780813 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:38.780813 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:41:38.780813 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:38.780813 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:41:38.781767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:38.781767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:41:38.781767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:41:38.781767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:41:38.781767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:38.781767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:41:38.781767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:38.781767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:41:38.781767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:38.781767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:41:38.781767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:38.781767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:41:38.781767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:38.781767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:41:38.781767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:41:38.782359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:38.782359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:41:38.782359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:38.782359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:41:38.782359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:41:38.782359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:38.782359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:41:38.782359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:38.782359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:41:38.783017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:38.783017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:41:38.783017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:41:38.783017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:41:38.783017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:41:38.783017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:41:38.783017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:41:38.783017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:41:38.783017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:41:38.783017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:41:38.783017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:38.783017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:41:38.783538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:38.783538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:41:38.783538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:38.783538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:41:38.783538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:38.783538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:41:38.783538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:38.783538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:41:38.783538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:41:38.783538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:38.783538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:41:38.783538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:38.783538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:41:38.783538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:41:38.784300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:38.784300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:41:38.784300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:38.784300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:41:38.784610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:38.784610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:41:38.784610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:41:38.784610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:41:38.784610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:41:38.784610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:41:38.784610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:41:38.784610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:41:38.784610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:41:38.784610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:41:38.784610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:41:38.784610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:41:38.784610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:41:38.784610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:41:38.784610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:41:38.784610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:41:38.785360 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:41:38.785360 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:41:38.785360 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:41:38.785360 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:41:38.785360 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:41:38.785360 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:38.785360 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:41:38.785360 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:38.785360 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:41:38.785360 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:38.785360 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:41:38.785360 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:38.785360 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:41:38.785360 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:38.785360 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:41:38.785360 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:41:38.785360 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:38.785360 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:41:38.786613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:38.786613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:41:38.786613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:41:38.786613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:38.786613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:41:38.786613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:38.786613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:41:38.786613 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:41:38.786613 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:41:38.786613 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:41:38.787086 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:41:38.787086 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:41:38.787086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:38.787320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:41:38.787320 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:38.787320 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:41:38.787320 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:41:38.787320 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:38.787320 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:41:38.787320 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:38.787320 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:41:38.788302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:38.788302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:41:38.788302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:41:38.788302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:41:38.788302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:38.788302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:41:38.788302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:38.788302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:41:38.788302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:38.788302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:41:38.788302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:38.788302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:41:38.788302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:38.788302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:41:38.788302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:41:38.788777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:38.788777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:41:38.788777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:38.788777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:41:38.788777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:41:38.788777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:38.788777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:41:38.788777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:38.788777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:41:38.789499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:38.789499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:41:38.789499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:41:38.789499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:41:38.789499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:41:38.789499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:41:38.789499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:41:38.789499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:41:38.789499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:41:38.789499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:38.789499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:41:38.789857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:38.789857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:41:38.789857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:38.789857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:41:38.789857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:38.789857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:41:38.789857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:38.789857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:41:38.789857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:41:38.789857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:38.789857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:41:38.789857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:38.789857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:41:38.789857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:41:38.790257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:38.790257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:41:38.790257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:38.790257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:41:38.790517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:38.790517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:41:38.790517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:41:38.790517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:41:38.790517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:41:38.790517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:41:38.790517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:41:38.790517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:41:38.790517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:41:38.790517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:41:38.790517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:41:38.790517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:41:38.790517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:41:38.790517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:41:38.790517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:41:38.790517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:41:38.790517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:41:38.791009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:41:38.791009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:41:38.791009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:41:38.791009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:38.791009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:41:38.791009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:38.791009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:41:38.791009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:38.791009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:41:38.791009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:38.791009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:41:38.791009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:38.791009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:41:38.791009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:41:38.791009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:38.791009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:41:38.791009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:38.791009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:41:38.791009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:41:38.791539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:38.791539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:41:38.791539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:38.791539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:41:38.791539 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:41:38.791539 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:41:38.791539 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:41:38.791539 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:41:38.791831 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:41:38.791929 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:41:38.792147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:38.792657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:41:38.792657 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:38.792657 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:41:38.792657 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:41:38.792657 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:38.792657 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:41:38.792657 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:38.792657 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:41:38.793364 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:38.793364 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:41:38.793364 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:41:38.793364 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:41:38.793364 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:41:38.793364 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:41:38.793364 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:41:38.793364 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:38.793364 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:41:38.793364 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:38.793364 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:41:38.793364 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:38.793681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:41:38.793681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:38.793681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:41:38.793681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:38.793681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:41:38.793681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:41:38.793681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:38.793681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:41:38.793681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:38.793681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:41:38.793681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:41:38.793681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:38.793681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:41:38.794069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:38.794069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:41:38.794330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:38.794330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:41:38.794330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:41:38.794330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:41:38.794330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:41:38.794330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:41:38.794330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:41:38.794330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:41:38.794330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:41:38.794330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:41:38.794330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:41:38.794330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:41:38.794330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:41:38.794330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:41:38.794330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:41:38.794330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:41:38.794792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:41:38.794792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:41:38.794792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:38.794792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:41:38.794792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:38.794792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:41:38.794792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:38.794792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:41:38.794792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:38.794792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:41:38.794792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:38.794792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:41:38.794792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:41:38.794792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:38.794792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:41:38.794792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:38.794792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:41:38.794792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:41:38.794792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:38.795394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:41:38.795394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:38.795394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:41:43.620388 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:41:43.621238 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:41:43.621482 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:41:43.622865 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:41:43.662735 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:41:43.663070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:43.663909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:41:43.663909 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:43.663909 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:41:43.663909 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:41:43.663909 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:43.663909 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:41:43.663909 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:43.663909 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:41:43.665003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:43.665003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:41:43.665003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:41:43.665003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:41:43.665003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:41:43.665003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:41:43.665003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:41:43.665003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:43.665003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:41:43.665003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:43.665003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:41:43.665003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:43.666072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:41:43.666072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:43.666072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:41:43.666072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:43.666072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:41:43.666072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:41:43.666072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:43.666072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:41:43.666072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:43.666072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:41:43.666072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:41:43.666072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:43.666072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:41:43.666722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:43.666722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:41:43.666916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:43.666916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:41:43.666916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:41:43.666916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:41:43.666916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:41:43.666916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:41:43.666916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:41:43.666916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:41:43.666916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:41:43.666916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:41:43.666916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:41:43.666916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:41:43.666916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:41:43.666916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:41:43.666916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:41:43.666916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:41:43.667738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:41:43.667738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:41:43.667738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:43.667738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:41:43.667738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:43.667738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:41:43.667738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:43.667738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:41:43.667738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:43.667738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:41:43.667738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:43.667738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:41:43.667738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:41:43.667738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:43.667738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:41:43.667738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:43.667738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:41:43.667738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:41:43.667738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:43.668729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:41:43.668729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:43.668729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:41:43.771469 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:41:43.772320 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:41:43.772554 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:41:43.773592 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:41:43.773987 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:41:43.774099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:43.774851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:41:43.774851 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:43.774851 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:41:43.774851 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:41:43.774851 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:43.774851 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:41:43.774851 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:43.774851 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:41:43.776426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:43.776426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:41:43.776426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:41:43.776426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:41:43.776426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:41:43.776426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:41:43.776426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:41:43.776426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:43.776426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:41:43.776426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:43.776426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:41:43.776426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:43.777197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:41:43.777197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:43.777197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:41:43.777197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:43.777197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:41:43.777197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:41:43.777197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:43.777197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:41:43.777197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:43.777197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:41:43.777197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:41:43.777197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:43.777197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:41:43.777767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:43.777767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:41:43.778547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:43.778547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:41:43.778547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:41:43.778547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:41:43.778547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:41:43.778547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:41:43.778547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:41:43.778547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:41:43.778547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:41:43.778547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:41:43.778547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:41:43.778547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:41:43.778547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:41:43.778547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:41:43.778547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:41:43.778547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:41:43.779390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:41:43.779390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:41:43.779390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:43.779390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:41:43.779390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:43.779390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:41:43.779390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:43.779390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:41:43.779390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:43.779390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:41:43.779390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:43.779390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:41:43.779390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:41:43.779390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:43.779390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:41:43.779390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:43.779390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:41:43.779390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:41:43.779390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:43.780348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:41:43.780348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:43.780348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:41:43.780348 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:41:43.780348 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:41:43.780348 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:41:43.780348 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:41:43.780348 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:41:43.781255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:43.781255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:41:43.781255 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:43.781255 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:41:43.781255 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:41:43.781255 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:43.781255 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:41:43.781255 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:43.781255 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:41:43.782033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:43.782033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:41:43.782033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:41:43.782033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:41:43.782033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:43.782033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:41:43.782033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:43.782033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:41:43.782033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:43.782033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:41:43.782033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:43.782033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:41:43.782033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:43.782033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:41:43.782033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:41:43.782904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:43.782904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:41:43.782904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:43.782904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:41:43.782904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:41:43.782904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:43.782904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:41:43.782904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:43.782904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:41:43.783265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:43.783265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:41:43.783265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:41:43.783265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:41:43.783265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:41:43.783265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:41:43.783265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:41:43.783265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:41:43.783265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:41:43.783265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:41:43.783265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:43.783265 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:41:43.783774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:43.783774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:41:43.783774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:43.783774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:41:43.783774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:43.783774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:41:43.783774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:43.783774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:41:43.783774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:41:43.783774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:43.783774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:41:43.783774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:43.783774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:41:43.783774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:41:43.784314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:43.784314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:41:43.784314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:43.784314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:41:43.784314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:43.784314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:41:43.784314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:41:43.784314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:41:43.784314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:41:43.784314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:41:43.784314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:41:43.784314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:41:43.784314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:41:43.784314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:41:43.784716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:41:43.784716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:41:43.784716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:41:43.784716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:41:43.784716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:41:43.784716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:41:43.784716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:41:43.784716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:41:43.784716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:41:43.784716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:41:43.784716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:41:43.784716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:43.784716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:41:43.784716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:43.784716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:41:43.784716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:43.784716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:41:43.784716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:43.784716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:41:43.785337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:43.785337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:41:43.785337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:41:43.785337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:43.785337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:41:43.785337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:43.785337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:41:43.785337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:41:43.785337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:43.785337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:41:43.785337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:43.785337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:41:43.785639 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:41:43.785684 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:41:43.786120 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:41:43.786120 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:41:43.786247 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:41:43.789781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:43.789781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:41:43.789781 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:43.789781 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:41:43.789781 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:41:43.789781 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:43.789781 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:41:43.789781 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:43.789781 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:41:43.789781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:43.789781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:41:43.789781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:41:43.789781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:41:43.789781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:43.789781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:41:43.789781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:43.789781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:41:43.790330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:43.790330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:41:43.790330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:43.790330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:41:43.790330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:43.790330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:41:43.790330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:41:43.790330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:43.790330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:41:43.790330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:43.790330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:41:43.790330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:41:43.790330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:43.790330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:41:43.790330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:43.790330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:41:43.790798 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:43.790798 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:41:43.790798 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:41:43.790798 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:41:43.790798 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:41:43.790798 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:41:43.790798 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:41:43.790798 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:41:43.790798 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:41:43.790798 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:43.790798 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:41:43.791204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:43.791204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:41:43.791204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:43.791204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:41:43.791204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:43.791204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:41:43.791204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:43.791204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:41:43.791204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:41:43.791204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:43.791204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:41:43.791204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:43.791204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:41:43.791204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:41:43.791590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:43.791590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:41:43.791590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:43.791590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:41:43.791590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:43.791590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:41:43.791590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:41:43.791590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:41:43.791590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:41:43.791590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:41:43.791590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:41:43.791590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:41:43.791590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:41:43.791590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:41:43.792020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:41:43.792020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:41:43.792020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:41:43.792020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:41:43.792020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:41:43.792020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:41:43.792020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:41:43.792020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:41:43.792020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:41:43.792020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:41:43.792020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:43.792020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:41:43.792020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:43.792020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:41:43.792020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:43.792020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:41:43.792020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:43.792020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:41:43.792020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:43.792563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:41:43.792563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:41:43.792563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:43.792563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:41:43.792563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:43.792563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:41:43.792563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:41:43.792563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:43.792563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:41:43.792563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:43.792563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:41:43.792563 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:41:43.792563 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:41:43.793027 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:41:43.793027 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:41:43.793027 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:41:43.793027 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:41:43.793027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:43.793334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:41:43.793334 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:43.793334 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:41:43.793334 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:41:43.793334 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:43.793334 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:41:43.793334 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:43.793334 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:41:43.794408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:43.794408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:41:43.794408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:41:43.794408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:41:43.794408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:41:43.794408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:41:43.794408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:41:43.794408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:43.794408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:41:43.794408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:43.794408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:41:43.794408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:43.794706 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:41:43.794706 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:43.794706 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:41:43.794706 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:43.794706 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:41:43.794706 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:41:43.794706 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:43.794706 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:41:43.794706 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:43.794706 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:41:43.794706 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:41:43.794706 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:43.794706 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:41:43.795184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:43.795184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:41:43.795888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:43.795888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:41:43.795888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:41:43.795888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:41:43.795888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:41:43.795888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:41:43.795888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:41:43.795888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:41:43.795888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:41:43.795888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:41:43.795888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:41:43.795888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:41:43.795888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:41:43.795888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:41:43.795888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:41:43.795888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:41:43.796294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:41:43.796294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:41:43.796294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:43.796294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:41:43.796294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:43.796294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:41:43.796294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:43.796294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:41:43.796294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:43.796294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:41:43.796294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:43.796294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:41:43.796294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:41:43.796294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:43.796294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:41:43.796294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:43.796294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:41:43.796294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:41:43.796294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:43.796733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:41:43.796733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:43.796733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:41:48.624060 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:41:48.624965 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:41:48.624965 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:41:48.625419 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:41:48.654923 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:41:48.654923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:48.655337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:41:48.655337 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:48.655337 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:41:48.655337 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:41:48.655337 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:48.655337 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:41:48.655337 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:48.655337 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:41:48.657504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.031 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:48.657504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:41:48.657504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:41:48.657504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:41:48.657504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:41:48.657504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:41:48.657504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:41:48.657504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:48.657504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:41:48.657504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:48.657504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:41:48.657504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:48.657910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:41:48.657910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:48.657910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:41:48.657910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:48.657910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:41:48.657910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:41:48.657910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:48.657910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:41:48.657910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:48.657910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:41:48.657910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:41:48.657910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:48.657910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:41:48.658181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:48.658181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:41:48.659899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:48.659899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:41:48.659899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:41:48.659899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:41:48.659899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:41:48.659899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:41:48.659899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:41:48.659899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:41:48.659899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:41:48.659899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:41:48.659899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:41:48.659899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:41:48.659899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:41:48.659899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:41:48.659899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:41:48.659899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:41:48.660269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:41:48.660269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:41:48.660269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:48.660269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:41:48.660269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:48.660269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:41:48.660269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:48.660269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:41:48.660269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:48.660269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:41:48.660269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:48.660269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:41:48.660269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:41:48.660269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:48.660269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:41:48.660269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:48.660269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:41:48.660269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:41:48.660269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:48.660679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:41:48.660679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:48.660679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:41:48.772153 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:41:48.772635 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:41:48.772936 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:41:48.774386 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:41:48.774914 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:41:48.775189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:48.776180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:41:48.776180 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:48.776180 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:41:48.776180 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:41:48.776180 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:48.776180 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:41:48.776180 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:48.776180 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:41:48.777930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:48.777930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:41:48.777930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:41:48.777930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:41:48.777930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:41:48.777930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:41:48.777930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:41:48.777930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:48.777930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:41:48.777930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:48.777930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:41:48.777930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:48.778389 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:41:48.778389 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:48.778389 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:41:48.778389 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:48.778389 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:41:48.778389 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:41:48.778389 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:48.778389 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:41:48.778389 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:48.778389 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:41:48.778389 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:41:48.778389 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:48.778389 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:41:48.778852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:48.778852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:41:48.779644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:48.779644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:41:48.779644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:41:48.779644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:41:48.779644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:41:48.779644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:41:48.779644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:41:48.779644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:41:48.779644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:41:48.779644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:41:48.779644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:41:48.779644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:41:48.779644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:41:48.779644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:41:48.779644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:41:48.779644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:41:48.780231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:41:48.780231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:41:48.780231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:48.780231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:41:48.780231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:48.780231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:41:48.780231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:48.780231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:41:48.780231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:48.780231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:41:48.780231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:48.780231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:41:48.780231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:41:48.780231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:48.780231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:41:48.780231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:48.780231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:41:48.780231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:41:48.780231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:48.780881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:41:48.780881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:48.780881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:41:48.780881 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:41:48.780881 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:41:48.780881 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:41:48.780881 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:41:48.781456 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:41:48.781783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:48.782936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:41:48.782936 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:48.782936 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:41:48.782936 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:41:48.782936 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:48.782936 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:41:48.782936 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:48.782936 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:41:48.784832 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:48.784832 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:41:48.784832 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:41:48.784832 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:41:48.784832 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:48.784832 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:41:48.784832 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:48.784832 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:41:48.784832 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:48.784832 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:41:48.784832 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:48.784832 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:41:48.784832 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:48.784832 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:41:48.784832 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:41:48.785551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:48.785551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:41:48.785551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:48.785551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:41:48.785551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:41:48.785551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:48.785551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:41:48.785551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:48.785551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:41:48.785999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:48.785999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:41:48.785999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:41:48.785999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:41:48.785999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:41:48.785999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:41:48.785999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:41:48.785999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:41:48.785999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:41:48.785999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:41:48.785999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:48.785999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:41:48.786730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:48.786730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:41:48.786730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:48.786730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:41:48.786730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:48.786730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:41:48.786730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:48.786730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:41:48.786730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:41:48.786730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:48.786730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:41:48.786730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:48.786730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:41:48.786730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:41:48.787114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:48.787114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:41:48.787114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:48.787114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:41:48.787114 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:41:48.787436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:48.787436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:41:48.787436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:41:48.787436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:41:48.787436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:41:48.787436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:41:48.787436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:41:48.787436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:41:48.787436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:41:48.787436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:41:48.787436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:41:48.787436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:41:48.787436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:41:48.787436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:41:48.787436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:41:48.787436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:41:48.787892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:41:48.787892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:41:48.787892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:41:48.787892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:41:48.787892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:41:48.787892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:48.787892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:41:48.787892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:48.787892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:41:48.787892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:48.787892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:41:48.787892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:48.787892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:41:48.787892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:48.787892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:41:48.787892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:41:48.787892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:48.787892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:41:48.788640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:48.788640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:41:48.788640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:41:48.788640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:48.788640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:41:48.788640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:48.788640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:41:48.788640 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:41:48.788640 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:41:48.788640 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:41:48.789343 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:41:48.789343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:48.790792 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:41:48.790792 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:48.790792 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:41:48.790792 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:41:48.790792 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:48.790792 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:41:48.790792 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:48.790792 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:41:48.791664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:48.791664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:41:48.791664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:41:48.791664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:41:48.791664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:48.791664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:41:48.791664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:48.791664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:41:48.791664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:48.791664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:41:48.791664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:48.791664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:41:48.791664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:48.791664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:41:48.791664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:41:48.792021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:48.792021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:41:48.792021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:48.792021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:41:48.792021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:41:48.792021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:48.792021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:41:48.792021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:48.792021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:41:48.793070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:48.793070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:41:48.793070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:41:48.793070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:41:48.793070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:41:48.793070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:41:48.793070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:41:48.793070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:41:48.793070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:41:48.793070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:48.793070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:41:48.793868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:48.793868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:41:48.793868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:48.793868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:41:48.793868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:48.793868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:41:48.793868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:48.793868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:41:48.793868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:41:48.793868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:48.793868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:41:48.793868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:48.793868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:41:48.793868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:41:48.794972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:48.794972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:41:48.794972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:48.794972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:41:48.794972 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:41:48.794972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:48.794972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:41:48.794972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:41:48.794972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:41:48.794972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:41:48.794972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:41:48.794972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:41:48.796247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:41:48.796247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:41:48.796247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:41:48.796247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:41:48.796247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:41:48.796247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:41:48.796247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:41:48.796247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:41:48.796247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:41:48.796247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:41:48.796247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:41:48.796247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:41:48.796247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:41:48.796247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:48.796247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:41:48.796247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:48.796247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:41:48.796247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:48.796247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:41:48.799803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:48.799803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:41:48.799803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:48.799803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:41:48.799803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:41:48.799803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:48.799803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:41:48.799803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:48.799803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:41:48.799803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:41:48.799803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:48.799803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:41:48.799803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:48.799803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:41:48.799803 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:41:48.800511 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:41:48.800511 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:41:48.800511 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:41:48.800511 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:41:48.800511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:48.800511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:41:48.800511 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:48.800511 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:41:48.800511 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:41:48.800952 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:48.800952 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:41:48.800952 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:48.800952 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:41:48.800952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:48.800952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:41:48.800952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:41:48.800952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:41:48.800952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:41:48.800952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:41:48.800952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:41:48.800952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:48.800952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:41:48.800952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:48.801361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:41:48.801361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:48.801361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:41:48.801361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:48.801361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:41:48.801361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:48.801361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:41:48.801361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:41:48.801361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:48.801361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:41:48.801361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:48.801361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:41:48.801361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:41:48.801361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:48.801761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:41:48.801761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:48.801761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:41:48.801761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:48.801761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:41:48.801761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:41:48.801761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:41:48.801761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:41:48.801761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:41:48.801761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:41:48.801761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:41:48.801761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:41:48.801761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:41:48.801761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:41:48.801761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:41:48.801761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:41:48.802221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:41:48.802221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:41:48.802221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:41:48.802221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:41:48.802221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:41:48.802221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:48.802221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:41:48.802221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:48.802221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:41:48.802221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:48.802221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:41:48.802221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:48.802221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:41:48.802221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:48.802221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:41:48.802221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:41:48.802221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:48.802221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:41:48.802221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:48.802725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:41:48.802725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:41:48.802725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:48.802725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:41:48.802725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:48.802725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:41:53.625944 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:41:53.626642 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:41:53.626823 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:41:53.628131 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:41:53.668630 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:41:53.669356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:53.670371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:41:53.670371 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:53.670371 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:41:53.670371 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:41:53.670371 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:53.670371 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:41:53.670371 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:53.670371 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:41:53.672539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.044 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:53.672539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:41:53.672539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:41:53.672539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:41:53.672539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:41:53.672539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:41:53.672539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:41:53.672539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:53.672539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:41:53.672539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:53.672539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:41:53.672539 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:53.673026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:41:53.673026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:53.673026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:41:53.673026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:53.673026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:41:53.673026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:41:53.673026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:53.673026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:41:53.673026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:53.673026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:41:53.673026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:41:53.673026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:53.673026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:41:53.673518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:53.673518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:41:53.673639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:53.673639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:41:53.673639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:41:53.673639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:41:53.673639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:41:53.673639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:41:53.673639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:41:53.673639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:41:53.673639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:41:53.673639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:41:53.673639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:41:53.673639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:41:53.673639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:41:53.673639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:41:53.673639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:41:53.673639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:41:53.674224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:41:53.674224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:41:53.674224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:53.674224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:41:53.674224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:53.674224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:41:53.674224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:53.674224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:41:53.674224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:53.674224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:41:53.674224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:53.674224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:41:53.674224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:41:53.674224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:53.674224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:41:53.674224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:53.674224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:41:53.674224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:41:53.674224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:53.674920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:41:53.674920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:53.674920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:41:53.772694 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:41:53.773440 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:41:53.773568 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:41:53.775370 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:41:53.775370 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:41:53.775370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:53.776045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:41:53.776045 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:53.776045 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:41:53.776045 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:41:53.776045 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:53.776045 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:41:53.776045 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:53.776045 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:41:53.778160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:53.778160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:41:53.778160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:41:53.778160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:41:53.778160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:41:53.778160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:41:53.778160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:41:53.778160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:53.778160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:41:53.778160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:53.778160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:41:53.778160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:53.778610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:41:53.778610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:53.778610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:41:53.778610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:53.778610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:41:53.778610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:41:53.778610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:53.778610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:41:53.778610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:53.778610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:41:53.778610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:41:53.778610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:53.778610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:41:53.779050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:53.779050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:41:53.780052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:53.780052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:41:53.780052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:41:53.780052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:41:53.780052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:41:53.780052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:41:53.780052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:41:53.780052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:41:53.780052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:41:53.780052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:41:53.780052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:41:53.780052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:41:53.780052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:41:53.780052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:41:53.780052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:41:53.780052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:41:53.780613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:41:53.780613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:41:53.780613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:53.780613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:41:53.780613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:53.780613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:41:53.780613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:53.780613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:41:53.780613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:53.780613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:41:53.780613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:53.780613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:41:53.780613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:41:53.780613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:53.780613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:41:53.780613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:53.780613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:41:53.780613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:41:53.780613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:53.781240 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:41:53.781240 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:53.781240 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:41:53.781240 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:41:53.781240 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:41:53.781240 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:41:53.781240 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:41:53.781594 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:41:53.781876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:53.782467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:41:53.782467 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:53.782467 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:41:53.782467 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:41:53.782467 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:53.782467 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:41:53.782467 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:53.782467 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:41:53.783308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:53.783308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:41:53.783308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:41:53.783308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:41:53.783308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:53.783308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:41:53.783308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:53.783308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:41:53.783308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:53.783308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:41:53.783308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:53.783308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:41:53.783308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:53.783308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:41:53.783308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:41:53.783853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:53.783853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:41:53.783853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:53.783853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:41:53.783853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:41:53.783853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:53.783853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:41:53.783853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:53.783853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:41:53.784695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:53.784695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:41:53.784695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:41:53.784695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:41:53.784695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:41:53.784695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:41:53.784695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:41:53.784695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:41:53.784695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:41:53.784695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:41:53.784695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:53.784695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:41:53.785123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:53.785123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:41:53.785123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:53.785123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:41:53.785123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:53.785123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:41:53.785123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:53.785123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:41:53.785123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:41:53.785123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:53.785123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:41:53.785123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:53.785123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:41:53.785123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:41:53.785591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:53.785591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:41:53.785591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:53.785591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:41:53.785591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:53.785591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:41:53.785591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:41:53.785591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:41:53.785591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:41:53.785591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:41:53.785591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:41:53.785591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:41:53.785591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:41:53.785591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:41:53.786019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:41:53.786019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:41:53.786019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:41:53.786019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:41:53.786019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:41:53.786019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:41:53.786019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:41:53.786019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:41:53.786019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:41:53.786019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:41:53.786019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:41:53.786019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:53.786019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:41:53.786019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:53.786019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:41:53.786019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:53.786019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:41:53.786019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:53.786019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:41:53.786536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:53.786536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:41:53.786536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:41:53.786536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:53.786536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:41:53.786536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:53.786536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:41:53.786536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:41:53.786536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:53.786536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:41:53.786536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:53.786536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:41:53.787954 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:41:53.788047 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:41:53.788128 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:41:53.788279 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:41:53.788577 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:41:53.788796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:53.789818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:41:53.789818 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:53.789818 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:41:53.789818 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:41:53.789818 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:53.789818 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:41:53.789818 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:53.789818 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:41:53.791109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:53.791109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:41:53.791109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:41:53.791109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:41:53.791109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:53.791109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:41:53.791109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:53.791109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:41:53.791109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:53.791109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:41:53.791109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:53.791109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:41:53.791109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:53.791109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:41:53.791109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:41:53.791633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:53.791633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:41:53.791633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:53.791633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:41:53.791633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:41:53.791633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:53.791633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:41:53.791633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:53.791633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:41:53.792110 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:53.792110 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:41:53.792110 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:41:53.792110 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:41:53.792110 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:41:53.792110 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:41:53.792110 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:41:53.792110 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:41:53.792110 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:41:53.792110 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:53.792110 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:41:53.792505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:53.792505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:41:53.792505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:53.792505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:41:53.792505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:53.792505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:41:53.792505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:53.792505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:41:53.792505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:41:53.792505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:53.792505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:41:53.792505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:53.792505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:41:53.792505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:41:53.792964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:53.792964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:41:53.792964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:53.792964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:41:53.793632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:53.793632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:41:53.793632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:41:53.793632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:41:53.793632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:41:53.793632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:41:53.793632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:41:53.793632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:41:53.793632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:41:53.793632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:41:53.793632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:41:53.793632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:41:53.793632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:41:53.793632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:41:53.793632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:41:53.793632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:41:53.793632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:41:53.794192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:41:53.794192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:41:53.794192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:41:53.794192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:53.794192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:41:53.794192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:53.794192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:41:53.794192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:53.794192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:41:53.794192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:53.794192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:41:53.794192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:53.794192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:41:53.794192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:41:53.794192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:53.794192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:41:53.794192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:53.794192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:41:53.794192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:41:53.794819 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:53.794819 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:41:53.794819 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:53.794819 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:41:53.794819 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:41:53.794819 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:41:53.795036 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:41:53.795153 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:41:53.795533 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:41:53.796154 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:41:53.796348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:53.796934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:41:53.796934 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:53.796934 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:41:53.796934 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:41:53.796934 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:53.796934 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:41:53.796934 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:53.796934 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:41:53.798142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:53.798142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:41:53.798142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:41:53.798142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:41:53.798142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:41:53.798142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:41:53.798142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:41:53.798142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:53.798142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:41:53.798142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:53.798142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:41:53.798142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:53.798548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:41:53.798548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:53.798548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:41:53.798548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:53.798548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:41:53.798548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:41:53.798548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:53.798548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:41:53.798548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:53.798548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:41:53.798548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:41:53.798548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:53.798548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:41:53.798941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:53.798941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:41:53.799447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:53.799447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:41:53.799447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:41:53.799447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:41:53.799447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:41:53.799447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:41:53.799447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:41:53.799447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:41:53.799447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:41:53.799447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:41:53.799447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:41:53.799447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:41:53.799447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:41:53.799447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:41:53.799447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:41:53.799447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:41:53.799922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:41:53.799922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:41:53.799922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:53.799922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:41:53.799922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:53.799922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:41:53.799922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:53.799922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:41:53.799922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:53.799922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:41:53.799922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:53.799922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:41:53.799922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:41:53.799922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:53.799922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:41:53.799922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:53.799922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:41:53.799922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:41:53.799922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:53.800399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:41:53.800399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:53.800399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:41:58.627214 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:41:58.627833 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:41:58.627833 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:41:58.628625 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:41:58.630116 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG dbcounter [-] [106314] Writing DB stats neutron:SELECT=240 {{(pid=106314) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 09 09:41:58.657964 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:41:58.658197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:58.658936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:41:58.658936 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:58.658936 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:41:58.658936 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:41:58.658936 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:58.658936 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:41:58.658936 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:58.658936 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:41:58.659715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.031 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:58.659715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:41:58.659715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:41:58.659715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:41:58.659715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:41:58.659715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:41:58.659715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:41:58.659715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:58.659715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:41:58.659715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:58.659715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:41:58.659715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:58.660519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:41:58.660519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:58.660519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:41:58.660519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:58.660519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:41:58.660519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:41:58.660519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:58.660519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:41:58.660519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:58.660519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:41:58.660519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:41:58.660519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:58.660519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:41:58.661274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:58.661274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:41:58.661274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:58.661274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:41:58.661274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:41:58.661274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:41:58.661274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:41:58.661274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:41:58.661274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:41:58.661274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:41:58.661274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:41:58.661274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:41:58.661274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:41:58.661274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:41:58.661274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:41:58.661745 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:41:58.661745 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:41:58.661745 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:41:58.661745 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:41:58.661745 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:41:58.661745 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:58.661745 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:41:58.661745 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:58.661745 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:41:58.661745 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:58.661745 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:41:58.661745 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:58.661745 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:41:58.661745 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:58.661745 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:41:58.661745 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:41:58.661745 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:58.661745 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:41:58.661745 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:58.662433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:41:58.662433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:41:58.662433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:58.662433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:41:58.662433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:58.662433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:41:58.773674 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:41:58.774549 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:41:58.774647 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:41:58.775545 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:41:58.775783 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:41:58.775864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:58.776405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:41:58.776405 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:58.776405 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:41:58.776405 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:41:58.776405 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:58.776405 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:41:58.776405 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:58.776405 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:41:58.777284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:58.777284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:41:58.777284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:41:58.777284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:41:58.777284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:41:58.777284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:41:58.777284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:41:58.777284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:58.777284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:41:58.777284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:58.777284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:41:58.777284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:58.778326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:41:58.778326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:58.778326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:41:58.778326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:58.778326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:41:58.778326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:41:58.778326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:58.778326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:41:58.778326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:58.778326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:41:58.778326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:41:58.778326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:58.778326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:41:58.779123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:58.779123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:41:58.779123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:58.779123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:41:58.779123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:41:58.779123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:41:58.779123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:41:58.779123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:41:58.779123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:41:58.779123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:41:58.779123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:41:58.779123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:41:58.779123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:41:58.779123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:41:58.779123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:41:58.779820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:41:58.779820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:41:58.779820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:41:58.779820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:41:58.779820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:41:58.779820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:58.779820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:41:58.779820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:58.779820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:41:58.779820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:58.779820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:41:58.779820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:58.779820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:41:58.779820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:58.779820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:41:58.779820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:41:58.779820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:58.779820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:41:58.779820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:58.780624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:41:58.780624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:41:58.780624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:58.780624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:41:58.780624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:58.780624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:41:58.781165 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:41:58.781165 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:41:58.781165 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:41:58.781165 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:41:58.781597 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:41:58.782127 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:58.782127 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:41:58.782127 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:58.782127 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:41:58.782127 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:41:58.782127 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:58.782127 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:41:58.782127 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:58.782127 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:41:58.782674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:58.782674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:41:58.782674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:41:58.782674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:41:58.782674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:58.782674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:41:58.782674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:58.782674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:41:58.782674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:58.782674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:41:58.782674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:58.782674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:41:58.782674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:58.782674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:41:58.782674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:41:58.783198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:58.783198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:41:58.783198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:58.783198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:41:58.783198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:41:58.783198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:58.783198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:41:58.783198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:58.783198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:41:58.783588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:58.783588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:41:58.783588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:41:58.783588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:41:58.783588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:41:58.783588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:41:58.783588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:41:58.783588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:41:58.783588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:41:58.783588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:41:58.783588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:58.783588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:41:58.784225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:58.784225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:41:58.784225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:58.784225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:41:58.784225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:58.784225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:41:58.784225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:58.784225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:41:58.784225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:41:58.784225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:58.784225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:41:58.784225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:58.784225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:41:58.784225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:41:58.784840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:58.784840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:41:58.784840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:58.784840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:41:58.784840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:58.784840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:41:58.784840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:41:58.784840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:41:58.784840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:41:58.784840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:41:58.784840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:41:58.784840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:41:58.784840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:41:58.784840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:41:58.785203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:41:58.785203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:41:58.785203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:41:58.785203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:41:58.785203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:41:58.785203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:41:58.785203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:41:58.785203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:41:58.785203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:41:58.785203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:41:58.785203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:41:58.785203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:58.785203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:41:58.785203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:58.785203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:41:58.785203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:58.785203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:41:58.785203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:58.785203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:41:58.786009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:58.786009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:41:58.786009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:41:58.786009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:58.786009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:41:58.786009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:58.786009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:41:58.786009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:41:58.786009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:58.786009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:41:58.786009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:58.786009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:41:58.788269 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:41:58.788560 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:41:58.788622 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:41:58.788729 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:41:58.789214 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:41:58.789681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:58.790510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:41:58.790510 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:58.790510 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:41:58.790510 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:41:58.790510 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:58.790510 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:41:58.790510 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:58.790510 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:41:58.791695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:58.791695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:41:58.791695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:41:58.791695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:41:58.791695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:58.791695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:41:58.791695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:58.791695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:41:58.791695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:58.791695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:41:58.791695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:58.791695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:41:58.791695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:58.791695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:41:58.791695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:41:58.792029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:58.792029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:41:58.792029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:58.792029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:41:58.792029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:41:58.792029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:58.792029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:41:58.792029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:58.792029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:41:58.793185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:58.793185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:41:58.793185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:41:58.793185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:41:58.793185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:41:58.793185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:41:58.793185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:41:58.793185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:41:58.793185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:41:58.793185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:58.793185 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:41:58.793465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:58.793465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:41:58.793465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:58.793465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:41:58.793465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:58.793465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:41:58.793465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:58.793465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:41:58.793465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:41:58.793465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:58.793465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:41:58.793465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:58.793465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:41:58.793465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:41:58.794184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:58.794184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:41:58.794184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:58.794184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:41:58.795566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:58.795566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:41:58.795566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:41:58.795566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:41:58.795566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:41:58.795566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:41:58.795566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:41:58.795566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:41:58.795566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:41:58.795566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:41:58.795566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:41:58.795566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:41:58.795566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:41:58.795566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:41:58.795566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:41:58.795566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:41:58.795566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:41:58.795998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:41:58.795998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:41:58.795998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:41:58.795998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:58.795998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:41:58.795998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:58.795998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:41:58.795998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:58.795998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:41:58.795998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:58.795998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:41:58.795998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:58.795998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:41:58.795998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:41:58.795998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:58.795998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:41:58.795998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:58.795998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:41:58.795998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:41:58.796385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:58.796385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:41:58.796385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:58.796385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:41:58.796385 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:41:58.796494 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:41:58.796663 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:41:58.796971 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:41:58.797366 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:41:58.797677 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:41:58.797870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:58.798411 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:41:58.798411 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:58.798411 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:41:58.798411 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:41:58.798411 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:58.798411 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:41:58.798411 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:58.798411 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:41:58.799072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:58.799072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:41:58.799072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:41:58.799072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:41:58.799072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:41:58.799072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:41:58.799072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:41:58.799072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:58.799072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:41:58.799072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:58.799072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:41:58.799072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:58.799468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:41:58.799468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:58.799468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:41:58.799468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:58.799468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:41:58.799468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:41:58.799468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:58.799468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:41:58.799468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:58.799468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:41:58.799468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:41:58.799468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:58.799468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:41:58.799877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:58.799877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:41:58.799877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:58.799877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:41:58.799877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:41:58.799877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:41:58.799877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:41:58.799877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:41:58.799877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:41:58.799877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:41:58.799877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:41:58.799877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:41:58.799877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:41:58.799877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:41:58.799877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:41:58.800349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:41:58.800349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:41:58.800349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:41:58.800349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:41:58.800349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:41:58.800349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:58.800349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:41:58.800349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:41:58.800349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:41:58.800349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:41:58.800349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:41:58.800349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:41:58.800349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:41:58.800349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:41:58.800349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:41:58.800349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:41:58.800349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:41:58.800349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:41:58.800349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:41:58.800928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:41:58.800928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:41:58.800928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:41:58.800928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:41:58.800928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:41:58.800928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:42:03.627952 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:42:03.629304 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:42:03.629304 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:42:03.630160 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:42:03.668820 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:42:03.668820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:03.669458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:42:03.669458 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:03.669458 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:42:03.669458 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:42:03.669458 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:03.669458 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:42:03.669458 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:03.669458 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:42:03.671211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:03.671211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:42:03.671211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:42:03.671211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:42:03.671211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:42:03.671211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:42:03.671211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:42:03.671211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:03.671211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:42:03.671211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:03.671211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:42:03.671211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:03.671533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:42:03.671533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:03.671533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:42:03.671533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:03.671533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:42:03.671533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:42:03.671533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:03.671533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:42:03.671533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:03.671533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:42:03.671533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:42:03.671533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:03.671533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:42:03.671810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:03.671810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:42:03.673890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:03.673890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:42:03.673890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:42:03.673890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:42:03.673890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:42:03.673890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:42:03.673890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:42:03.673890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:42:03.673890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:42:03.673890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:42:03.673890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:42:03.673890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:42:03.673890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:42:03.673890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:42:03.673890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:42:03.673890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:42:03.674266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:42:03.674266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:42:03.674266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:03.674266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:42:03.674266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:03.674266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:42:03.674266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:03.674266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:42:03.674266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:03.674266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:42:03.674266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:03.674266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:42:03.674266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:42:03.674266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:03.674266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:42:03.674266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:03.674266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:42:03.674266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:42:03.674266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:03.674693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:42:03.674693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:03.674693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:42:03.774709 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:42:03.775334 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:42:03.775533 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:42:03.776556 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:42:03.776764 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:42:03.776917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:03.777555 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:42:03.777555 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:03.777555 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:42:03.777555 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:42:03.777555 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:03.777555 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:42:03.777555 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:03.777555 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:42:03.779256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:03.779256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:42:03.779256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:42:03.779256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:42:03.779256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:42:03.779256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:42:03.779256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:42:03.779256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:03.779256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:42:03.779256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:03.779256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:42:03.779256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:03.779803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:42:03.779803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:03.779803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:42:03.779803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:03.779803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:42:03.779803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:42:03.779803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:03.779803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:42:03.779803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:03.779803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:42:03.779803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:42:03.779803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:03.779803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:42:03.780255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:03.780255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:42:03.781114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:03.781114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:42:03.781114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:42:03.781114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:42:03.781114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:42:03.781114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:42:03.781114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:42:03.781114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:42:03.781114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:42:03.781114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:42:03.781114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:42:03.781114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:42:03.781114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:42:03.781114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:42:03.781114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:42:03.781114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:42:03.782256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:42:03.782256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:42:03.782256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:03.782256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:42:03.782256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:03.782256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:42:03.782256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:03.782256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:42:03.782256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:03.782256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:42:03.782256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:03.782256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:42:03.782256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:42:03.782256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:03.782256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:42:03.782256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:03.782256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:42:03.782256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:42:03.782256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:03.782948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:42:03.782948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:03.782948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:42:03.782948 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:42:03.782948 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:42:03.782948 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:42:03.782948 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:42:03.783218 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:42:03.783218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:03.784237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:42:03.784237 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:03.784237 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:42:03.784237 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:42:03.784237 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:03.784237 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:42:03.784237 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:03.784237 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:42:03.785529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:03.785529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:42:03.785529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:42:03.785529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:42:03.785529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:03.785529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:42:03.785529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:03.785529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:42:03.785529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:03.785529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:42:03.785529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:03.785529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:42:03.785529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:03.785529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:42:03.785529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:42:03.786042 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:03.786042 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:42:03.786042 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:03.786042 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:42:03.786042 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:42:03.786042 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:03.786042 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:42:03.786042 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:03.786042 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:42:03.787024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:03.787024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:42:03.787024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:42:03.787024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:42:03.787024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:42:03.787024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:42:03.787024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:42:03.787024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:42:03.787024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:42:03.787024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:42:03.787024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:03.787024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:42:03.787498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:03.787498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:42:03.787498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:03.787498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:42:03.787498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:03.787498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:42:03.787498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:03.787498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:42:03.787498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:42:03.787498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:03.787498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:42:03.787498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:03.787498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:42:03.787498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:42:03.787984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:03.787984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:42:03.787984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:03.787984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:42:03.788560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:03.788560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:42:03.788560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:42:03.788560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:42:03.788560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:42:03.788560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:42:03.788560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:42:03.788560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:42:03.788560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:42:03.788560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:42:03.788560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:42:03.788560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:42:03.788560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:42:03.788560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:42:03.788560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:42:03.788560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:42:03.789180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:42:03.789180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:42:03.789180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:42:03.789180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:42:03.789180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:42:03.789180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:03.789180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:42:03.789180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:03.789180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:42:03.789180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:03.789180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:42:03.789180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:03.789180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:42:03.789180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:03.789180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:42:03.789180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:42:03.789180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:03.789180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:42:03.789890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:03.789890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:42:03.789890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:42:03.789890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:03.789890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:42:03.789890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:03.789890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:42:03.790443 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:42:03.790779 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:42:03.790942 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:42:03.791142 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:42:03.792857 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:42:03.792857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:03.793301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:42:03.793301 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:03.793301 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:42:03.793301 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:42:03.793301 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:03.793301 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:42:03.793301 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:03.793301 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:42:03.794534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:03.794534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:42:03.794534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:42:03.794534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:42:03.794534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:03.794534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:42:03.794534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:03.794534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:42:03.794534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:03.794534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:42:03.794534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:03.794534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:42:03.794534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:03.794534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:42:03.794534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:42:03.794970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:03.794970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:42:03.794970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:03.794970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:42:03.794970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:42:03.794970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:03.794970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:42:03.794970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:03.794970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:42:03.795751 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:03.795751 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:42:03.795751 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:42:03.795751 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:42:03.795751 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:42:03.795751 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:42:03.795751 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:42:03.795751 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:42:03.795751 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:42:03.795751 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:03.795751 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:42:03.796097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:03.796097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:42:03.796097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:03.796097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:42:03.796097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:03.796097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:42:03.796097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:03.796097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:42:03.796097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:42:03.796097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:03.796097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:42:03.796097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:03.796097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:42:03.796097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:42:03.796460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:03.796460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:42:03.796460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:03.796460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:42:03.797987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:03.797987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:42:03.797987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:42:03.797987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:42:03.797987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:42:03.797987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:42:03.797987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:42:03.797987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:42:03.797987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:42:03.797987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:42:03.797987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:42:03.797987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:42:03.797987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:42:03.797987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:42:03.797987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:42:03.797987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:42:03.797987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:42:03.798488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:42:03.798488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:42:03.798488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:42:03.798488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:03.798488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:42:03.798488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:03.798488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:42:03.798488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:03.798488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:42:03.798488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:03.798488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:42:03.798488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:03.798488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:42:03.798488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:42:03.798488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:03.798488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:42:03.798488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:03.798488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:42:03.798488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:42:03.799018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:03.799018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:42:03.799018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:03.799018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:42:03.799018 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:42:03.799018 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:42:03.799018 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:42:03.799018 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:42:03.799249 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:42:03.799706 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:42:03.799706 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:03.800244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:42:03.800244 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:03.800244 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:42:03.800244 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:42:03.800244 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:03.800244 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:42:03.800244 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:03.800244 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:42:03.801186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:03.801186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:42:03.801186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:42:03.801186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:42:03.801186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:42:03.801186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:42:03.801186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:42:03.801186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:03.801186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:42:03.801186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:03.801186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:42:03.801186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:03.802322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:42:03.802322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:03.802322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:42:03.802322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:03.802322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:42:03.802322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:42:03.802322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:03.802322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:42:03.802322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:03.802322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:42:03.802322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:42:03.802322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:03.802322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:42:03.802651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:03.802651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:42:03.802651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:03.802651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:42:03.802651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:42:03.802651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:42:03.802651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:42:03.802651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:42:03.802651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:42:03.802651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:42:03.802651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:42:03.802651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:42:03.802651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:42:03.802651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:42:03.802651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:42:03.803453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:42:03.803453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:42:03.803453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:42:03.803453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:42:03.803453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:42:03.803453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:03.803453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:42:03.803453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:03.803453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:42:03.803453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:03.803453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:42:03.803453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:03.803453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:42:03.803453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:03.803453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:42:03.803453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:42:03.803453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:03.803453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:42:03.803453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:03.804286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:42:03.804286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:42:03.804286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:03.804286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:42:03.804286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:03.804286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:42:08.653531 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:42:08.653531 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:42:08.653531 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:42:08.653531 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:42:08.679804 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:42:08.680083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:08.680872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:42:08.680872 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:08.680872 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:42:08.680872 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:42:08.680872 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:08.680872 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:42:08.680872 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:08.680872 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:42:08.681843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.051 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:08.681843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:42:08.681843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:42:08.681843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:42:08.681843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:42:08.681843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:42:08.681843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:42:08.681843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:08.681843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:42:08.681843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:08.681843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:42:08.681843 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:08.682341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:42:08.682341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:08.682341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:42:08.682341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:08.682341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:42:08.682341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:42:08.682341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:08.682341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:42:08.682341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:08.682341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:42:08.682341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:42:08.682341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:08.682341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:42:08.682822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:08.682822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:42:08.682822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:08.682822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:42:08.682822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:42:08.682822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:42:08.682822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:42:08.682822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:42:08.682822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:42:08.682822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:42:08.682822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:42:08.682822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:42:08.682822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:42:08.682822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:42:08.682822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:42:08.683493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:42:08.683493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:42:08.683493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:42:08.683493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:42:08.683493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:42:08.683493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:08.683493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:42:08.683493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:08.683493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:42:08.683493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:08.683493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:42:08.683493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:08.683493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:42:08.683493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:08.683493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:42:08.683493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:42:08.683493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:08.683493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:42:08.683493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:08.684158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:42:08.684158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:42:08.684158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:08.684158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:42:08.684158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:08.684158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:42:08.775308 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:42:08.776142 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:42:08.776142 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:42:08.776864 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:42:08.777170 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:42:08.777358 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:08.778174 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:42:08.778174 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:08.778174 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:42:08.778174 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:42:08.778174 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:08.778174 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:42:08.778174 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:08.778174 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:42:08.779266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:08.779266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:42:08.779266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:42:08.779266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:42:08.779266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:42:08.779266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:42:08.779266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:42:08.779266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:08.779266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:42:08.779266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:08.779266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:42:08.779266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:08.779734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:42:08.779734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:08.779734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:42:08.779734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:08.779734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:42:08.779734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:42:08.779734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:08.779734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:42:08.779734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:08.779734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:42:08.779734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:42:08.779734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:08.779734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:42:08.780170 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:08.780170 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:42:08.780170 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:08.780170 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:42:08.780170 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:42:08.780170 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:42:08.780170 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:42:08.780170 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:42:08.780170 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:42:08.780170 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:42:08.780170 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:42:08.780170 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:42:08.780170 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:42:08.780170 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:42:08.780170 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:42:08.780638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:42:08.780638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:42:08.780638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:42:08.780638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:42:08.780638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:42:08.780638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:08.780638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:42:08.780638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:08.780638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:42:08.780638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:08.780638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:42:08.780638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:08.780638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:42:08.780638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:08.780638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:42:08.780638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:42:08.780638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:08.780638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:42:08.780638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:08.781342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:42:08.781342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:42:08.781342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:08.781342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:42:08.781342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:08.781342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:42:08.783595 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:42:08.784224 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:42:08.784224 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:42:08.784224 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:42:08.784409 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:42:08.784625 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:08.785337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:42:08.785337 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:08.785337 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:42:08.785337 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:42:08.785337 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:08.785337 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:42:08.785337 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:08.785337 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:42:08.786365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:08.786365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:42:08.786365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:42:08.786365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:42:08.786365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:08.786365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:42:08.786365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:08.786365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:42:08.786365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:08.786365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:42:08.786365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:08.786365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:42:08.786365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:08.786365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:42:08.786365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:42:08.786737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:08.786737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:42:08.786737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:08.786737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:42:08.786737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:42:08.786737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:08.786737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:42:08.786737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:08.786737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:42:08.787027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:08.787027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:42:08.787027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:42:08.787027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:42:08.787027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:42:08.787027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:42:08.787027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:42:08.787027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:42:08.787027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:42:08.787027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:42:08.787027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:08.787027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:42:08.787295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:08.787295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:42:08.787295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:08.787295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:42:08.787295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:08.787295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:42:08.787295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:08.787295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:42:08.787295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:42:08.787295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:08.787295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:42:08.787295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:08.787295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:42:08.787295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:42:08.787566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:08.787566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:42:08.787566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:08.787566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:42:08.788119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:08.788119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:42:08.788119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:42:08.788119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:42:08.788119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:42:08.788119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:42:08.788119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:42:08.788119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:42:08.788119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:42:08.788119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:42:08.788119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:42:08.788119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:42:08.788119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:42:08.788119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:42:08.788119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:42:08.788119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:42:08.788454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:42:08.788454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:42:08.788454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:42:08.788454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:42:08.788454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:42:08.788454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:08.788454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:42:08.788454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:08.788454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:42:08.788454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:08.788454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:42:08.788454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:08.788454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:42:08.788454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:08.788454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:42:08.788454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:42:08.788454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:08.788454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:42:08.788963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:08.788963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:42:08.788963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:42:08.788963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:08.788963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:42:08.788963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:08.788963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:42:08.794239 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:42:08.795649 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:42:08.795649 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:42:08.795649 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:42:08.795649 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:42:08.795950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:08.796419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:42:08.796419 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:08.796419 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:42:08.796419 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:42:08.796419 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:08.796419 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:42:08.796419 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:08.796419 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:42:08.797263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:08.797263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:42:08.797263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:42:08.797263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:42:08.797263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:08.797263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:42:08.797263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:08.797263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:42:08.797263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:08.797263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:42:08.797263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:08.797263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:42:08.797263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:08.797263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:42:08.797263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:42:08.797936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:08.797936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:42:08.797936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:08.797936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:42:08.797936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:42:08.797936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:08.797936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:42:08.797936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:08.797936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:42:08.798454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:08.798454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:42:08.798454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:42:08.798454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:42:08.798454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:42:08.798454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:42:08.798454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:42:08.798454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:42:08.798454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:42:08.798454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:08.798454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:42:08.798933 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:08.798933 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:42:08.798933 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:08.798933 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:42:08.798933 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:08.798933 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:42:08.798933 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:08.798933 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:42:08.798933 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:42:08.798933 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:08.798933 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:42:08.798933 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:08.798933 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:42:08.798933 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:42:08.799360 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:08.799360 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:42:08.799360 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:08.799360 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:42:08.800889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:08.800889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:42:08.800889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:42:08.800889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:42:08.800889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:42:08.800889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:42:08.800889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:42:08.800889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:42:08.800889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:42:08.800889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:42:08.800889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:42:08.800889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:42:08.800889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:42:08.800889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:42:08.800889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:42:08.800889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:42:08.800889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:42:08.801491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:42:08.801491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:42:08.801491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:42:08.801491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:08.801491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:42:08.801491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:08.801491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:42:08.801491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:08.801491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:42:08.801491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:08.801491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:42:08.801491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:08.801491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:42:08.801491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:42:08.801491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:08.801491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:42:08.801491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:08.801491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:42:08.801491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:42:08.802170 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:08.802170 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:42:08.802170 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:08.802170 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:42:08.802170 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:42:08.802170 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:42:08.802170 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:42:08.802170 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:42:08.802389 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:42:08.802389 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:42:08.802605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:08.804192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:42:08.804192 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:08.804192 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:42:08.804192 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:42:08.804192 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:08.804192 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:42:08.804192 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:08.804192 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:42:08.805827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:08.805827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:42:08.805827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:42:08.805827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:42:08.805827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:42:08.805827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:42:08.805827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:42:08.805827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:08.805827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:42:08.805827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:08.805827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:42:08.805827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:08.806127 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:42:08.806127 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:08.806127 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:42:08.806127 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:08.806127 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:42:08.806127 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:42:08.806127 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:08.806127 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:42:08.806127 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:08.806127 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:42:08.806127 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:42:08.806127 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:08.806127 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:42:08.806406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:08.806406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:42:08.807973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:08.807973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:42:08.807973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:42:08.807973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:42:08.807973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:42:08.807973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:42:08.807973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:42:08.807973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:42:08.807973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:42:08.807973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:42:08.807973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:42:08.807973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:42:08.807973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:42:08.807973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:42:08.807973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:42:08.807973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:42:08.808315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:42:08.808315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:42:08.808315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:08.808315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:42:08.808315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:08.808315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:42:08.808315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:08.808315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:42:08.808315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:08.808315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:42:08.808315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:08.808315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:42:08.808315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:42:08.808315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:08.808315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:42:08.808315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:08.808315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:42:08.808315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:42:08.808315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:08.808723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:42:08.808723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:08.808723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:42:13.630267 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:42:13.630886 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:42:13.630926 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:42:13.632319 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:42:13.670525 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:42:13.670828 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:13.671577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:42:13.671577 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:13.671577 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:42:13.671577 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:42:13.671577 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:13.671577 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:42:13.671577 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:13.671577 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:42:13.672783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:13.672783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:42:13.672783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:42:13.672783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:42:13.672783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:42:13.672783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:42:13.672783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:42:13.672783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:13.672783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:42:13.672783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:13.672783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:42:13.672783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:13.673258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:42:13.673258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:13.673258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:42:13.673258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:13.673258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:42:13.673258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:42:13.673258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:13.673258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:42:13.673258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:13.673258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:42:13.673258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:42:13.673258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:13.673258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:42:13.674116 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:13.674116 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:42:13.674116 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:13.674116 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:42:13.674116 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:42:13.674116 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:42:13.674116 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:42:13.674116 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:42:13.674116 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:42:13.674116 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:42:13.674116 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:42:13.674116 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:42:13.674116 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:42:13.674116 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:42:13.674116 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:42:13.674657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:42:13.674657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:42:13.674657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:42:13.674657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:42:13.674657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:42:13.674657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:13.674657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:42:13.674657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:13.674657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:42:13.674657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:13.674657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:42:13.674657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:13.674657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:42:13.674657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:13.674657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:42:13.674657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:42:13.674657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:13.674657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:42:13.674657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:13.675371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:42:13.675371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:42:13.675371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:13.675371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:42:13.675371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:13.675371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:42:13.776082 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:42:13.776699 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:42:13.776967 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:42:13.777925 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:42:13.778309 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:42:13.778584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:13.779548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:42:13.779548 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:13.779548 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:42:13.779548 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:42:13.779548 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:13.779548 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:42:13.779548 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:13.779548 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:42:13.780576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:13.780576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:42:13.780576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:42:13.780576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:42:13.780576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:42:13.780576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:42:13.780576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:42:13.780576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:13.780576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:42:13.780576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:13.780576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:42:13.780576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:13.781975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:42:13.781975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:13.781975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:42:13.781975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:13.781975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:42:13.781975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:42:13.781975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:13.781975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:42:13.781975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:13.781975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:42:13.781975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:42:13.781975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:13.781975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:42:13.782515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:13.782515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:42:13.782515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:13.782515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:42:13.782515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:42:13.782515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:42:13.782515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:42:13.782515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:42:13.782515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:42:13.782515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:42:13.782515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:42:13.782515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:42:13.782515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:42:13.782515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:42:13.782515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:42:13.784013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:42:13.784013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:42:13.784013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:42:13.784013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:42:13.784013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:42:13.784013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:13.784013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:42:13.784013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:13.784013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:42:13.784013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:13.784013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:42:13.784013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:13.784013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:42:13.784013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:13.784013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:42:13.784013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:42:13.784013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:13.784013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:42:13.784013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:13.785617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:42:13.785617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:42:13.785617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:13.785617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:42:13.785617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:13.785617 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:42:13.785617 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:42:13.785617 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:42:13.785617 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:42:13.785617 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:42:13.786123 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:42:13.786123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:13.786236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:42:13.786236 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:13.786236 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:42:13.786236 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:42:13.786236 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:13.786236 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:42:13.786236 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:13.786236 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:42:13.787667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:13.787667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:42:13.787667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:42:13.787667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:42:13.787667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:13.787667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:42:13.787667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:13.787667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:42:13.787667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:13.787667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:42:13.787667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:13.787667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:42:13.787667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:13.787667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:42:13.787667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:42:13.788558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:13.788558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:42:13.788558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:13.788558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:42:13.788558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:42:13.788558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:13.788558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:42:13.788558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:13.788558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:42:13.789257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:13.789257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:42:13.789257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:42:13.789257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:42:13.789257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:42:13.789257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:42:13.789257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:42:13.789257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:42:13.789257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:42:13.789257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:42:13.789257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:13.789257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:42:13.789845 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:13.789845 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:42:13.789845 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:13.789845 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:42:13.789845 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:13.789845 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:42:13.789845 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:13.789845 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:42:13.789845 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:42:13.789845 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:13.789845 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:42:13.789845 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:13.789845 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:42:13.789845 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:42:13.790237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:13.790237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:42:13.790237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:13.790237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:42:13.791311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:13.791311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:42:13.791311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:42:13.791311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:42:13.791311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:42:13.791311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:42:13.791311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:42:13.791311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:42:13.791311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:42:13.791311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:42:13.791311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:42:13.791311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:42:13.791311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:42:13.791311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:42:13.791311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:42:13.791311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:42:13.791767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:42:13.791767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:42:13.791767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:42:13.791767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:42:13.791767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:42:13.791767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:13.791767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:42:13.791767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:13.791767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:42:13.791767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:13.791767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:42:13.791767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:13.791767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:42:13.791767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:13.791767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:42:13.791767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:42:13.791767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:13.791767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:42:13.792275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:13.792275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:42:13.792275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:42:13.792275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:13.792275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:42:13.792275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:13.792275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:42:13.794643 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:42:13.794900 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:42:13.795007 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:42:13.795254 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:42:13.795603 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:42:13.795902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:13.796388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:42:13.796388 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:13.796388 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:42:13.796388 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:42:13.796388 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:13.796388 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:42:13.796388 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:13.796388 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:42:13.797235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:13.797235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:42:13.797235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:42:13.797235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:42:13.797235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:13.797235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:42:13.797235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:13.797235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:42:13.797235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:13.797235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:42:13.797235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:13.797235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:42:13.797235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:13.797235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:42:13.797235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:42:13.797666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:13.797666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:42:13.797666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:13.797666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:42:13.797666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:42:13.797666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:13.797666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:42:13.797666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:13.797666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:42:13.798187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:13.798187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:42:13.798187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:42:13.798187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:42:13.798187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:42:13.798187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:42:13.798187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:42:13.798187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:42:13.798187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:42:13.798187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:13.798187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:42:13.798536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:13.798536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:42:13.798536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:13.798536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:42:13.798536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:13.798536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:42:13.798536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:13.798536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:42:13.798536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:42:13.798536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:13.798536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:42:13.798536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:13.798536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:42:13.798536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:42:13.798983 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:13.798983 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:42:13.798983 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:13.798983 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:42:13.799399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:13.799399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:42:13.799399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:42:13.799399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:42:13.799399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:42:13.799399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:42:13.799399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:42:13.799399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:42:13.799399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:42:13.799399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:42:13.799399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:42:13.799399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:42:13.799399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:42:13.799399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:42:13.799399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:42:13.799399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:42:13.799399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:42:13.799995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:42:13.799995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:42:13.799995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:42:13.799995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:13.799995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:42:13.799995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:13.799995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:42:13.799995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:13.799995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:42:13.799995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:13.799995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:42:13.799995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:13.799995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:42:13.799995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:42:13.799995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:13.799995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:42:13.799995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:13.799995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:42:13.799995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:42:13.800616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:13.800616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:42:13.800616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:13.800616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:42:13.801041 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:42:13.801365 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:42:13.801561 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:42:13.801759 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:42:13.802027 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:42:13.802251 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:42:13.802397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:13.802952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:42:13.802952 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:13.802952 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:42:13.802952 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:42:13.802952 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:13.802952 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:42:13.802952 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:13.802952 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:42:13.803913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:13.803913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:42:13.803913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:42:13.803913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:42:13.803913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:42:13.803913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:42:13.803913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:42:13.803913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:13.803913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:42:13.803913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:13.803913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:42:13.803913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:13.804386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:42:13.804386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:13.804386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:42:13.804386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:13.804386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:42:13.804386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:42:13.804386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:13.804386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:42:13.804386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:13.804386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:42:13.804386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:42:13.804386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:13.804386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:42:13.804704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:13.804704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:42:13.805034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:13.805034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:42:13.805034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:42:13.805034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:42:13.805034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:42:13.805034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:42:13.805034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:42:13.805034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:42:13.805034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:42:13.805034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:42:13.805034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:42:13.805034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:42:13.805034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:42:13.805034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:42:13.805034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:42:13.805034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:42:13.805711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:42:13.805711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:42:13.805711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:13.805711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:42:13.805711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:13.805711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:42:13.805711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:13.805711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:42:13.805711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:13.805711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:42:13.805711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:13.805711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:42:13.805711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:42:13.805711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:13.805711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:42:13.805711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:13.805711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:42:13.805711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:42:13.805711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:13.806204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:42:13.806204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:13.806204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:42:18.630842 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:42:18.631392 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:42:18.631514 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:42:18.632393 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:42:18.670389 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:42:18.670644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:18.671449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:42:18.671449 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:18.671449 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:42:18.671449 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:42:18.671449 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:18.671449 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:42:18.671449 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:18.671449 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:42:18.672613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:18.672613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:42:18.672613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:42:18.672613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:42:18.672613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:42:18.672613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:42:18.672613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:42:18.672613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:18.672613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:42:18.672613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:18.672613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:42:18.672613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:18.672919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:42:18.672919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:18.672919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:42:18.672919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:18.672919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:42:18.672919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:42:18.672919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:18.672919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:42:18.672919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:18.672919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:42:18.672919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:42:18.672919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:18.672919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:42:18.673222 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:18.673222 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:42:18.673814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:18.673814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:42:18.673814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:42:18.673814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:42:18.673814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:42:18.673814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:42:18.673814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:42:18.673814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:42:18.673814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:42:18.673814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:42:18.673814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:42:18.673814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:42:18.673814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:42:18.673814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:42:18.673814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:42:18.673814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:42:18.674154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:42:18.674154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:42:18.674154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:18.674154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:42:18.674154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:18.674154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:42:18.674154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:18.674154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:42:18.674154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:18.674154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:42:18.674154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:18.674154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:42:18.674154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:42:18.674154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:18.674154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:42:18.674154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:18.674154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:42:18.674154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:42:18.674154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:18.674493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:42:18.674493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:18.674493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:42:18.776795 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:42:18.777765 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:42:18.777993 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:42:18.778843 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:42:18.779086 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:42:18.779237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:18.779901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:42:18.779901 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:18.779901 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:42:18.779901 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:42:18.779901 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:18.779901 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:42:18.779901 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:18.779901 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:42:18.781027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:18.781027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:42:18.781027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:42:18.781027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:42:18.781027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:42:18.781027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:42:18.781027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:42:18.781027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:18.781027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:42:18.781027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:18.781027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:42:18.781027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:18.781943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:42:18.781943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:18.781943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:42:18.781943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:18.781943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:42:18.781943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:42:18.781943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:18.781943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:42:18.781943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:18.781943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:42:18.781943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:42:18.781943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:18.781943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:42:18.782399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:18.782399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:42:18.782399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:18.782399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:42:18.782399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:42:18.782399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:42:18.782399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:42:18.782399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:42:18.782399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:42:18.782399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:42:18.782399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:42:18.782399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:42:18.782399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:42:18.782399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:42:18.782399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:42:18.782943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:42:18.782943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:42:18.782943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:42:18.782943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:42:18.782943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:42:18.782943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:18.782943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:42:18.782943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:18.782943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:42:18.782943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:18.782943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:42:18.782943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:18.782943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:42:18.782943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:18.782943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:42:18.782943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:42:18.782943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:18.782943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:42:18.782943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:18.783615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:42:18.783615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:42:18.783615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:18.783615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:42:18.783615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:18.783615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:42:18.784340 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:42:18.784644 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:42:18.784840 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:42:18.785006 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:42:18.785529 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:42:18.785777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:18.786568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:42:18.786568 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:18.786568 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:42:18.786568 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:42:18.786568 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:18.786568 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:42:18.786568 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:18.786568 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:42:18.787767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:18.787767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:42:18.787767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:42:18.787767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:42:18.787767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:18.787767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:42:18.787767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:18.787767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:42:18.787767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:18.787767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:42:18.787767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:18.787767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:42:18.787767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:18.787767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:42:18.787767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:42:18.788154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:18.788154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:42:18.788154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:18.788154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:42:18.788154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:42:18.788154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:18.788154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:42:18.788154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:18.788154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:42:18.790944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:18.790944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:42:18.790944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:42:18.790944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:42:18.790944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:42:18.790944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:42:18.790944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:42:18.790944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:42:18.790944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:42:18.790944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:42:18.790944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:18.790944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:42:18.791370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:18.791370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:42:18.791370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:18.791370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:42:18.791370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:18.791370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:42:18.791370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:18.791370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:42:18.791370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:42:18.791370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:18.791370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:42:18.791370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:18.791370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:42:18.791370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:42:18.791826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:18.791826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:42:18.791826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:18.791826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:42:18.793450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:18.793450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:42:18.793450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:42:18.793450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:42:18.793450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:42:18.793450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:42:18.793450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:42:18.793450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:42:18.793450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:42:18.793450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:42:18.793450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:42:18.793450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:42:18.793450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:42:18.793450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:42:18.793450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:42:18.793450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:42:18.793919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:42:18.793919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:42:18.793919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:42:18.793919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:42:18.793919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:42:18.793919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:18.793919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:42:18.793919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:18.793919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:42:18.793919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:18.793919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:42:18.793919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:18.793919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:42:18.793919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:18.793919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:42:18.793919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:42:18.793919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:18.793919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:42:18.794426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:18.794426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:42:18.794426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:42:18.794426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:18.794426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:42:18.794426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:18.794426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:42:18.795049 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:42:18.795369 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:42:18.795475 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:42:18.795609 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:42:18.796108 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:42:18.796506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:18.799003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:42:18.799003 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:18.799003 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:42:18.799003 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:42:18.799003 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:18.799003 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:42:18.799003 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:18.799003 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:42:18.799003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:18.799003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:42:18.799003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:42:18.799003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:42:18.799003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:18.799003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:42:18.799003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:18.799003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:42:18.799003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:18.799003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:42:18.799877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:18.799877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:42:18.799877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:18.799877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:42:18.799877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:42:18.799877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:18.799877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:42:18.799877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:18.799877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:42:18.799877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:42:18.799877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:18.799877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:42:18.799877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:18.799877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:42:18.799877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:18.800461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:42:18.800461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:42:18.800461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:42:18.800461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:42:18.800461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:42:18.800461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:42:18.800461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:42:18.800461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:42:18.800461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:18.800461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:42:18.800461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:18.800461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:42:18.800461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:18.800862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:42:18.800862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:18.800862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:42:18.800862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:18.800862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:42:18.800862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:42:18.800862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:18.800862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:42:18.800862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:18.800862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:42:18.800862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:42:18.800862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:18.800862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:42:18.801366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:18.801366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:42:18.801366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:18.801366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:42:18.801366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:42:18.801366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:42:18.801366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:42:18.801366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:42:18.801366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:42:18.801366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:42:18.801366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:42:18.801366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:42:18.801366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:42:18.801366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:42:18.801366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:42:18.801976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:42:18.801976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:42:18.801976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:42:18.801976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:42:18.801976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:42:18.801976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:42:18.801976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:42:18.801976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:18.801976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:42:18.801976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:18.801976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:42:18.801976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:18.801976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:42:18.801976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:18.801976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:42:18.801976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:18.801976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:42:18.801976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:42:18.801976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:18.802577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:42:18.802577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:18.802577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:42:18.802577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:42:18.802577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:18.802577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:42:18.802577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:18.802577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:42:18.802577 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:42:18.802577 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:42:18.802577 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:42:18.802956 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:42:18.802956 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:42:18.803021 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:42:18.803184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:18.803677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:42:18.803677 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:18.803677 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:42:18.803677 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:42:18.803677 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:18.803677 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:42:18.803677 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:18.803677 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:42:18.804564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:18.804564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:42:18.804564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:42:18.804564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:42:18.804564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:42:18.804564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:42:18.804564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:42:18.804564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:18.804564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:42:18.804564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:18.804564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:42:18.804564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:18.804968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:42:18.804968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:18.804968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:42:18.804968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:18.804968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:42:18.804968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:42:18.804968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:18.804968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:42:18.804968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:18.804968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:42:18.804968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:42:18.804968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:18.804968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:42:18.805425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:18.805425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:42:18.805677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:18.805677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:42:18.805677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:42:18.805677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:42:18.805677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:42:18.805677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:42:18.805677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:42:18.805677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:42:18.805677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:42:18.805677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:42:18.805677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:42:18.805677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:42:18.805677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:42:18.805677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:42:18.805677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:42:18.805677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:42:18.806318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:42:18.806318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:42:18.806318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:18.806318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:42:18.806318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:18.806318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:42:18.806318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:18.806318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:42:18.806318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:18.806318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:42:18.806318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:18.806318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:42:18.806318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:42:18.806318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:18.806318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:42:18.806318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:18.806318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:42:18.806318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:42:18.806318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:18.806923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:42:18.806923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:18.806923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:42:23.631667 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:42:23.632292 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:42:23.632292 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:42:23.633708 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:42:23.676410 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:42:23.676857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:23.677862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:42:23.677862 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:23.677862 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:42:23.677862 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:42:23.677862 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:23.677862 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:42:23.677862 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:23.677862 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:42:23.679313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.046 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:23.679313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:42:23.679313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:42:23.679313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:42:23.679313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:42:23.679313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:42:23.679313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:42:23.679313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:23.679313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:42:23.679313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:23.679313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:42:23.679313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:23.680125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:42:23.680125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:23.680125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:42:23.680125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:23.680125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:42:23.680125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:42:23.680125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:23.680125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:42:23.680125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:23.680125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:42:23.680125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:42:23.680125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:23.680125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:42:23.680846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:23.680846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:42:23.680846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:23.680846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:42:23.680846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:42:23.680846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:42:23.680846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:42:23.680846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:42:23.680846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:42:23.680846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:42:23.680846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:42:23.680846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:42:23.680846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:42:23.680846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:42:23.680846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:42:23.681652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:42:23.681652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:42:23.681652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:42:23.681652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:42:23.681652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:42:23.681652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:23.681652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:42:23.681652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:23.681652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:42:23.681652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:23.681652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:42:23.681652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:23.681652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:42:23.681652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:23.681652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:42:23.681652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:42:23.681652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:23.681652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:42:23.681652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:23.683263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:42:23.683263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:42:23.683263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:23.683263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:42:23.683263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:23.683263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:42:23.777679 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:42:23.778413 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:42:23.778559 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:42:23.779575 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:42:23.780010 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:42:23.780284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:23.781230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:42:23.781230 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:23.781230 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:42:23.781230 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:42:23.781230 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:23.781230 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:42:23.781230 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:23.781230 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:42:23.782322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:23.782322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:42:23.782322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:42:23.782322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:42:23.782322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:42:23.782322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:42:23.782322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:42:23.782322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:23.782322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:42:23.782322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:23.782322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:42:23.782322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:23.783029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:42:23.783029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:23.783029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:42:23.783029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:23.783029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:42:23.783029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:42:23.783029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:23.783029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:42:23.783029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:23.783029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:42:23.783029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:42:23.783029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:23.783029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:42:23.783606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:23.783606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:42:23.783606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:23.783606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:42:23.783606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:42:23.783606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:42:23.783606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:42:23.783606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:42:23.783606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:42:23.783606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:42:23.783606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:42:23.783606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:42:23.783606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:42:23.783606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:42:23.783606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:42:23.784369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:42:23.784369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:42:23.784369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:42:23.784369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:42:23.784369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:42:23.784369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:23.784369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:42:23.784369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:23.784369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:42:23.784369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:23.784369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:42:23.784369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:23.784369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:42:23.784369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:23.784369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:42:23.784369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:42:23.784369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:23.784369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:42:23.784369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:23.785230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:42:23.785230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:42:23.785230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:23.785230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:42:23.785230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:23.785230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:42:23.785230 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:42:23.785230 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:42:23.785230 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:42:23.785230 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:42:23.786098 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:42:23.786098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:23.786905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:42:23.786905 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:23.786905 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:42:23.786905 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:42:23.786905 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:23.786905 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:42:23.786905 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:23.786905 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:42:23.787989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:23.787989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:42:23.787989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:42:23.787989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:42:23.787989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:23.787989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:42:23.787989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:23.787989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:42:23.787989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:23.787989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:42:23.787989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:23.787989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:42:23.787989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:23.787989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:42:23.787989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:42:23.788569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:23.788569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:42:23.788569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:23.788569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:42:23.788569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:42:23.788569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:23.788569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:42:23.788569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:23.788569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:42:23.788961 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:23.788961 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:42:23.788961 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:42:23.788961 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:42:23.788961 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:42:23.788961 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:42:23.788961 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:42:23.788961 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:42:23.788961 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:42:23.788961 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:42:23.788961 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:23.788961 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:42:23.789317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:23.789317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:42:23.789317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:23.789317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:42:23.789317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:23.789317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:42:23.789317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:23.789317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:42:23.789317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:42:23.789317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:23.789317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:42:23.789317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:23.789317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:42:23.789317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:42:23.789668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:23.789668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:42:23.789668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:23.789668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:42:23.790029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:23.790029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:42:23.790029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:42:23.790029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:42:23.790029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:42:23.790029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:42:23.790029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:42:23.790029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:42:23.790029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:42:23.790029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:42:23.790029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:42:23.790029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:42:23.790029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:42:23.790029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:42:23.790029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:42:23.790029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:42:23.790450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:42:23.790450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:42:23.790450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:42:23.790450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:42:23.790450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:42:23.790450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:23.790450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:42:23.790450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:23.790450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:42:23.790450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:23.790450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:42:23.790450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:23.790450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:42:23.790450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:23.790450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:42:23.790450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:42:23.790450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:23.790450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:42:23.790996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:23.790996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:42:23.790996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:42:23.790996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:23.790996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:42:23.790996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:23.790996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:42:23.795456 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:42:23.795729 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:42:23.795861 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:42:23.796003 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:42:23.796369 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:42:23.796592 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:23.797156 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:42:23.797156 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:23.797156 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:42:23.797156 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:42:23.797156 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:23.797156 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:42:23.797156 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:23.797156 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:42:23.797976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:23.797976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:42:23.797976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:42:23.797976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:42:23.797976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:23.797976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:42:23.797976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:23.797976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:42:23.797976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:23.797976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:42:23.797976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:23.797976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:42:23.797976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:23.797976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:42:23.797976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:42:23.798468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:23.798468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:42:23.798468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:23.798468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:42:23.798468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:42:23.798468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:23.798468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:42:23.798468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:23.798468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:42:23.798925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:23.798925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:42:23.798925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:42:23.798925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:42:23.798925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:42:23.798925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:42:23.798925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:42:23.798925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:42:23.798925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:42:23.798925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:23.798925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:42:23.799312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:23.799312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:42:23.799312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:23.799312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:42:23.799312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:23.799312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:42:23.799312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:23.799312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:42:23.799312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:42:23.799312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:23.799312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:42:23.799312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:23.799312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:42:23.799312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:42:23.800072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:23.800072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:42:23.800072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:23.800072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:42:23.800072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:23.800072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:42:23.800072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:42:23.800072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:42:23.800072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:42:23.800072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:42:23.800072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:42:23.800072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:42:23.800072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:42:23.800072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:42:23.800502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:42:23.800502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:42:23.800502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:42:23.800502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:42:23.800502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:42:23.800502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:42:23.800502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:42:23.800502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:42:23.800502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:42:23.800502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:42:23.800502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:23.800502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:42:23.800502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:23.800502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:42:23.800502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:23.800502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:42:23.800502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:23.800502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:42:23.800502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:23.801222 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:42:23.801222 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:42:23.801222 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:23.801222 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:42:23.801222 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:23.801222 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:42:23.801222 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:42:23.801222 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:23.801222 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:42:23.801222 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:23.801222 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:42:23.802033 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:42:23.802319 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:42:23.802431 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:42:23.802735 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:42:23.803302 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:42:23.803644 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:42:23.803916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:23.804784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:42:23.804784 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:23.804784 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:42:23.804784 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:42:23.804784 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:23.804784 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:42:23.804784 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:23.804784 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:42:23.805973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:23.805973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:42:23.805973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:42:23.805973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:42:23.805973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:42:23.805973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:42:23.805973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:42:23.805973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:23.805973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:42:23.805973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:23.805973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:42:23.805973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:23.806503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:42:23.806503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:23.806503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:42:23.806503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:23.806503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:42:23.806503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:42:23.806503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:23.806503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:42:23.806503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:23.806503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:42:23.806503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:42:23.806503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:23.806503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:42:23.807015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:23.807015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:42:23.807361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:23.807361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:42:23.807361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:42:23.807361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:42:23.807361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:42:23.807361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:42:23.807361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:42:23.807361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:42:23.807361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:42:23.807361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:42:23.807361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:42:23.807361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:42:23.807361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:42:23.807361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:42:23.807361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:42:23.807361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:42:23.808025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:42:23.808025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:42:23.808025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:23.808025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:42:23.808025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:23.808025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:42:23.808025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:23.808025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:42:23.808025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:23.808025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:42:23.808025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:23.808025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:42:23.808025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:42:23.808025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:23.808025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:42:23.808025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:23.808025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:42:23.808025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:42:23.808025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:23.808605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:42:23.808605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:23.808605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:42:28.632192 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:42:28.633061 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:42:28.633061 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:42:28.633786 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:42:28.667638 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:42:28.668249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:28.669146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:42:28.669146 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:28.669146 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:42:28.669146 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:42:28.669146 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:28.669146 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:42:28.669146 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:28.669146 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:42:28.670235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:28.670235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:42:28.670235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:42:28.670235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:42:28.670235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:42:28.670235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:42:28.670235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:42:28.670235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:28.670235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:42:28.670235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:28.670235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:42:28.670235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:28.670822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:42:28.670822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:28.670822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:42:28.670822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:28.670822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:42:28.670822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:42:28.670822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:28.670822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:42:28.670822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:28.670822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:42:28.670822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:42:28.670822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:28.670822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:42:28.671244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:28.671244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:42:28.671244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:28.671244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:42:28.671244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:42:28.671244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:42:28.671244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:42:28.671244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:42:28.671244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:42:28.671244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:42:28.671244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:42:28.671244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:42:28.671244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:42:28.671244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:42:28.671244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:42:28.671796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:42:28.671796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:42:28.671796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:42:28.671796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:42:28.671796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:42:28.671796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:28.671796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:42:28.671796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:28.671796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:42:28.671796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:28.671796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:42:28.671796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:28.671796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:42:28.671796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:28.671796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:42:28.671796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:42:28.671796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:28.671796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:42:28.671796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:28.672335 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:42:28.672335 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:42:28.672335 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:28.672335 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:42:28.672335 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:28.672335 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:42:28.778620 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:42:28.779317 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:42:28.779522 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:42:28.780735 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:42:28.781247 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:42:28.781381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:28.782577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:42:28.782577 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:28.782577 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:42:28.782577 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:42:28.782577 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:28.782577 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:42:28.782577 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:28.782577 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:42:28.784215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:28.784215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:42:28.784215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:42:28.784215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:42:28.784215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:42:28.784215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:42:28.784215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:42:28.784215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:28.784215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:42:28.784215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:28.784215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:42:28.784215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:28.784732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:42:28.784732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:28.784732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:42:28.784732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:28.784732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:42:28.784732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:42:28.784732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:28.784732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:42:28.784732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:28.784732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:42:28.784732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:42:28.784732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:28.784732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:42:28.785250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:28.785250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:42:28.786226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:28.786226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:42:28.786226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:42:28.786226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:42:28.786226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:42:28.786226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:42:28.786226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:42:28.786226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:42:28.786226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:42:28.786226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:42:28.786226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:42:28.786226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:42:28.786226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:42:28.786226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:42:28.786226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:42:28.786226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:42:28.786791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:42:28.786791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:42:28.786791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:28.786791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:42:28.786791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:28.786791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:42:28.786791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:28.786791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:42:28.786791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:28.786791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:42:28.786791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:28.786791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:42:28.786791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:42:28.786791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:28.786791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:42:28.786791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:28.786791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:42:28.786791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:42:28.786791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:28.787430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:42:28.787430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:28.787430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:42:28.787430 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:42:28.787430 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:42:28.787430 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:42:28.787430 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:42:28.787800 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:42:28.787800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:28.788256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:42:28.788256 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:28.788256 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:42:28.788256 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:42:28.788256 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:28.788256 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:42:28.788256 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:28.788256 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:42:28.789100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:28.789100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:42:28.789100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:42:28.789100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:42:28.789100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:28.789100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:42:28.789100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:28.789100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:42:28.789100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:28.789100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:42:28.789100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:28.789100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:42:28.789100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:28.789100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:42:28.789100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:42:28.789849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:28.789849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:42:28.789849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:28.789849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:42:28.789849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:42:28.789849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:28.789849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:42:28.789849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:28.789849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:42:28.790165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:28.790165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:42:28.790165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:42:28.790165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:42:28.790165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:42:28.790165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:42:28.790165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:42:28.790165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:42:28.790165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:42:28.790165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:42:28.790165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:28.790165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:42:28.790594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:28.790594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:42:28.790594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:28.790594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:42:28.790594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:28.790594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:42:28.790594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:28.790594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:42:28.790594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:42:28.790594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:28.790594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:42:28.790594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:28.790594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:42:28.790594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:42:28.791046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:28.791046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:42:28.791046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:28.791046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:42:28.791046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:28.791046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:42:28.791046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:42:28.791046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:42:28.791046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:42:28.791046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:42:28.791046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:42:28.791046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:42:28.791046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:42:28.791046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:42:28.791471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:42:28.791471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:42:28.791471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:42:28.791471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:42:28.791471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:42:28.791471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:42:28.791471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:42:28.791471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:42:28.791471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:42:28.791471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:42:28.791471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:42:28.791471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:28.791471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:42:28.791471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:28.791471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:42:28.791471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:28.791471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:42:28.791471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:28.791471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:42:28.792132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:28.792132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:42:28.792132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:42:28.792132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:28.792132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:42:28.792132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:28.792132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:42:28.792132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:42:28.792132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:28.792132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:42:28.792132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:28.792132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:42:28.796044 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:42:28.796738 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:42:28.796738 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:42:28.796738 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:42:28.797018 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:42:28.797450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:28.798285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:42:28.798285 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:28.798285 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:42:28.798285 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:42:28.798285 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:28.798285 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:42:28.798285 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:28.798285 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:42:28.799212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:28.799212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:42:28.799212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:42:28.799212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:42:28.799212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:28.799212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:42:28.799212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:28.799212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:42:28.799212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:28.799212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:42:28.799212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:28.799212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:42:28.799212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:28.799212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:42:28.799212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:42:28.799667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:28.799667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:42:28.799667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:28.799667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:42:28.799667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:42:28.799667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:28.799667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:42:28.799667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:28.799667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:42:28.799968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:28.799968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:42:28.799968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:42:28.799968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:42:28.799968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:42:28.799968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:42:28.799968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:42:28.799968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:42:28.799968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:42:28.799968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:28.799968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:42:28.800273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:28.800273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:42:28.800273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:28.800273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:42:28.800273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:28.800273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:42:28.800273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:28.800273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:42:28.800273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:42:28.800273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:28.800273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:42:28.800273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:28.800273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:42:28.800273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:42:28.800799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:28.800799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:42:28.800799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:28.800799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:42:28.801017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:28.801017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:42:28.801017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:42:28.801017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:42:28.801017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:42:28.801017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:42:28.801017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:42:28.801017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:42:28.801017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:42:28.801017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:42:28.801017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:42:28.801017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:42:28.801017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:42:28.801017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:42:28.801017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:42:28.801017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:42:28.801017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:42:28.801659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:42:28.801659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:42:28.801659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:42:28.801659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:28.801659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:42:28.801659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:28.801659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:42:28.801659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:28.801659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:42:28.801659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:28.801659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:42:28.801659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:28.801659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:42:28.801659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:42:28.801659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:28.801659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:42:28.801659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:28.801659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:42:28.801659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:42:28.802193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:28.802193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:42:28.802193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:28.802193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:42:28.802372 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:42:28.802689 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:42:28.802819 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:42:28.802969 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:42:28.803417 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:42:28.803542 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:42:28.803704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:28.804258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:42:28.804258 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:28.804258 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:42:28.804258 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:42:28.804258 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:28.804258 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:42:28.804258 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:28.804258 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:42:28.805135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:28.805135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:42:28.805135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:42:28.805135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:42:28.805135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:42:28.805135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:42:28.805135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:42:28.805135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:28.805135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:42:28.805135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:28.805135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:42:28.805135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:28.805739 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:42:28.805739 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:28.805739 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:42:28.805739 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:28.805739 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:42:28.805739 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:42:28.805739 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:28.805739 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:42:28.805739 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:28.805739 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:42:28.805739 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:42:28.805739 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:28.805739 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:42:28.806014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:28.806014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:42:28.806145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:28.806145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:42:28.806145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:42:28.806145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:42:28.806145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:42:28.806145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:42:28.806145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:42:28.806145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:42:28.806145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:42:28.806145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:42:28.806145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:42:28.806145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:42:28.806145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:42:28.806145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:42:28.806145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:42:28.806145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:42:28.806436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:42:28.806436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:42:28.806436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:28.806436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:42:28.806436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:28.806436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:42:28.806436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:28.806436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:42:28.806436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:28.806436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:42:28.806436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:28.806436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:42:28.806436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:42:28.806436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:28.806436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:42:28.806436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:28.806436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:42:28.806436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:42:28.806436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:28.806801 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:42:28.806801 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:28.806801 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:42:33.632811 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:42:33.633820 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:42:33.633820 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:42:33.635544 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:42:33.678076 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:42:33.678253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:33.679025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:42:33.679025 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:33.679025 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:42:33.679025 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:42:33.679025 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:33.679025 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:42:33.679025 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:33.679025 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:42:33.679974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.045 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:33.679974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:42:33.679974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:42:33.679974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:42:33.679974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:42:33.679974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:42:33.679974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:42:33.679974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:33.679974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:42:33.679974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:33.679974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:42:33.679974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:33.680296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:42:33.680296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:33.680296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:42:33.680296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:33.680296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:42:33.680296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:42:33.680296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:33.680296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:42:33.680296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:33.680296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:42:33.680296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:42:33.680296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:33.680296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:42:33.680588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:33.680588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:42:33.680756 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:33.680756 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:42:33.680756 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:42:33.680756 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:42:33.680756 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:42:33.680756 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:42:33.680756 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:42:33.680756 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:42:33.680756 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:42:33.680756 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:42:33.680756 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:42:33.680756 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:42:33.680756 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:42:33.680756 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:42:33.680756 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:42:33.680756 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:42:33.681131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:42:33.681131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:42:33.681131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:33.681131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:42:33.681131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:33.681131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:42:33.681131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:33.681131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:42:33.681131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:33.681131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:42:33.681131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:33.681131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:42:33.681131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:42:33.681131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:33.681131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:42:33.681131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:33.681131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:42:33.681131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:42:33.681131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:33.681528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:42:33.681528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:33.681528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:42:33.807352 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:42:33.807352 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:42:33.807352 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:42:33.807352 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:42:33.807352 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:42:33.807352 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:33.807556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:42:33.807556 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:33.807556 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:42:33.807556 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:42:33.807556 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:33.807556 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:42:33.807556 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:33.807556 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:42:33.807556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:33.807556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:42:33.807556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:42:33.807556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:42:33.807556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:42:33.807556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:42:33.807556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:42:33.808182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:33.808182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:42:33.808182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:33.808182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:42:33.808182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:33.808182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:42:33.808182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:33.808182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:42:33.808182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:33.808182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:42:33.808182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:42:33.808182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:33.808182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:42:33.808182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:33.808466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:42:33.808466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:42:33.808466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:33.808466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:42:33.808466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:33.808466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:42:33.808466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:33.808466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:42:33.808466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:42:33.808466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:42:33.808466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:42:33.808466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:42:33.808466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:42:33.808466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:42:33.808733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:42:33.808733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:42:33.808733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:42:33.808733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:42:33.808733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:42:33.808733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:42:33.808733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:42:33.808733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:42:33.808733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:42:33.808733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:42:33.808733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:33.808733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:42:33.808733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:33.808733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:42:33.808733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:33.808733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:42:33.808733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:33.808733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:42:33.808733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:33.808733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:42:33.809118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:42:33.809118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:33.809118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:42:33.809118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:33.809118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:42:33.809118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:42:33.809118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:33.809118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:42:33.809118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:33.809118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:42:33.809118 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:42:33.809118 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:42:33.809349 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:42:33.809349 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:42:33.809349 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:42:33.809349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:33.809349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:42:33.809349 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:33.809349 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:42:33.809349 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:42:33.809349 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:33.809349 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:42:33.809349 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:33.809349 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:42:33.809590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:33.809590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:42:33.809590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:42:33.809590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:42:33.809590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:33.809590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:42:33.809590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:33.809590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:42:33.809590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:33.809590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:42:33.809590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:33.809590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:42:33.809590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:33.809590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:42:33.809590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:42:33.809924 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:33.809924 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:42:33.809924 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:33.809924 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:42:33.809924 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:42:33.809924 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:33.809924 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:42:33.809924 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:33.809924 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:42:33.809924 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:33.809924 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:42:33.809924 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:42:33.809924 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:42:33.809924 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:42:33.812404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:42:33.812404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:42:33.812404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:42:33.812404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:42:33.812404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:42:33.812404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:33.812404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:42:33.812404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:33.812404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:42:33.812404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:33.812404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:42:33.812404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:33.812404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:42:33.812740 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:33.812740 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:42:33.812740 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:42:33.812740 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:33.812740 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:42:33.812740 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:33.812740 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:42:33.812740 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:42:33.812740 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:33.812740 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:42:33.812740 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:33.812740 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:42:33.813267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:33.813267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:42:33.813267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:42:33.813267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:42:33.813267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:42:33.813267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:42:33.813267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:42:33.813267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:42:33.813267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:42:33.813267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:42:33.813267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:42:33.813267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:42:33.813267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:42:33.813267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:42:33.813267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:42:33.813267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:42:33.813559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:42:33.813559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:42:33.813559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:42:33.813559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:42:33.813559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:42:33.813559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:33.813559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:42:33.813559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:33.813559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:42:33.813559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:33.813559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:42:33.813559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:33.813559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:42:33.813559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:33.813559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:42:33.813559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:42:33.813559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:33.813559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:42:33.813914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:33.813914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:42:33.813914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:42:33.813914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:33.813914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:42:33.813914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:33.813914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:42:33.813914 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:42:33.813914 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:42:33.813914 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:42:33.814118 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:42:33.814118 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:42:33.814118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:33.814118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:42:33.814118 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:33.814118 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:42:33.814118 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:42:33.814118 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:33.814118 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:42:33.814118 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:33.814118 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:42:33.814118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:33.814340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:42:33.814340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:42:33.814340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:42:33.814340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:33.814340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:42:33.814340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:33.814340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:42:33.814340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:33.814340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:42:33.814340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:33.814340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:42:33.814340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:33.814340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:42:33.814340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:42:33.814340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:33.814340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:42:33.814340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:33.814644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:42:33.814644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:42:33.814644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:33.814644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:42:33.814644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:33.814644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:42:33.814644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:33.814644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:42:33.814644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:42:33.814644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:42:33.814644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:42:33.814644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:42:33.814644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:42:33.814934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:42:33.814934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:42:33.814934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:33.814934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:42:33.814934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:33.814934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:42:33.814934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:33.814934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:42:33.814934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:33.814934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:42:33.814934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:33.814934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:42:33.814934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:42:33.815205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:33.815205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:42:33.815205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:33.815205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:42:33.815205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:42:33.815205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:33.815205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:42:33.815205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:33.815205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:42:33.815205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:33.815205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:42:33.815205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:42:33.815438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:42:33.815438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:42:33.815438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:42:33.815438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:42:33.815438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:42:33.815438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:42:33.815438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:42:33.815438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:42:33.815438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:42:33.815438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:42:33.815438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:42:33.815438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:42:33.815438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:42:33.815438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:42:33.815438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:42:33.815438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:42:33.815438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:42:33.815438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:33.815438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:42:33.815814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:33.815814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:42:33.815814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:33.815814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:42:33.815814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:33.815814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:42:33.815814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:33.815814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:42:33.815814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:42:33.815814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:33.815814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:42:33.815814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:33.815814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:42:33.815814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:42:33.815814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:33.815814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:42:33.815814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:33.815814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:42:33.816127 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:42:33.816127 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:42:33.816127 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:42:33.816127 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:42:33.816127 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:42:33.816127 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:42:33.816288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:33.816288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:42:33.816288 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:33.816288 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:42:33.816288 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:42:33.816288 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:33.816288 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:42:33.816288 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:33.816288 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:42:33.816288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:33.816288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:42:33.816288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:42:33.816288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:42:33.816288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:42:33.816288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:42:33.816566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:42:33.816566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:33.816566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:42:33.816566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:33.816566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:42:33.816566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:33.816566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:42:33.816566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:33.816566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:42:33.816566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:33.816566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:42:33.816566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:42:33.816566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:33.816566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:42:33.816842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:33.816842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:42:33.816842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:42:33.816842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:33.816842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:42:33.816842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:33.816842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:42:33.816842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:33.816842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:42:33.816842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:42:33.816842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:42:33.816842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:42:33.816842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:42:33.817078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:42:33.817078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:42:33.817078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:42:33.817078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:42:33.817078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:42:33.817078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:42:33.817078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:42:33.817078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:42:33.817078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:42:33.817078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:42:33.817078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:42:33.817078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:42:33.817078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:33.817078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:42:33.817078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:33.817078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:42:33.817078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:33.817078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:42:33.817078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:33.817078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:42:33.817432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:33.817432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:42:33.817432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:42:33.817432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:33.817432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:42:33.817432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:33.817432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:42:33.817432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:42:33.817432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:33.817432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:42:33.817432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:33.817432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:42:38.633460 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:42:38.635346 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:42:38.635494 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:42:38.636402 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:42:38.673042 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:42:38.673382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:38.674306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:42:38.674306 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:38.674306 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:42:38.674306 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:42:38.674306 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:38.674306 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:42:38.674306 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:38.674306 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:42:38.675246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:38.675246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:42:38.675246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:42:38.675246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:42:38.675246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:42:38.675246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:42:38.675246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:42:38.675246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:38.675246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:42:38.675246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:38.675246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:42:38.675246 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:38.675557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:42:38.675557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:38.675557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:42:38.675557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:38.675557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:42:38.675557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:42:38.675557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:38.675557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:42:38.675557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:38.675557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:42:38.675557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:42:38.675557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:38.675557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:42:38.675880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:38.675880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:42:38.676108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:38.676108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:42:38.676108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:42:38.676108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:42:38.676108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:42:38.676108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:42:38.676108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:42:38.676108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:42:38.676108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:42:38.676108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:42:38.676108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:42:38.676108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:42:38.676108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:42:38.676108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:42:38.676108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:42:38.676108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:42:38.676483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:42:38.676483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:42:38.676483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:38.676483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:42:38.676483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:38.676483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:42:38.676483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:38.676483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:42:38.676483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:38.676483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:42:38.676483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:38.676483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:42:38.676483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:42:38.676483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:38.676483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:42:38.676483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:38.676483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:42:38.676483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:42:38.676483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:38.677069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:42:38.677069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:38.677069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:42:38.780319 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:42:38.781299 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:42:38.781452 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:42:38.782838 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:42:38.783264 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:42:38.783642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:38.784780 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:42:38.784780 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:38.784780 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:42:38.784780 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:42:38.784780 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:38.784780 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:42:38.784780 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:38.784780 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:42:38.786672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:38.786672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:42:38.786672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:42:38.786672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:42:38.786672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:42:38.786672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:42:38.786672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:42:38.786672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:38.786672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:42:38.786672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:38.786672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:42:38.786672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:38.787272 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:42:38.787272 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:38.787272 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:42:38.787272 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:38.787272 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:42:38.787272 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:42:38.787272 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:38.787272 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:42:38.787272 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:38.787272 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:42:38.787272 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:42:38.787272 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:38.787272 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:42:38.787634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:38.787634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:42:38.787854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:38.787854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:42:38.787854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:42:38.787854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:42:38.787854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:42:38.787854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:42:38.787854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:42:38.787854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:42:38.787854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:42:38.787854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:42:38.787854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:42:38.787854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:42:38.787854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:42:38.787854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:42:38.787854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:42:38.787854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:42:38.788331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:42:38.788331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:42:38.788331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:38.788331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:42:38.788331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:38.788331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:42:38.788331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:38.788331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:42:38.788331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:38.788331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:42:38.788331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:38.788331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:42:38.788331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:42:38.788331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:38.788331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:42:38.788331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:38.788331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:42:38.788331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:42:38.788331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:38.788923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:42:38.788923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:38.788923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:42:38.788923 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:42:38.788923 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:42:38.788923 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:42:38.789199 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:42:38.790234 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:42:38.790709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:38.791701 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:42:38.791701 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:38.791701 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:42:38.791701 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:42:38.791701 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:38.791701 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:42:38.791701 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:38.791701 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:42:38.792675 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:38.792675 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:42:38.792675 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:42:38.792675 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:42:38.792675 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:38.792675 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:42:38.792675 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:38.792675 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:42:38.792675 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:38.792675 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:42:38.792675 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:38.792675 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:42:38.792675 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:38.792675 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:42:38.792675 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:42:38.793186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:38.793186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:42:38.793186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:38.793186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:42:38.793186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:42:38.793186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:38.793186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:42:38.793186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:38.793186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:42:38.793830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:38.793830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:42:38.793830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:42:38.793830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:42:38.793830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:42:38.793830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:42:38.793830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:42:38.793830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:42:38.793830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:42:38.793830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:42:38.793830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:38.793830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:42:38.794296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:38.794296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:42:38.794296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:38.794296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:42:38.794296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:38.794296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:42:38.794296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:38.794296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:42:38.794296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:42:38.794296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:38.794296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:42:38.794296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:38.794296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:42:38.794296 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:42:38.794678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:38.794678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:42:38.794678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:38.794678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:42:38.795226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:38.795226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:42:38.795226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:42:38.795226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:42:38.795226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:42:38.795226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:42:38.795226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:42:38.795226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:42:38.795226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:42:38.795226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:42:38.795226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:42:38.795226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:42:38.795226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:42:38.795226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:42:38.795226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:42:38.795226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:42:38.795670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:42:38.795670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:42:38.795670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:42:38.795670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:42:38.795670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:42:38.795670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:38.795670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:42:38.795670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:38.795670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:42:38.795670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:38.795670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:42:38.795670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:38.795670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:42:38.795670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:38.795670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:42:38.795670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:42:38.795670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:38.795670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:42:38.796226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:38.796226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:42:38.796226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:42:38.796226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:38.796226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:42:38.796226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:38.796226 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:42:38.796778 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:42:38.797169 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:42:38.797330 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:42:38.797529 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:42:38.797959 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:42:38.798328 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:38.799069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:42:38.799069 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:38.799069 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:42:38.799069 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:42:38.799069 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:38.799069 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:42:38.799069 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:38.799069 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:42:38.800285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:38.800285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:42:38.800285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:42:38.800285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:42:38.800285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:38.800285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:42:38.800285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:38.800285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:42:38.800285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:38.800285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:42:38.800285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:38.800285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:42:38.800285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:38.800285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:42:38.800285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:42:38.800698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:38.800698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:42:38.800698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:38.800698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:42:38.800698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:42:38.800698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:38.800698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:42:38.800698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:38.800698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:42:38.801613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:38.801613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:42:38.801613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:42:38.801613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:42:38.801613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:42:38.801613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:42:38.801613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:42:38.801613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:42:38.801613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:42:38.801613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:38.801613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:42:38.801982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:38.801982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:42:38.801982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:38.801982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:42:38.801982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:38.801982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:42:38.801982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:38.801982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:42:38.801982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:42:38.801982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:38.801982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:42:38.801982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:38.801982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:42:38.801982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:42:38.802405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:38.802405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:42:38.802405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:38.802405 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:42:38.803214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:38.803214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:42:38.803214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:42:38.803214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:42:38.803214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:42:38.803214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:42:38.803214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:42:38.803214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:42:38.803214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:42:38.803214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:42:38.803214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:42:38.803214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:42:38.803214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:42:38.803214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:42:38.803214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:42:38.803214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:42:38.803214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:42:38.803659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:42:38.803659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:42:38.803659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:42:38.803659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:38.803659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:42:38.803659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:38.803659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:42:38.803659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:38.803659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:42:38.803659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:38.803659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:42:38.803659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:38.803659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:42:38.803659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:42:38.803659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:38.803659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:42:38.803659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:38.803659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:42:38.803659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:42:38.804173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:38.804173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:42:38.804173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:38.804173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:42:38.808148 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:42:38.808553 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:42:38.808676 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:42:38.808902 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:42:38.809393 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:42:38.809822 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:42:38.810058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:38.811007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:42:38.811007 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:38.811007 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:42:38.811007 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:42:38.811007 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:38.811007 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:42:38.811007 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:38.811007 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:42:38.812999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:38.812999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:42:38.812999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:42:38.812999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:42:38.812999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:42:38.812999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:42:38.812999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:42:38.812999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:38.812999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:42:38.812999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:38.812999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:42:38.812999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:38.813370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:42:38.813370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:38.813370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:42:38.813370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:38.813370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:42:38.813370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:42:38.813370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:38.813370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:42:38.813370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:38.813370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:42:38.813370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:42:38.813370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:38.813370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:42:38.813683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:38.813683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:42:38.815018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:38.815018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:42:38.815018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:42:38.815018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:42:38.815018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:42:38.815018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:42:38.815018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:42:38.815018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:42:38.815018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:42:38.815018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:42:38.815018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:42:38.815018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:42:38.815018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:42:38.815018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:42:38.815018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:42:38.815018 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:42:38.815466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:42:38.815466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:42:38.815466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:38.815466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:42:38.815466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:38.815466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:42:38.815466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:38.815466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:42:38.815466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:38.815466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:42:38.815466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:38.815466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:42:38.815466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:42:38.815466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:38.815466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:42:38.815466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:38.815466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:42:38.815466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:42:38.815466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:38.815974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:42:38.815974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:38.815974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:42:43.634031 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:42:43.634722 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:42:43.634722 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:42:43.635672 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:42:43.670022 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:42:43.670484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:43.671618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:42:43.671618 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:43.671618 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:42:43.671618 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:42:43.671618 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:43.671618 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:42:43.671618 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:43.671618 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:42:43.672534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:43.672534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:42:43.672534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:42:43.672534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:42:43.672534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:42:43.672534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:42:43.672534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:42:43.672534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:43.672534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:42:43.672534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:43.672534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:42:43.672534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:43.673417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:42:43.673417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:43.673417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:42:43.673417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:43.673417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:42:43.673417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:42:43.673417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:43.673417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:42:43.673417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:43.673417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:42:43.673417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:42:43.673417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:43.673417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:42:43.673672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:43.673672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:42:43.673672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:43.673672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:42:43.673672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:42:43.673672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:42:43.673672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:42:43.673672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:42:43.673672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:42:43.673672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:42:43.673672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:42:43.673672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:42:43.673672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:42:43.673672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:42:43.673672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:42:43.674083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:42:43.674083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:42:43.674083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:42:43.674083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:42:43.674083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:42:43.674083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:43.674083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:42:43.674083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:43.674083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:42:43.674083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:43.674083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:42:43.674083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:43.674083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:42:43.674083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:43.674083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:42:43.674083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:42:43.674083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:43.674083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:42:43.674083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:43.674445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:42:43.674445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:42:43.674445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:43.674445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:42:43.674445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:43.674445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:42:43.781102 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:42:43.781709 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:42:43.781985 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:42:43.783342 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:42:43.783718 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:42:43.784004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:43.784972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:42:43.784972 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:43.784972 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:42:43.784972 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:42:43.784972 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:43.784972 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:42:43.784972 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:43.784972 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:42:43.786818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:43.786818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:42:43.786818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:42:43.786818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:42:43.786818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:42:43.786818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:42:43.786818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:42:43.786818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:43.786818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:42:43.786818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:43.786818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:42:43.786818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:43.787850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:42:43.787850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:43.787850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:42:43.787850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:43.787850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:42:43.787850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:42:43.787850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:43.787850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:42:43.787850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:43.787850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:42:43.787850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:42:43.787850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:43.787850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:42:43.788270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:43.788270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:42:43.788771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:43.788771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:42:43.788771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:42:43.788771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:42:43.788771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:42:43.788771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:42:43.788771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:42:43.788771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:42:43.788771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:42:43.788771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:42:43.788771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:42:43.788771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:42:43.788771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:42:43.788771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:42:43.788771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:42:43.788771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:42:43.789337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:42:43.789337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:42:43.789337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:43.789337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:42:43.789337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:43.789337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:42:43.789337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:43.789337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:42:43.789337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:43.789337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:42:43.789337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:43.789337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:42:43.789337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:42:43.789337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:43.789337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:42:43.789337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:43.789337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:42:43.789337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:42:43.789337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:43.789973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:42:43.789973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:43.789973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:42:43.789973 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:42:43.789973 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:42:43.789973 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:42:43.789973 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:42:43.790404 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:42:43.790718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:43.791570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:42:43.791570 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:43.791570 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:42:43.791570 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:42:43.791570 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:43.791570 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:42:43.791570 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:43.791570 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:42:43.792337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:43.792337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:42:43.792337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:42:43.792337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:42:43.792337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:43.792337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:42:43.792337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:43.792337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:42:43.792337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:43.792337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:42:43.792337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:43.792337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:42:43.792337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:43.792337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:42:43.792337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:42:43.792734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:43.792734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:42:43.792734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:43.792734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:42:43.792734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:42:43.792734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:43.792734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:42:43.792734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:43.792734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:42:43.793220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:43.793220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:42:43.793220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:42:43.793220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:42:43.793220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:42:43.793220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:42:43.793220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:42:43.793220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:42:43.793220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:42:43.793220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:42:43.793220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:43.793220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:42:43.793569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:43.793569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:42:43.793569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:43.793569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:42:43.793569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:43.793569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:42:43.793569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:43.793569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:42:43.793569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:42:43.793569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:43.793569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:42:43.793569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:43.793569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:42:43.793569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:42:43.793936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:43.793936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:42:43.793936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:43.793936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:42:43.794393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:43.794393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:42:43.794393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:42:43.794393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:42:43.794393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:42:43.794393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:42:43.794393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:42:43.794393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:42:43.794393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:42:43.794393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:42:43.794393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:42:43.794393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:42:43.794393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:42:43.794393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:42:43.794393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:42:43.794393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:42:43.794869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:42:43.794869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:42:43.794869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:42:43.794869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:42:43.794869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:42:43.794869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:43.794869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:42:43.794869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:43.794869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:42:43.794869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:43.794869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:42:43.794869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:43.794869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:42:43.794869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:43.794869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:42:43.794869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:42:43.794869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:43.794869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:42:43.795419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:43.795419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:42:43.795419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:42:43.795419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:43.795419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:42:43.795419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:43.795419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:42:43.797232 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:42:43.797514 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:42:43.797597 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:42:43.797737 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:42:43.798152 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:42:43.798389 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:43.798944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:42:43.798944 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:43.798944 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:42:43.798944 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:42:43.798944 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:43.798944 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:42:43.798944 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:43.798944 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:42:43.799941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:43.799941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:42:43.799941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:42:43.799941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:42:43.799941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:43.799941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:42:43.799941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:43.799941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:42:43.799941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:43.799941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:42:43.799941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:43.799941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:42:43.799941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:43.799941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:42:43.799941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:42:43.800321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:43.800321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:42:43.800321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:43.800321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:42:43.800321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:42:43.800321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:43.800321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:42:43.800321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:43.800321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:42:43.800882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:43.800882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:42:43.800882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:42:43.800882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:42:43.800882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:42:43.800882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:42:43.800882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:42:43.800882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:42:43.800882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:42:43.800882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:43.800882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:42:43.801236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:43.801236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:42:43.801236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:43.801236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:42:43.801236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:43.801236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:42:43.801236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:43.801236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:42:43.801236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:42:43.801236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:43.801236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:42:43.801236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:43.801236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:42:43.801236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:42:43.801636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:43.801636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:42:43.801636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:43.801636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:42:43.801958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:43.801958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:42:43.801958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:42:43.801958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:42:43.801958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:42:43.801958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:42:43.801958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:42:43.801958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:42:43.801958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:42:43.801958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:42:43.801958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:42:43.801958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:42:43.801958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:42:43.801958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:42:43.801958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:42:43.801958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:42:43.801958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:42:43.802473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:42:43.802473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:42:43.802473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:42:43.802473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:43.802473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:42:43.802473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:43.802473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:42:43.802473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:43.802473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:42:43.802473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:43.802473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:42:43.802473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:43.802473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:42:43.802473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:42:43.802473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:43.802473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:42:43.802473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:43.802473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:42:43.802473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:42:43.803020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:43.803020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:42:43.803020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:43.803020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:42:43.808652 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:42:43.809010 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:42:43.809116 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:42:43.809291 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:42:43.809767 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:42:43.810204 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:42:43.810442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:43.811714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:42:43.811714 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:43.811714 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:42:43.811714 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:42:43.811714 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:43.811714 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:42:43.811714 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:43.811714 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:42:43.813703 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:43.813703 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:42:43.813703 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:42:43.813703 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:42:43.813703 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:42:43.813703 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:42:43.813703 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:42:43.813703 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:43.813703 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:42:43.813703 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:43.813703 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:42:43.813703 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:43.814123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:42:43.814123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:43.814123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:42:43.814123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:43.814123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:42:43.814123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:42:43.814123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:43.814123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:42:43.814123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:43.814123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:42:43.814123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:42:43.814123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:43.814123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:42:43.814529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:43.814529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:42:43.815851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:43.815851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:42:43.815851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:42:43.815851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:42:43.815851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:42:43.815851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:42:43.815851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:42:43.815851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:42:43.815851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:42:43.815851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:42:43.815851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:42:43.815851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:42:43.815851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:42:43.815851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:42:43.815851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:42:43.815851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:42:43.816329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:42:43.816329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:42:43.816329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:43.816329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:42:43.816329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:43.816329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:42:43.816329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:43.816329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:42:43.816329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:43.816329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:42:43.816329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:43.816329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:42:43.816329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:42:43.816329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:43.816329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:42:43.816329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:43.816329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:42:43.816329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:42:43.816329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:43.816857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:42:43.816857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:43.816857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:42:48.634990 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:42:48.635997 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:42:48.635997 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:42:48.637235 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:42:48.688377 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:42:48.688591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:48.692063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:42:48.692063 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:48.692063 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:42:48.692063 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:42:48.692063 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:48.692063 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:42:48.692063 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:48.692063 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:42:48.692063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.054 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:48.692063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:42:48.692063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:42:48.692063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:42:48.692063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:42:48.692063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:42:48.692063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:42:48.692618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:48.692618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:42:48.692618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:48.692618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:42:48.692618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:48.692618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:42:48.692618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:48.692618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:42:48.692618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:48.692618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:42:48.692618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:42:48.692618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:48.692618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:42:48.692618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:48.693067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:42:48.693067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:42:48.693067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:48.693067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:42:48.693067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:48.693067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:42:48.693067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:48.693067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:42:48.693067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:42:48.693067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:42:48.693067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:42:48.693067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:42:48.693067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:42:48.693067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:42:48.693482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:42:48.693482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:42:48.693482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:42:48.693482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:42:48.693482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:42:48.693482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:42:48.693482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:42:48.693482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:42:48.693482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:42:48.693482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:42:48.693482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:48.693482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:42:48.693482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:48.693482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:42:48.693482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:48.693482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:42:48.693482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:48.693482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:42:48.693482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:48.693482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:42:48.693482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:42:48.694088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:48.694088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:42:48.694088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:48.694088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:42:48.694088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:42:48.694088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:48.694088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:42:48.694088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:48.694088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:42:48.781959 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:42:48.783382 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:42:48.783382 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:42:48.784027 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:42:48.784314 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:42:48.784515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:48.786903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:42:48.786903 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:48.786903 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:42:48.786903 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:42:48.786903 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:48.786903 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:42:48.786903 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:48.786903 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:42:48.786903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:48.786903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:42:48.786903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:42:48.786903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:42:48.786903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:42:48.786903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:42:48.786903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:42:48.787444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:48.787444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:42:48.787444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:48.787444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:42:48.787444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:48.787444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:42:48.787444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:48.787444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:42:48.787444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:48.787444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:42:48.787444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:42:48.787444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:48.787444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:42:48.787444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:48.787710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:42:48.787710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:42:48.787710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:48.787710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:42:48.787710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:48.787710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:42:48.787857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:48.787857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:42:48.787857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:42:48.787857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:42:48.787857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:42:48.787857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:42:48.787857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:42:48.787857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:42:48.787857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:42:48.787857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:42:48.787857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:42:48.787857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:42:48.787857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:42:48.787857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:42:48.787857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:42:48.787857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:42:48.788165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:42:48.788165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:42:48.788165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:48.788165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:42:48.788165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:48.788165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:42:48.788165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:48.788165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:42:48.788165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:48.788165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:42:48.788165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:48.788165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:42:48.788165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:42:48.788165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:48.788165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:42:48.788165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:48.788165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:42:48.788165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:42:48.788165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:48.788538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:42:48.788538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:48.788538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:42:48.789687 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:42:48.790002 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:42:48.790146 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:42:48.790272 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:42:48.791018 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:42:48.791434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:48.792445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:42:48.792445 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:48.792445 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:42:48.792445 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:42:48.792445 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:48.792445 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:42:48.792445 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:48.792445 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:42:48.793256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:48.793256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:42:48.793256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:42:48.793256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:42:48.793256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:48.793256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:42:48.793256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:48.793256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:42:48.793256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:48.793256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:42:48.793256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:48.793256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:42:48.793256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:48.793256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:42:48.793256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:42:48.793628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:48.793628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:42:48.793628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:48.793628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:42:48.793628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:42:48.793628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:48.793628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:42:48.793628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:48.793628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:42:48.794223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:48.794223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:42:48.794223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:42:48.794223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:42:48.794223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:42:48.794223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:42:48.794223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:42:48.794223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:42:48.794223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:42:48.794223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:42:48.794223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:48.794223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:42:48.794482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:48.794482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:42:48.794482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:48.794482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:42:48.794482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:48.794482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:42:48.794482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:48.794482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:42:48.794482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:42:48.794482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:48.794482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:42:48.794482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:48.794482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:42:48.794482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:42:48.794728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:48.794728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:42:48.794728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:48.794728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:42:48.795434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:48.795434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:42:48.795434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:42:48.795434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:42:48.795434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:42:48.795434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:42:48.795434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:42:48.795434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:42:48.795434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:42:48.795434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:42:48.795434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:42:48.795434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:42:48.795434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:42:48.795434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:42:48.795434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:42:48.795434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:42:48.795729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:42:48.795729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:42:48.795729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:42:48.795729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:42:48.795729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:42:48.795729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:48.795729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:42:48.795729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:48.795729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:42:48.795729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:48.795729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:42:48.795729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:48.795729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:42:48.795729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:48.795729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:42:48.795729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:42:48.795729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:48.795729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:42:48.796207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:48.796207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:42:48.796207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:42:48.796207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:48.796207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:42:48.796207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:48.796207 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:42:48.797657 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:42:48.797952 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:42:48.798066 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:42:48.798227 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:42:48.798766 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:42:48.799116 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:48.799731 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:42:48.799731 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:48.799731 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:42:48.799731 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:42:48.799731 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:48.799731 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:42:48.799731 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:48.799731 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:42:48.800512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:48.800512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:42:48.800512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:42:48.800512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:42:48.800512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:48.800512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:42:48.800512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:48.800512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:42:48.800512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:48.800512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:42:48.800512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:48.800512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:42:48.800512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:48.800512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:42:48.800512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:42:48.800977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:48.800977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:42:48.800977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:48.800977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:42:48.800977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:42:48.800977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:48.800977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:42:48.800977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:48.800977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:42:48.801568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:48.801568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:42:48.801568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:42:48.801568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:42:48.801568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:42:48.801568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:42:48.801568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:42:48.801568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:42:48.801568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:42:48.801568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:48.801568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:42:48.802026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:48.802026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:42:48.802026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:48.802026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:42:48.802026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:48.802026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:42:48.802026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:48.802026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:42:48.802026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:42:48.802026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:48.802026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:42:48.802026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:48.802026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:42:48.802026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:42:48.802414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:48.802414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:42:48.802414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:48.802414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:42:48.803155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:48.803155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:42:48.803155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:42:48.803155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:42:48.803155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:42:48.803155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:42:48.803155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:42:48.803155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:42:48.803155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:42:48.803155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:42:48.803155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:42:48.803155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:42:48.803155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:42:48.803155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:42:48.803155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:42:48.803155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:42:48.803155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:42:48.803682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:42:48.803682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:42:48.803682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:42:48.803682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:48.803682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:42:48.803682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:48.803682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:42:48.803682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:48.803682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:42:48.803682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:48.803682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:42:48.803682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:48.803682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:42:48.803682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:42:48.803682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:48.803682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:42:48.803682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:48.803682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:42:48.803682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:42:48.804390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:48.804390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:42:48.804390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:48.804390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:42:48.809308 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:42:48.809595 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:42:48.809710 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:42:48.809905 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:42:48.810302 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:42:48.810567 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:42:48.810727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:48.811493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:42:48.811493 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:48.811493 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:42:48.811493 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:42:48.811493 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:48.811493 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:42:48.811493 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:48.811493 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:42:48.812441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:48.812441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:42:48.812441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:42:48.812441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:42:48.812441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:42:48.812441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:42:48.812441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:42:48.812441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:48.812441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:42:48.812441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:48.812441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:42:48.812441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:48.813223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:42:48.813223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:48.813223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:42:48.813223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:48.813223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:42:48.813223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:42:48.813223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:48.813223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:42:48.813223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:48.813223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:42:48.813223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:42:48.813223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:48.813223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:42:48.813812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:48.813812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:42:48.813812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:48.813812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:42:48.813812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:42:48.813812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:42:48.813812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:42:48.813812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:42:48.813812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:42:48.813812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:42:48.813812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:42:48.813812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:42:48.813812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:42:48.813812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:42:48.813812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:42:48.814477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:42:48.814477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:42:48.814477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:42:48.814477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:42:48.814477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:42:48.814477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:48.814477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:42:48.814477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:48.814477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:42:48.814477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:48.814477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:42:48.814477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:48.814477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:42:48.814477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:48.814477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:42:48.814477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:42:48.814477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:48.814477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:42:48.814477 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:48.815345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:42:48.815345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:42:48.815345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:48.815345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:42:48.815345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:48.815345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:42:53.635882 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:42:53.637226 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:42:53.637493 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:42:53.638869 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:42:53.677838 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:42:53.678179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:53.678949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:42:53.678949 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:53.678949 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:42:53.678949 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:42:53.678949 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:53.678949 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:42:53.678949 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:53.678949 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:42:53.680031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:53.680031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:42:53.680031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:42:53.680031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:42:53.680031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:42:53.680031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:42:53.680031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:42:53.680031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:53.680031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:42:53.680031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:53.680031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:42:53.680031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:53.680451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:42:53.680451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:53.680451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:42:53.680451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:53.680451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:42:53.680451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:42:53.680451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:53.680451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:42:53.680451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:53.680451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:42:53.680451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:42:53.680451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:53.680451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:42:53.680861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:53.680861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:42:53.681123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:53.681123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:42:53.681123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:42:53.681123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:42:53.681123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:42:53.681123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:42:53.681123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:42:53.681123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:42:53.681123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:42:53.681123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:42:53.681123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:42:53.681123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:42:53.681123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:42:53.681123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:42:53.681123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:42:53.681123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:42:53.681654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:42:53.681654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:42:53.681654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:53.681654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:42:53.681654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:53.681654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:42:53.681654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:53.681654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:42:53.681654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:53.681654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:42:53.681654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:53.681654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:42:53.681654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:42:53.681654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:53.681654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:42:53.681654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:53.681654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:42:53.681654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:42:53.681654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:53.682239 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:42:53.682239 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:53.682239 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:42:53.782979 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:42:53.783713 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:42:53.783931 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:42:53.785171 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:42:53.785599 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:42:53.785968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:53.786998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:42:53.786998 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:53.786998 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:42:53.786998 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:42:53.786998 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:53.786998 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:42:53.786998 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:53.786998 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:42:53.788667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:53.788667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:42:53.788667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:42:53.788667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:42:53.788667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:42:53.788667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:42:53.788667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:42:53.788667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:53.788667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:42:53.788667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:53.788667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:42:53.788667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:53.789250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:42:53.789250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:53.789250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:42:53.789250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:53.789250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:42:53.789250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:42:53.789250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:53.789250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:42:53.789250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:53.789250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:42:53.789250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:42:53.789250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:53.789250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:42:53.789958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:53.789958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:42:53.790667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:53.790667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:42:53.790667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:42:53.790667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:42:53.790667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:42:53.790667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:42:53.790667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:42:53.790667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:42:53.790667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:42:53.790667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:42:53.790667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:42:53.790667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:42:53.790667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:42:53.790667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:42:53.790667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:42:53.790667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:42:53.791959 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:42:53.791959 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:42:53.791959 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:53.791959 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:42:53.791959 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:53.791959 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:42:53.791959 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:53.791959 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:42:53.791959 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:53.791959 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:42:53.791959 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:53.791959 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:42:53.791959 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:42:53.791959 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:53.791959 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:42:53.791959 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:53.791959 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:42:53.791959 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:42:53.791959 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:53.792535 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:42:53.792535 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:53.792535 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:42:53.792535 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:42:53.792535 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:42:53.792535 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:42:53.792535 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:42:53.792535 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:42:53.792838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:53.793636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:42:53.793636 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:53.793636 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:42:53.793636 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:42:53.793636 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:53.793636 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:42:53.793636 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:53.793636 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:42:53.794904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:53.794904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:42:53.794904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:42:53.794904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:42:53.794904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:53.794904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:42:53.794904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:53.794904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:42:53.794904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:53.794904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:42:53.794904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:53.794904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:42:53.794904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:53.794904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:42:53.794904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:42:53.796183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:53.796183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:42:53.796183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:53.796183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:42:53.796183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:42:53.796183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:53.796183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:42:53.796183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:53.796183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:42:53.796183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:53.796183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:42:53.796183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:42:53.796183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:42:53.796183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:42:53.797812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:42:53.797812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:42:53.797812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:42:53.797812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:42:53.797812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:42:53.797812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:53.797812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:42:53.797812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:53.797812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:42:53.797812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:53.797812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:42:53.797812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:53.797812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:42:53.798365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:53.798365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:42:53.798365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:42:53.798365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:53.798365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:42:53.798365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:53.798365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:42:53.798365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:42:53.798365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:53.798365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:42:53.798365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:53.798365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:42:53.798664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:53.798664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:42:53.798664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:42:53.798664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:42:53.798664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:42:53.798664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:42:53.798664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:42:53.798664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:42:53.798664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:42:53.798664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:42:53.798664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:42:53.798664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:42:53.798664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:42:53.798664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:42:53.798664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:42:53.798664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:42:53.799222 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:42:53.799222 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:42:53.799222 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:42:53.799222 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:42:53.799222 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:42:53.799222 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:53.799222 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:42:53.799222 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:53.799222 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:42:53.799222 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:53.799222 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:42:53.799222 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:53.799222 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:42:53.799222 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:53.799222 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:42:53.799222 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:42:53.799222 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:53.799222 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:42:53.799934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:53.799934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:42:53.799934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:42:53.799934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:53.799934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:42:53.799934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:53.799934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:42:53.799934 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:42:53.799934 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:42:53.799934 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:42:53.800223 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:42:53.800223 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:42:53.800223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:53.800223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:42:53.800223 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:53.800223 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:42:53.800223 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:42:53.800223 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:53.800223 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:42:53.800223 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:53.800223 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:42:53.800652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:53.800652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:42:53.800652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:42:53.800652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:42:53.800652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:53.800652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:42:53.800652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:53.800652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:42:53.800652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:53.800652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:42:53.800652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:53.800652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:42:53.800652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:53.800652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:42:53.800652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:42:53.801057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:53.801057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:42:53.801057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:53.801057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:42:53.801057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:42:53.801057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:53.801057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:42:53.801057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:53.801057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:42:53.801416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:53.801416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:42:53.801416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:42:53.801416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:42:53.801416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:42:53.801416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:42:53.801416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:42:53.801416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:42:53.801416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:42:53.801416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:53.801416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:42:53.802188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:53.802188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:42:53.802188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:53.802188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:42:53.802188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:53.802188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:42:53.802188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:53.802188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:42:53.802188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:42:53.802188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:53.802188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:42:53.802188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:53.802188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:42:53.802188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:42:53.802833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:53.802833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:42:53.802833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:53.802833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:42:53.802833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:53.802833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:42:53.802833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:42:53.802833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:42:53.802833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:42:53.802833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:42:53.802833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:42:53.802833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:42:53.802833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:42:53.802833 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:42:53.803289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:42:53.803289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:42:53.803289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:42:53.803289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:42:53.803289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:42:53.803289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:42:53.803289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:42:53.803289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:42:53.803289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:42:53.803289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:42:53.803289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:53.803289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:42:53.803289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:53.803289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:42:53.803289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:53.803289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:42:53.803289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:53.803289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:42:53.803289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:53.803920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:42:53.803920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:42:53.803920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:53.803920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:42:53.803920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:53.803920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:42:53.803920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:42:53.803920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:53.803920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:42:53.803920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:53.803920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:42:53.809763 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:42:53.810111 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:42:53.810227 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:42:53.810380 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:42:53.810605 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:42:53.810849 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:42:53.810984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:53.811425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:42:53.811425 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:53.811425 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:42:53.811425 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:42:53.811425 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:53.811425 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:42:53.811425 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:53.811425 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:42:53.812361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:53.812361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:42:53.812361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:42:53.812361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:42:53.812361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:42:53.812361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:42:53.812361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:42:53.812361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:53.812361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:42:53.812361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:53.812361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:42:53.812361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:53.812712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:42:53.812712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:53.812712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:42:53.812712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:53.812712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:42:53.812712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:42:53.812712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:53.812712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:42:53.812712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:53.812712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:42:53.812712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:42:53.812712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:53.812712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:42:53.813063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:53.813063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:42:53.813370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:53.813370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:42:53.813370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:42:53.813370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:42:53.813370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:42:53.813370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:42:53.813370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:42:53.813370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:42:53.813370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:42:53.813370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:42:53.813370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:42:53.813370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:42:53.813370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:42:53.813370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:42:53.813370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:42:53.813370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:42:53.813996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:42:53.813996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:42:53.813996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:53.813996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:42:53.813996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:53.813996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:42:53.813996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:53.813996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:42:53.813996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:53.813996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:42:53.813996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:53.813996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:42:53.813996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:42:53.813996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:53.813996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:42:53.813996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:53.813996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:42:53.813996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:42:53.813996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:53.814537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:42:53.814537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:53.814537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:42:58.637625 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:42:58.639060 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:42:58.639060 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:42:58.639469 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:42:58.641602 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG dbcounter [-] [106314] Writing DB stats neutron:SELECT=240 {{(pid=106314) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 09 09:42:58.681086 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:42:58.681520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:58.682364 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:42:58.682364 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:58.682364 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:42:58.682364 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:42:58.682364 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:58.682364 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:42:58.682364 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:58.682364 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:42:58.683250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.044 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:58.683250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:42:58.683250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:42:58.683250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:42:58.683250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:42:58.683250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:42:58.683250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:42:58.683250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:58.683250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:42:58.683250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:58.683250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:42:58.683250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:58.683677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:42:58.683677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:58.683677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:42:58.683677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:58.683677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:42:58.683677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:42:58.683677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:58.683677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:42:58.683677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:58.683677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:42:58.683677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:42:58.683677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:58.683677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:42:58.684073 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:58.684073 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:42:58.684664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:58.684664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:42:58.684664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:42:58.684664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:42:58.684664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:42:58.684664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:42:58.684664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:42:58.684664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:42:58.684664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:42:58.684664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:42:58.684664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:42:58.684664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:42:58.684664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:42:58.684664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:42:58.684664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:42:58.684664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:42:58.685233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:42:58.685233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:42:58.685233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:58.685233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:42:58.685233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:58.685233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:42:58.685233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:58.685233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:42:58.685233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:58.685233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:42:58.685233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:58.685233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:42:58.685233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:42:58.685233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:58.685233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:42:58.685233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:58.685233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:42:58.685233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:42:58.685233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:58.685975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:42:58.685975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:58.685975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:42:58.783790 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:42:58.784276 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:42:58.784462 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:42:58.785922 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:42:58.786443 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:42:58.786844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:58.788052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:42:58.788052 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:58.788052 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:42:58.788052 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:42:58.788052 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:58.788052 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:42:58.788052 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:58.788052 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:42:58.790282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:58.790282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:42:58.790282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:42:58.790282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:42:58.790282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:42:58.790282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:42:58.790282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:42:58.790282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:58.790282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:42:58.790282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:58.790282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:42:58.790282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:58.791033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:42:58.791033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:58.791033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:42:58.791033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:58.791033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:42:58.791033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:42:58.791033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:58.791033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:42:58.791033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:58.791033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:42:58.791033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:42:58.791033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:58.791033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:42:58.791699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:58.791699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:42:58.792158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:58.792158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:42:58.792158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:42:58.792158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:42:58.792158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:42:58.792158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:42:58.792158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:42:58.792158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:42:58.792158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:42:58.792158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:42:58.792158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:42:58.792158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:42:58.792158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:42:58.792158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:42:58.792158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:42:58.792158 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:42:58.792919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:42:58.792919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:42:58.792919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:58.792919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:42:58.792919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:58.792919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:42:58.792919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:58.792919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:42:58.792919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:58.792919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:42:58.792919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:58.792919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:42:58.792919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:42:58.792919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:58.792919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:42:58.792919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:58.792919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:42:58.792919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:42:58.792919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:58.793929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:42:58.793929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:58.793929 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:42:58.793929 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:42:58.793929 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:42:58.793929 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:42:58.793929 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:42:58.794338 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:42:58.794447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:58.795218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:42:58.795218 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:58.795218 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:42:58.795218 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:42:58.795218 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:58.795218 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:42:58.795218 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:58.795218 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:42:58.795997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:58.795997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:42:58.795997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:42:58.795997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:42:58.795997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:58.795997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:42:58.795997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:58.795997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:42:58.795997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:58.795997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:42:58.795997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:58.795997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:42:58.795997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:58.795997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:42:58.795997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:42:58.796619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:58.796619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:42:58.796619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:58.796619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:42:58.796619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:42:58.796619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:58.796619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:42:58.796619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:58.796619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:42:58.796914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:58.796914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:42:58.796914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:42:58.796914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:42:58.796914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:42:58.796914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:42:58.796914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:42:58.796914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:42:58.796914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:42:58.796914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:42:58.796914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:58.796914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:42:58.797244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:58.797244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:42:58.797244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:58.797244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:42:58.797244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:58.797244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:42:58.797244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:58.797244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:42:58.797244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:42:58.797244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:58.797244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:42:58.797244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:58.797244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:42:58.797244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:42:58.797724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:58.797724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:42:58.797724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:58.797724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:42:58.797908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:58.797908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:42:58.797908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:42:58.797908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:42:58.797908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:42:58.797908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:42:58.797908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:42:58.797908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:42:58.797908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:42:58.797908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:42:58.797908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:42:58.797908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:42:58.797908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:42:58.797908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:42:58.797908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:42:58.797908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:42:58.798318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:42:58.798318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:42:58.798318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:42:58.798318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:42:58.798318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:42:58.798318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:58.798318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:42:58.798318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:58.798318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:42:58.798318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:58.798318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:42:58.798318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:58.798318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:42:58.798318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:58.798318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:42:58.798318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:42:58.798318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:58.798318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:42:58.798812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:58.798812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:42:58.798812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:42:58.798812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:58.798812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:42:58.798812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:58.798812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:42:58.798812 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:42:58.799024 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:42:58.799024 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:42:58.799241 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:42:58.799621 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:42:58.800165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:58.800831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:42:58.800831 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:58.800831 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:42:58.800831 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:42:58.800831 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:58.800831 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:42:58.800831 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:58.800831 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:42:58.801527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:58.801527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:42:58.801527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:42:58.801527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:42:58.801527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:58.801527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:42:58.801527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:58.801527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:42:58.801527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:58.801527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:42:58.801527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:58.801527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:42:58.801527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:58.801527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:42:58.801527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:42:58.801934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:58.801934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:42:58.801934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:58.801934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:42:58.801934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:42:58.801934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:58.801934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:42:58.801934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:58.801934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:42:58.802441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:58.802441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:42:58.802441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:42:58.802441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:42:58.802441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:42:58.802441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:42:58.802441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:42:58.802441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:42:58.802441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:42:58.802441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:58.802441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:42:58.802734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:58.802734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:42:58.802734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:58.802734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:42:58.802734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:58.802734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:42:58.802734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:58.802734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:42:58.802734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:42:58.802734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:58.802734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:42:58.802734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:58.802734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:42:58.802734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:42:58.803096 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:58.803096 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:42:58.803096 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:58.803096 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:42:58.803570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:58.803570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:42:58.803570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:42:58.803570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:42:58.803570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:42:58.803570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:42:58.803570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:42:58.803570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:42:58.803570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:42:58.803570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:42:58.803570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:42:58.803570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:42:58.803570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:42:58.803570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:42:58.803570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:42:58.803570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:42:58.803570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:42:58.804056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:42:58.804056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:42:58.804056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:42:58.804056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:58.804056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:42:58.804056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:58.804056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:42:58.804056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:58.804056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:42:58.804056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:58.804056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:42:58.804056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:58.804056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:42:58.804056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:42:58.804056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:58.804056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:42:58.804056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:58.804056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:42:58.804056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:42:58.804517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:58.804517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:42:58.804517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:58.804517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:42:58.810252 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:42:58.810546 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:42:58.810665 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:42:58.810844 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:42:58.811246 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:42:58.811855 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:42:58.812008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:58.812925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:42:58.812925 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:58.812925 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:42:58.812925 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:42:58.812925 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:58.812925 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:42:58.812925 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:58.812925 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:42:58.813934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:58.813934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:42:58.813934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:42:58.813934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:42:58.813934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:42:58.813934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:42:58.813934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:42:58.813934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:58.813934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:42:58.813934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:58.813934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:42:58.813934 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:58.814331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:42:58.814331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:58.814331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:42:58.814331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:58.814331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:42:58.814331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:42:58.814331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:58.814331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:42:58.814331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:58.814331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:42:58.814331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:42:58.814331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:58.814331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:42:58.814845 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:58.814845 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:42:58.815302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:58.815302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:42:58.815302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:42:58.815302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:42:58.815302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:42:58.815302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:42:58.815302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:42:58.815302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:42:58.815302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:42:58.815302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:42:58.815302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:42:58.815302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:42:58.815302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:42:58.815302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:42:58.815302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:42:58.815302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:42:58.816118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:42:58.816118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:42:58.816118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:58.816118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:42:58.816118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:42:58.816118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:42:58.816118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:42:58.816118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:42:58.816118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:42:58.816118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:42:58.816118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:42:58.816118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:42:58.816118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:42:58.816118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:42:58.816118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:42:58.816118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:42:58.816118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:42:58.816118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:42:58.816118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:42:58.816721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:42:58.816721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:42:58.816721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:43:03.638668 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:43:03.639908 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:43:03.639908 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:43:03.641076 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:43:03.679358 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:43:03.679629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:03.680688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:43:03.680688 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:03.680688 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:43:03.680688 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:43:03.680688 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:03.680688 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:43:03.680688 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:03.680688 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:43:03.681715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:03.681715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:43:03.681715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:43:03.681715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:43:03.681715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:43:03.681715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:43:03.681715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:43:03.681715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:03.681715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:43:03.681715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:03.681715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:43:03.681715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:03.682406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:43:03.682406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:03.682406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:43:03.682406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:03.682406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:43:03.682406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:43:03.682406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:03.682406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:43:03.682406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:03.682406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:43:03.682406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:43:03.682406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:03.682406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:43:03.683300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:03.683300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:43:03.683300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:03.683300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:43:03.683300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:43:03.683300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:43:03.683300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:43:03.683300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:43:03.683300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:43:03.683300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:43:03.683300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:43:03.683300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:43:03.683300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:43:03.683300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:43:03.683300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:43:03.684413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:43:03.684413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:43:03.684413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:43:03.684413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:43:03.684413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:43:03.684413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:03.684413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:43:03.684413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:03.684413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:43:03.684413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:03.684413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:43:03.684413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:03.684413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:43:03.684413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:03.684413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:43:03.684413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:43:03.684413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:03.684413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:43:03.684413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:03.686176 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:43:03.686176 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:43:03.686176 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:03.686176 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:43:03.686176 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:03.686176 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:43:03.784449 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:43:03.785138 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:43:03.785332 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:43:03.786351 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:43:03.786538 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:43:03.786716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:03.787491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:43:03.787491 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:03.787491 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:43:03.787491 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:43:03.787491 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:03.787491 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:43:03.787491 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:03.787491 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:43:03.789320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:03.789320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:43:03.789320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:43:03.789320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:43:03.789320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:43:03.789320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:43:03.789320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:43:03.789320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:03.789320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:43:03.789320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:03.789320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:43:03.789320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:03.790164 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:43:03.790164 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:03.790164 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:43:03.790164 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:03.790164 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:43:03.790164 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:43:03.790164 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:03.790164 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:43:03.790164 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:03.790164 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:43:03.790164 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:43:03.790164 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:03.790164 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:43:03.791023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:03.791023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:43:03.791325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:03.791325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:43:03.791325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:43:03.791325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:43:03.791325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:43:03.791325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:43:03.791325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:43:03.791325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:43:03.791325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:43:03.791325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:43:03.791325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:43:03.791325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:43:03.791325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:43:03.791325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:43:03.791325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:43:03.791325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:43:03.792278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:43:03.792278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:43:03.792278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:03.792278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:43:03.792278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:03.792278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:43:03.792278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:03.792278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:43:03.792278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:03.792278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:43:03.792278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:03.792278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:43:03.792278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:43:03.792278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:03.792278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:43:03.792278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:03.792278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:43:03.792278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:43:03.792278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:03.793470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:43:03.793470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:03.793470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:43:03.793470 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:43:03.793470 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:43:03.793886 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:43:03.793886 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:43:03.794339 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:43:03.794682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:03.795400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:43:03.795400 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:03.795400 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:43:03.795400 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:43:03.795400 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:03.795400 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:43:03.795400 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:03.795400 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:43:03.796191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:03.796191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:43:03.796191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:43:03.796191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:43:03.796191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:03.796191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:43:03.796191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:03.796191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:43:03.796191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:03.796191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:43:03.796191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:03.796191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:43:03.796191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:03.796191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:43:03.796191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:43:03.796949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:03.796949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:43:03.796949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:03.796949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:43:03.796949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:43:03.796949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:03.796949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:43:03.796949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:03.796949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:43:03.797541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:03.797541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:43:03.797541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:43:03.797541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:43:03.797541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:43:03.797541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:43:03.797541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:43:03.797541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:43:03.797541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:43:03.797541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:43:03.797541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:03.797541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:43:03.798085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:03.798085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:43:03.798085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:03.798085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:43:03.798085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:03.798085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:43:03.798085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:03.798085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:43:03.798085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:43:03.798085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:03.798085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:43:03.798085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:03.798085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:43:03.798085 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:43:03.798641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:03.798641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:43:03.798641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:03.798641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:43:03.798641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:03.798641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:43:03.798641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:43:03.798641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:43:03.798641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:43:03.798641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:43:03.798641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:43:03.798641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:43:03.798641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:43:03.798641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:43:03.799284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:43:03.799284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:43:03.799284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:43:03.799284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:43:03.799284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:43:03.799284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:43:03.799284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:43:03.799284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:43:03.799284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:43:03.799284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:43:03.799284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:43:03.799284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:03.799284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:43:03.799284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:03.799284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:43:03.799284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:03.799284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:43:03.799284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:03.799284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:43:03.800122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:03.800122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:43:03.800122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:43:03.800122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:03.800122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:43:03.800122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:03.800122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:43:03.800122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:43:03.800122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:03.800122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:43:03.800122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:03.800122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:43:03.800122 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:43:03.800122 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:43:03.800714 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:43:03.800714 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:43:03.800714 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:43:03.800714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:03.801055 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:43:03.801055 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:03.801055 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:43:03.801055 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:43:03.801055 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:03.801055 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:43:03.801055 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:03.801055 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:43:03.801882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:03.801882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:43:03.801882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:43:03.801882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:43:03.801882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:03.801882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:43:03.801882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:03.801882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:43:03.801882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:03.801882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:43:03.801882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:03.801882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:43:03.801882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:03.801882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:43:03.801882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:43:03.802507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:03.802507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:43:03.802507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:03.802507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:43:03.802507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:43:03.802507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:03.802507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:43:03.802507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:03.802507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:43:03.802858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:03.802858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:43:03.802858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:43:03.802858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:43:03.802858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:43:03.802858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:43:03.802858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:43:03.802858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:43:03.802858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:43:03.802858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:03.802858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:43:03.803343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:03.803343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:43:03.803343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:03.803343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:43:03.803343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:03.803343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:43:03.803343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:03.803343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:43:03.803343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:43:03.803343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:03.803343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:43:03.803343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:03.803343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:43:03.803343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:43:03.803896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:03.803896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:43:03.803896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:03.803896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:43:03.804066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:03.804066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:43:03.804066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:43:03.804066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:43:03.804066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:43:03.804066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:43:03.804066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:43:03.804066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:43:03.804066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:43:03.804066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:43:03.804066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:43:03.804066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:43:03.804066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:43:03.804066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:43:03.804066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:43:03.804066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:43:03.804066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:43:03.804819 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:43:03.804819 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:43:03.804819 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:43:03.804819 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:03.804819 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:43:03.804819 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:03.804819 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:43:03.804819 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:03.804819 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:43:03.804819 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:03.804819 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:43:03.804819 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:03.804819 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:43:03.804819 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:43:03.804819 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:03.804819 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:43:03.804819 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:03.804819 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:43:03.804819 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:43:03.805514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:03.805514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:43:03.805514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:03.805514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:43:03.811513 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:43:03.811886 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:43:03.812118 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:43:03.812374 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:43:03.812938 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:43:03.813334 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:43:03.813586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:03.814490 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:43:03.814490 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:03.814490 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:43:03.814490 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:43:03.814490 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:03.814490 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:43:03.814490 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:03.814490 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:43:03.816097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:03.816097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:43:03.816097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:43:03.816097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:43:03.816097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:43:03.816097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:43:03.816097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:43:03.816097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:03.816097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:43:03.816097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:03.816097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:43:03.816097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:03.816470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:43:03.816470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:03.816470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:43:03.816470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:03.816470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:43:03.816470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:43:03.816470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:03.816470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:43:03.816470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:03.816470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:43:03.816470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:43:03.816470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:03.816470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:43:03.816825 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:03.816825 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:43:03.817723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:03.817723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:43:03.817723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:43:03.817723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:43:03.817723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:43:03.817723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:43:03.817723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:43:03.817723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:43:03.817723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:43:03.817723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:43:03.817723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:43:03.817723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:43:03.817723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:43:03.817723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:43:03.817723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:43:03.817723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:43:03.818254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:43:03.818254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:43:03.818254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:03.818254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:43:03.818254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:03.818254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:43:03.818254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:03.818254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:43:03.818254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:03.818254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:43:03.818254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:03.818254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:43:03.818254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:43:03.818254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:03.818254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:43:03.818254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:03.818254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:43:03.818254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:43:03.818254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:03.818799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:43:03.818799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:03.818799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:43:08.639498 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:43:08.640388 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:43:08.640388 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:43:08.642122 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:43:08.672463 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:43:08.672887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:08.673850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:43:08.673850 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:08.673850 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:43:08.673850 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:43:08.673850 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:08.673850 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:43:08.673850 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:08.673850 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:43:08.674728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.033 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:08.674728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:43:08.674728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:43:08.674728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:43:08.674728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:43:08.674728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:43:08.674728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:43:08.674728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:08.674728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:43:08.674728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:08.674728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:43:08.674728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:08.675176 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:43:08.675176 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:08.675176 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:43:08.675176 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:08.675176 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:43:08.675176 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:43:08.675176 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:08.675176 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:43:08.675176 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:08.675176 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:43:08.675176 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:43:08.675176 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:08.675176 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:43:08.675541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:08.675541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:43:08.675541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:08.675541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:43:08.675541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:43:08.675541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:43:08.675541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:43:08.675541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:43:08.675541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:43:08.675541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:43:08.675541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:43:08.675541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:43:08.675541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:43:08.675541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:43:08.675541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:43:08.677347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:43:08.677347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:43:08.677347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:43:08.677347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:43:08.677347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:43:08.677347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:08.677347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:43:08.677347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:08.677347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:43:08.677347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:08.677347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:43:08.677347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:08.677347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:43:08.677347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:08.677347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:43:08.677347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:43:08.677347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:08.677347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:43:08.677347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:08.681180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:43:08.681180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:43:08.681180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:08.681180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:43:08.681180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:08.681180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:43:08.785158 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:43:08.785707 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:43:08.785848 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:43:08.787029 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:43:08.787421 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:43:08.787713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:08.788761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:43:08.788761 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:08.788761 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:43:08.788761 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:43:08.788761 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:08.788761 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:43:08.788761 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:08.788761 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:43:08.790538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:08.790538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:43:08.790538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:43:08.790538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:43:08.790538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:43:08.790538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:43:08.790538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:43:08.790538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:08.790538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:43:08.790538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:08.790538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:43:08.790538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:08.791161 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:43:08.791161 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:08.791161 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:43:08.791161 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:08.791161 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:43:08.791161 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:43:08.791161 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:08.791161 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:43:08.791161 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:08.791161 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:43:08.791161 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:43:08.791161 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:08.791161 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:43:08.791923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:08.791923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:43:08.792343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:08.792343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:43:08.792343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:43:08.792343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:43:08.792343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:43:08.792343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:43:08.792343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:43:08.792343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:43:08.792343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:43:08.792343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:43:08.792343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:43:08.792343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:43:08.792343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:43:08.792343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:43:08.792343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:43:08.792343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:43:08.793134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:43:08.793134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:43:08.793134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:08.793134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:43:08.793134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:08.793134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:43:08.793134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:08.793134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:43:08.793134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:08.793134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:43:08.793134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:08.793134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:43:08.793134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:43:08.793134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:08.793134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:43:08.793134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:08.793134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:43:08.793134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:43:08.793134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:08.794026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:43:08.794026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:08.794026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:43:08.794026 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:43:08.794255 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:43:08.794418 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:43:08.794718 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:43:08.795519 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:43:08.795919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:08.796762 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:43:08.796762 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:08.796762 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:43:08.796762 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:43:08.796762 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:08.796762 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:43:08.796762 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:08.796762 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:43:08.797542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:08.797542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:43:08.797542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:43:08.797542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:43:08.797542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:08.797542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:43:08.797542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:08.797542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:43:08.797542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:08.797542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:43:08.797542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:08.797542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:43:08.797542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:08.797542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:43:08.797542 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:43:08.797942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:08.797942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:43:08.797942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:08.797942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:43:08.797942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:43:08.797942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:08.797942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:43:08.797942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:08.797942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:43:08.798371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:08.798371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:43:08.798371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:43:08.798371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:43:08.798371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:43:08.798371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:43:08.798371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:43:08.798371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:43:08.798371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:43:08.798371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:43:08.798371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:08.798371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:43:08.798726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:08.798726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:43:08.798726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:08.798726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:43:08.798726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:08.798726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:43:08.798726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:08.798726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:43:08.798726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:43:08.798726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:08.798726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:43:08.798726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:08.798726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:43:08.798726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:43:08.799130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:08.799130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:43:08.799130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:08.799130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:43:08.799381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:08.799381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:43:08.799381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:43:08.799381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:43:08.799381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:43:08.799381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:43:08.799381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:43:08.799381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:43:08.799381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:43:08.799381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:43:08.799381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:43:08.799381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:43:08.799381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:43:08.799381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:43:08.799381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:43:08.799381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:43:08.799804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:43:08.799804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:43:08.799804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:43:08.799804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:43:08.799804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:43:08.799804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:08.799804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:43:08.799804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:08.799804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:43:08.799804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:08.799804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:43:08.799804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:08.799804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:43:08.799804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:08.799804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:43:08.799804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:43:08.799804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:08.799804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:43:08.800261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:08.800261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:43:08.800261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:43:08.800261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:08.800261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:43:08.800261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:08.800261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:43:08.800261 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:43:08.800261 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:43:08.800261 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:43:08.800554 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:43:08.800828 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:43:08.801248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:08.802060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:43:08.802060 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:08.802060 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:43:08.802060 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:43:08.802060 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:08.802060 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:43:08.802060 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:08.802060 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:43:08.802836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:08.802836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:43:08.802836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:43:08.802836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:43:08.802836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:08.802836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:43:08.802836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:08.802836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:43:08.802836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:08.802836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:43:08.802836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:08.802836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:43:08.802836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:08.802836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:43:08.802836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:43:08.803208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:08.803208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:43:08.803208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:08.803208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:43:08.803208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:43:08.803208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:08.803208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:43:08.803208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:08.803208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:43:08.803678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:08.803678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:43:08.803678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:43:08.803678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:43:08.803678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:43:08.803678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:43:08.803678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:43:08.803678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:43:08.803678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:43:08.803678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:08.803678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:43:08.804002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:08.804002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:43:08.804002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:08.804002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:43:08.804002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:08.804002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:43:08.804002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:08.804002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:43:08.804002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:43:08.804002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:08.804002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:43:08.804002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:08.804002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:43:08.804002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:43:08.804373 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:08.804373 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:43:08.804373 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:08.804373 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:43:08.804755 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:08.804755 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:43:08.804755 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:43:08.804755 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:43:08.804755 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:43:08.804755 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:43:08.804755 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:43:08.804755 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:43:08.804755 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:43:08.804755 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:43:08.804755 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:43:08.804755 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:43:08.804755 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:43:08.804755 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:43:08.804755 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:43:08.804755 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:43:08.804755 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:43:08.805195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:43:08.805195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:43:08.805195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:43:08.805195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:08.805195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:43:08.805195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:08.805195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:43:08.805195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:08.805195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:43:08.805195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:08.805195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:43:08.805195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:08.805195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:43:08.805195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:43:08.805195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:08.805195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:43:08.805195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:08.805195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:43:08.805195 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:43:08.805765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:08.805765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:43:08.805765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:08.805765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:43:08.811932 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:43:08.812198 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:43:08.812298 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:43:08.812495 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:43:08.812939 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:43:08.813213 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:43:08.813369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:08.813941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:43:08.813941 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:08.813941 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:43:08.813941 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:43:08.813941 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:08.813941 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:43:08.813941 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:08.813941 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:43:08.815298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:08.815298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:43:08.815298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:43:08.815298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:43:08.815298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:43:08.815298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:43:08.815298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:43:08.815298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:08.815298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:43:08.815298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:08.815298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:43:08.815298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:08.816050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:43:08.816050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:08.816050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:43:08.816050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:08.816050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:43:08.816050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:43:08.816050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:08.816050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:43:08.816050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:08.816050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:43:08.816050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:43:08.816050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:08.816050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:43:08.816718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:08.816718 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:43:08.817024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:08.817024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:43:08.817024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:43:08.817024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:43:08.817024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:43:08.817024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:43:08.817024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:43:08.817024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:43:08.817024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:43:08.817024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:43:08.817024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:43:08.817024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:43:08.817024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:43:08.817024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:43:08.817024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:43:08.817024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:43:08.817880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:43:08.817880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:43:08.817880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:08.817880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:43:08.817880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:08.817880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:43:08.817880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:08.817880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:43:08.817880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:08.817880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:43:08.817880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:08.817880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:43:08.817880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:43:08.817880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:08.817880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:43:08.817880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:08.817880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:43:08.817880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:43:08.817880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:08.818781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:43:08.818781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:08.818781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:43:13.640115 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:43:13.641015 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:43:13.641015 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:43:13.641693 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:43:13.678033 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:43:13.678416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:13.679259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:43:13.679259 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:13.679259 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:43:13.679259 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:43:13.679259 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:13.679259 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:43:13.679259 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:13.679259 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:43:13.680446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:13.680446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:43:13.680446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:43:13.680446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:43:13.680446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:43:13.680446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:43:13.680446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:43:13.680446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:13.680446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:43:13.680446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:13.680446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:43:13.680446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:13.681016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:43:13.681016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:13.681016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:43:13.681016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:13.681016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:43:13.681016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:43:13.681016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:13.681016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:43:13.681016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:13.681016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:43:13.681016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:43:13.681016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:13.681016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:43:13.681549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:13.681549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:43:13.681549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:13.681549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:43:13.681549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:43:13.681549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:43:13.681549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:43:13.681549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:43:13.681549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:43:13.681549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:43:13.681549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:43:13.681549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:43:13.681549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:43:13.681549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:43:13.681549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:43:13.682020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:43:13.682020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:43:13.682020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:43:13.682020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:43:13.682020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:43:13.682020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:13.682020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:43:13.682020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:13.682020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:43:13.682020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:13.682020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:43:13.682020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:13.682020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:43:13.682020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:13.682020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:43:13.682020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:43:13.682020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:13.682020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:43:13.682020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:13.682562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:43:13.682562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:43:13.682562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:13.682562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:43:13.682562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:13.682562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:43:13.785883 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:43:13.786827 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:43:13.787014 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:43:13.788291 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:43:13.788662 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:43:13.788947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:13.789893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:43:13.789893 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:13.789893 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:43:13.789893 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:43:13.789893 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:13.789893 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:43:13.789893 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:13.789893 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:43:13.791513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:13.791513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:43:13.791513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:43:13.791513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:43:13.791513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:43:13.791513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:43:13.791513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:43:13.791513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:13.791513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:43:13.791513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:13.791513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:43:13.791513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:13.792316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:43:13.792316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:13.792316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:43:13.792316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:13.792316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:43:13.792316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:43:13.792316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:13.792316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:43:13.792316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:13.792316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:43:13.792316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:43:13.792316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:13.792316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:43:13.793009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:13.793009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:43:13.793307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:13.793307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:43:13.793307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:43:13.793307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:43:13.793307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:43:13.793307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:43:13.793307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:43:13.793307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:43:13.793307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:43:13.793307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:43:13.793307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:43:13.793307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:43:13.793307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:43:13.793307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:43:13.793307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:43:13.793307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:43:13.794372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:43:13.794372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:43:13.794372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:13.794372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:43:13.794372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:13.794372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:43:13.794372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:13.794372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:43:13.794372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:13.794372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:43:13.794372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:13.794372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:43:13.794372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:43:13.794372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:13.794372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:43:13.794372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:13.794372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:43:13.794372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:43:13.794372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:13.795291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:43:13.795291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:13.795291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:43:13.795291 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:43:13.795291 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:43:13.795291 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:43:13.795291 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:43:13.795291 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:43:13.795853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:13.796570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:43:13.796570 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:13.796570 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:43:13.796570 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:43:13.796570 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:13.796570 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:43:13.796570 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:13.796570 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:43:13.797917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:13.797917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:43:13.797917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:43:13.797917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:43:13.797917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:13.797917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:43:13.797917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:13.797917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:43:13.797917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:13.797917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:43:13.797917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:13.797917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:43:13.797917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:13.797917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:43:13.797917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:43:13.798666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:13.798666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:43:13.798666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:13.798666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:43:13.798666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:43:13.798666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:13.798666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:43:13.798666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:13.798666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:43:13.799409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:13.799409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:43:13.799409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:43:13.799409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:43:13.799409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:43:13.799409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:43:13.799409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:43:13.799409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:43:13.799409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:43:13.799409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:43:13.799409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:13.799409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:43:13.799987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:13.799987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:43:13.799987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:13.799987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:43:13.799987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:13.799987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:43:13.799987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:13.799987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:43:13.799987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:43:13.799987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:13.799987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:43:13.799987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:13.799987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:43:13.799987 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:43:13.800460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:13.800460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:43:13.800460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:13.800460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:43:13.801561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:13.801561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:43:13.801561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:43:13.801561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:43:13.801561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:43:13.801561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:43:13.801561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:43:13.801561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:43:13.801561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:43:13.801561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:43:13.801561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:43:13.801561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:43:13.801561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:43:13.801561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:43:13.801561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:43:13.801561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:43:13.802212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:43:13.802212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:43:13.802212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:43:13.802212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:43:13.802212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:43:13.802212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:13.802212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:43:13.802212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:13.802212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:43:13.802212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:13.802212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:43:13.802212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:13.802212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:43:13.802212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:13.802212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:43:13.802212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:43:13.802212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:13.802212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:43:13.802969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:13.802969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:43:13.802969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:43:13.802969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:13.802969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:43:13.802969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:13.802969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:43:13.802969 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:43:13.802969 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:43:13.802969 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:43:13.803396 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:43:13.803396 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:43:13.803396 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:13.804342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:43:13.804342 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:13.804342 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:43:13.804342 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:43:13.804342 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:13.804342 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:43:13.804342 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:13.804342 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:43:13.805850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:13.805850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:43:13.805850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:43:13.805850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:43:13.805850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:13.805850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:43:13.805850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:13.805850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:43:13.805850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:13.805850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:43:13.805850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:13.805850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:43:13.805850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:13.805850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:43:13.805850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:43:13.806572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:13.806572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:43:13.806572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:13.806572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:43:13.806572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:43:13.806572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:13.806572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:43:13.806572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:13.806572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:43:13.807681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:13.807681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:43:13.807681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:43:13.807681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:43:13.807681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:43:13.807681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:43:13.807681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:43:13.807681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:43:13.807681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:43:13.807681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:13.807681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:43:13.808375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:13.808375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:43:13.808375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:13.808375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:43:13.808375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:13.808375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:43:13.808375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:13.808375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:43:13.808375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:43:13.808375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:13.808375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:43:13.808375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:13.808375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:43:13.808375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:43:13.808982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:13.808982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:43:13.808982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:13.808982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:43:13.808982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:13.808982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:43:13.808982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:43:13.808982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:43:13.808982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:43:13.808982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:43:13.808982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:43:13.808982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:43:13.808982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:43:13.808982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:43:13.809647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:43:13.809647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:43:13.809647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:43:13.809647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:43:13.809647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:43:13.809647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:43:13.809647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:43:13.809647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:43:13.809647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:43:13.809647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:43:13.809647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:13.809647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:43:13.809647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:13.809647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:43:13.809647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:13.809647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:43:13.809647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:13.809647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:43:13.809647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:13.810944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:43:13.810944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:43:13.810944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:13.810944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:43:13.810944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:13.810944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:43:13.810944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:43:13.810944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:13.810944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:43:13.810944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:13.810944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:43:13.812278 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:43:13.812508 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:43:13.812568 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:43:13.812691 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:43:13.813187 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:43:13.813537 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:43:13.813760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:13.814478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:43:13.814478 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:13.814478 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:43:13.814478 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:43:13.814478 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:13.814478 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:43:13.814478 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:13.814478 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:43:13.815204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:13.815204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:43:13.815204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:43:13.815204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:43:13.815204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:43:13.815204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:43:13.815204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:43:13.815204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:13.815204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:43:13.815204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:13.815204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:43:13.815204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:13.815603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:43:13.815603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:13.815603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:43:13.815603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:13.815603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:43:13.815603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:43:13.815603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:13.815603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:43:13.815603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:13.815603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:43:13.815603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:43:13.815603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:13.815603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:43:13.816028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:13.816028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:43:13.816233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:13.816233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:43:13.816233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:43:13.816233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:43:13.816233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:43:13.816233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:43:13.816233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:43:13.816233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:43:13.816233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:43:13.816233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:43:13.816233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:43:13.816233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:43:13.816233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:43:13.816233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:43:13.816233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:43:13.816233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:43:13.816722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:43:13.816722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:43:13.816722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:13.816722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:43:13.816722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:13.816722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:43:13.816722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:13.816722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:43:13.816722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:13.816722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:43:13.816722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:13.816722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:43:13.816722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:43:13.816722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:13.816722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:43:13.816722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:13.816722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:43:13.816722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:43:13.816722 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:13.817286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:43:13.817286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:13.817286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:43:18.640931 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:43:18.641606 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:43:18.641606 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:43:18.642683 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:43:18.684426 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:43:18.684704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:18.685331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:43:18.685331 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:18.685331 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:43:18.685331 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:43:18.685331 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:18.685331 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:43:18.685331 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:18.685331 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:43:18.686485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.044 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:18.686485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:43:18.686485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:43:18.686485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:43:18.686485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:43:18.686485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:43:18.686485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:43:18.686485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:18.686485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:43:18.686485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:18.686485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:43:18.686485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:18.686848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:43:18.686848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:18.686848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:43:18.686848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:18.686848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:43:18.686848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:43:18.686848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:18.686848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:43:18.686848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:18.686848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:43:18.686848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:43:18.686848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:18.686848 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:43:18.687114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:18.687114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:43:18.687114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:18.687114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:43:18.687114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:43:18.687114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:43:18.687114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:43:18.687114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:43:18.687114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:43:18.687114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:43:18.687114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:43:18.687114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:43:18.687114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:43:18.687114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:43:18.687114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:43:18.687434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:43:18.687434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:43:18.687434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:43:18.687434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:43:18.687434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:43:18.687434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:18.687434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:43:18.687434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:18.687434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:43:18.687434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:18.687434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:43:18.687434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:18.687434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:43:18.687434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:18.687434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:43:18.687434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:43:18.687434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:18.687434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:43:18.687434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:18.687888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:43:18.687888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:43:18.687888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:18.687888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:43:18.687888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:18.687888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:43:18.787052 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:43:18.787442 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:43:18.787663 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:43:18.788567 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:43:18.788766 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:43:18.788971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:18.789631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:43:18.789631 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:18.789631 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:43:18.789631 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:43:18.789631 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:18.789631 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:43:18.789631 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:18.789631 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:43:18.791219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:18.791219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:43:18.791219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:43:18.791219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:43:18.791219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:43:18.791219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:43:18.791219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:43:18.791219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:18.791219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:43:18.791219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:18.791219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:43:18.791219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:18.791963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:43:18.791963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:18.791963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:43:18.791963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:18.791963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:43:18.791963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:43:18.791963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:18.791963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:43:18.791963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:18.791963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:43:18.791963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:43:18.791963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:18.791963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:43:18.792566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:18.792566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:43:18.793143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:18.793143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:43:18.793143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:43:18.793143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:43:18.793143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:43:18.793143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:43:18.793143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:43:18.793143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:43:18.793143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:43:18.793143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:43:18.793143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:43:18.793143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:43:18.793143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:43:18.793143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:43:18.793143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:43:18.793143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:43:18.795029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:43:18.795029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:43:18.795029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:18.795029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:43:18.795029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:18.795029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:43:18.795029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:18.795029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:43:18.795029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:18.795029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:43:18.795029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:18.795029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:43:18.795029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:43:18.795029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:18.795029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:43:18.795029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:18.795029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:43:18.795029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:43:18.795029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:18.796455 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:43:18.796455 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:18.796455 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:43:18.796455 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:43:18.796455 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:43:18.796455 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:43:18.796455 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:43:18.796455 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:43:18.797124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:18.797381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:43:18.797381 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:18.797381 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:43:18.797381 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:43:18.797381 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:18.797381 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:43:18.797381 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:18.797381 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:43:18.798141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:18.798141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:43:18.798141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:43:18.798141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:43:18.798141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:18.798141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:43:18.798141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:18.798141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:43:18.798141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:18.798141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:43:18.798141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:18.798141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:43:18.798141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:18.798141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:43:18.798141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:43:18.798968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:18.798968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:43:18.798968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:18.798968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:43:18.798968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:43:18.798968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:18.798968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:43:18.798968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:18.798968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:43:18.798968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:18.798968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:43:18.798968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:43:18.798968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:43:18.798968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:43:18.799785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:43:18.799785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:43:18.799785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:43:18.799785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:43:18.799785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:43:18.799785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:18.799785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:43:18.799785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:18.799785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:43:18.799785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:18.799785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:43:18.799785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:18.799785 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:43:18.800627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:18.800627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:43:18.800627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:43:18.800627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:18.800627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:43:18.800627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:18.800627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:43:18.800627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:43:18.800627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:18.800627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:43:18.800627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:18.800627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:43:18.801495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:18.801495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:43:18.801495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:43:18.801495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:43:18.801495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:43:18.801495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:43:18.801495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:43:18.801495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:43:18.801495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:43:18.801495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:43:18.801495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:43:18.801495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:43:18.801495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:43:18.801495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:43:18.801495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:43:18.801495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:43:18.802850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:43:18.802850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:43:18.802850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:43:18.802850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:43:18.802850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:43:18.802850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:18.802850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:43:18.802850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:18.802850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:43:18.802850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:18.802850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:43:18.802850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:18.802850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:43:18.802850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:18.802850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:43:18.802850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:43:18.802850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:18.802850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:43:18.804672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:18.804672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:43:18.804672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:43:18.804672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:18.804672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:43:18.804672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:18.804672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:43:18.804672 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:43:18.804672 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:43:18.804672 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:43:18.804981 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:43:18.804981 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:43:18.804981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:18.804981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:43:18.804981 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:18.804981 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:43:18.804981 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:43:18.804981 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:18.804981 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:43:18.804981 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:18.804981 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:43:18.804981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:18.805232 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:43:18.805232 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:43:18.805232 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:43:18.805232 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:18.805232 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:43:18.805232 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:18.805232 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:43:18.805232 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:18.805232 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:43:18.805232 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:18.805232 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:43:18.805232 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:18.805232 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:43:18.805232 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:43:18.805232 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:18.805232 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:43:18.805232 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:18.805587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:43:18.805587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:43:18.805587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:18.805587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:43:18.805587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:18.805587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:43:18.805705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:18.805705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:43:18.805705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:43:18.805705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:43:18.805705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:43:18.805705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:43:18.805705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:43:18.805705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:43:18.805705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:43:18.805705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:18.805705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:43:18.806007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:18.806007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:43:18.806007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:18.806007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:43:18.806007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:18.806007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:43:18.806007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:18.806007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:43:18.806007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:43:18.806007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:18.806007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:43:18.806007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:18.806007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:43:18.806007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:43:18.806313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:18.806313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:43:18.806313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:18.806313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:43:18.807381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:18.807381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:43:18.807381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:43:18.807381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:43:18.807381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:43:18.807381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:43:18.807381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:43:18.807381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:43:18.807381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:43:18.807381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:43:18.807381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:43:18.807381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:43:18.807381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:43:18.807381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:43:18.807381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:43:18.807381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:43:18.807381 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:43:18.808388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:43:18.808388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:43:18.808388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:43:18.808388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:18.808388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:43:18.808388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:18.808388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:43:18.808388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:18.808388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:43:18.808388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:18.808388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:43:18.808388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:18.808388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:43:18.808388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:43:18.808388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:18.808388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:43:18.808388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:18.808388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:43:18.808388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:43:18.809249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:18.809249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:43:18.809249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:18.809249 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:43:18.812557 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:43:18.812768 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:43:18.812892 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:43:18.813056 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:43:18.813344 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:43:18.813577 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:43:18.813717 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:18.814233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:43:18.814233 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:18.814233 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:43:18.814233 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:43:18.814233 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:18.814233 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:43:18.814233 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:18.814233 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:43:18.815058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:18.815058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:43:18.815058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:43:18.815058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:43:18.815058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:43:18.815058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:43:18.815058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:43:18.815058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:18.815058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:43:18.815058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:18.815058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:43:18.815058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:18.815699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:43:18.815699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:18.815699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:43:18.815699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:18.815699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:43:18.815699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:43:18.815699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:18.815699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:43:18.815699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:18.815699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:43:18.815699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:43:18.815699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:18.815699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:43:18.816354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:18.816354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:43:18.816354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:18.816354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:43:18.816354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:43:18.816354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:43:18.816354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:43:18.816354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:43:18.816354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:43:18.816354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:43:18.816354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:43:18.816354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:43:18.816354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:43:18.816354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:43:18.816354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:43:18.817083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:43:18.817083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:43:18.817083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:43:18.817083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:43:18.817083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:43:18.817083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:18.817083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:43:18.817083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:18.817083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:43:18.817083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:18.817083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:43:18.817083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:18.817083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:43:18.817083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:18.817083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:43:18.817083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:43:18.817083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:18.817083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:43:18.817083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:18.818004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:43:18.818004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:43:18.818004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:18.818004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:43:18.818004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:18.818004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:43:23.641478 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:43:23.642314 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:43:23.642314 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:43:23.643651 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:43:23.680592 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:43:23.680891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:23.681466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:43:23.681466 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:23.681466 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:43:23.681466 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:43:23.681466 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:23.681466 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:43:23.681466 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:23.681466 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:43:23.682508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:23.682508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:43:23.682508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:43:23.682508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:43:23.682508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:43:23.682508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:43:23.682508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:43:23.682508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:23.682508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:43:23.682508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:23.682508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:43:23.682508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:23.683269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:43:23.683269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:23.683269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:43:23.683269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:23.683269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:43:23.683269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:43:23.683269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:23.683269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:43:23.683269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:23.683269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:43:23.683269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:43:23.683269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:23.683269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:43:23.684111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:23.684111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:43:23.684111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:23.684111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:43:23.684111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:43:23.684111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:43:23.684111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:43:23.684111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:43:23.684111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:43:23.684111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:43:23.684111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:43:23.684111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:43:23.684111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:43:23.684111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:43:23.684111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:43:23.684583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:43:23.684583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:43:23.684583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:43:23.684583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:43:23.684583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:43:23.684583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:23.684583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:43:23.684583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:23.684583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:43:23.684583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:23.684583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:43:23.684583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:23.684583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:43:23.684583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:23.684583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:43:23.684583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:43:23.684583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:23.684583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:43:23.684583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:23.685150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:43:23.685150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:43:23.685150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:23.685150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:43:23.685150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:23.685150 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:43:23.787718 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:43:23.788243 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:43:23.788390 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:43:23.789740 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:43:23.790182 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:43:23.790446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:23.791480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:43:23.791480 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:23.791480 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:43:23.791480 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:43:23.791480 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:23.791480 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:43:23.791480 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:23.791480 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:43:23.793129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:23.793129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:43:23.793129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:43:23.793129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:43:23.793129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:43:23.793129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:43:23.793129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:43:23.793129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:23.793129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:43:23.793129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:23.793129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:43:23.793129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:23.793775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:43:23.793775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:23.793775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:43:23.793775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:23.793775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:43:23.793775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:43:23.793775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:23.793775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:43:23.793775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:23.793775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:43:23.793775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:43:23.793775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:23.793775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:43:23.794329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:23.794329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:43:23.794856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:23.794856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:43:23.794856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:43:23.794856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:43:23.794856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:43:23.794856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:43:23.794856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:43:23.794856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:43:23.794856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:43:23.794856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:43:23.794856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:43:23.794856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:43:23.794856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:43:23.794856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:43:23.794856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:43:23.794856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:43:23.795838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:43:23.795838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:43:23.795838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:23.795838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:43:23.795838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:23.795838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:43:23.795838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:23.795838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:43:23.795838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:23.795838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:43:23.795838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:23.795838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:43:23.795838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:43:23.795838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:23.795838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:43:23.795838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:23.795838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:43:23.795838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:43:23.795838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:23.797161 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:43:23.797161 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:23.797161 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:43:23.797161 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:43:23.797161 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:43:23.797161 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:43:23.797161 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:43:23.797161 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:43:23.797830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:23.798121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:43:23.798121 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:23.798121 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:43:23.798121 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:43:23.798121 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:23.798121 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:43:23.798121 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:23.798121 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:43:23.798886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:23.798886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:43:23.798886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:43:23.798886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:43:23.798886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:23.798886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:43:23.798886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:23.798886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:43:23.798886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:23.798886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:43:23.798886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:23.798886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:43:23.798886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:23.798886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:43:23.798886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:43:23.799347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:23.799347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:43:23.799347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:23.799347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:43:23.799347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:43:23.799347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:23.799347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:43:23.799347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:23.799347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:43:23.799733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:23.799733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:43:23.799733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:43:23.799733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:43:23.799733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:43:23.799733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:43:23.799733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:43:23.799733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:43:23.799733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:43:23.799733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:43:23.799733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:23.799733 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:43:23.800080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:23.800080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:43:23.800080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:23.800080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:43:23.800080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:23.800080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:43:23.800080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:23.800080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:43:23.800080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:43:23.800080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:23.800080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:43:23.800080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:23.800080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:43:23.800080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:43:23.800430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:23.800430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:43:23.800430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:23.800430 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:43:23.800941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:23.800941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:43:23.800941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:43:23.800941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:43:23.800941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:43:23.800941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:43:23.800941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:43:23.800941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:43:23.800941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:43:23.800941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:43:23.800941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:43:23.800941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:43:23.800941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:43:23.800941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:43:23.800941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:43:23.800941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:43:23.801348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:43:23.801348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:43:23.801348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:43:23.801348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:43:23.801348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:43:23.801348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:23.801348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:43:23.801348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:23.801348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:43:23.801348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:23.801348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:43:23.801348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:23.801348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:43:23.801348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:23.801348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:43:23.801348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:43:23.801348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:23.801348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:43:23.801819 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:23.801819 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:43:23.801819 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:43:23.801819 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:23.801819 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:43:23.801819 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:23.801819 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:43:23.802383 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:43:23.802677 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:43:23.802827 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:43:23.802937 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:43:23.803347 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:43:23.803758 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:23.804516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:43:23.804516 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:23.804516 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:43:23.804516 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:43:23.804516 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:23.804516 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:43:23.804516 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:23.804516 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:43:23.805237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:23.805237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:43:23.805237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:43:23.805237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:43:23.805237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:23.805237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:43:23.805237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:23.805237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:43:23.805237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:23.805237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:43:23.805237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:23.805237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:43:23.805237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:23.805237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:43:23.805237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:43:23.805626 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:23.805626 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:43:23.805626 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:23.805626 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:43:23.805626 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:43:23.805626 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:23.805626 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:43:23.805626 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:23.805626 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:43:23.806122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:23.806122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:43:23.806122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:43:23.806122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:43:23.806122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:43:23.806122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:43:23.806122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:43:23.806122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:43:23.806122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:43:23.806122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:23.806122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:43:23.806547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:23.806547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:43:23.806547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:23.806547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:43:23.806547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:23.806547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:43:23.806547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:23.806547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:43:23.806547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:43:23.806547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:23.806547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:43:23.806547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:23.806547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:43:23.806547 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:43:23.807050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:23.807050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:43:23.807050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:23.807050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:43:23.807244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:23.807244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:43:23.807244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:43:23.807244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:43:23.807244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:43:23.807244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:43:23.807244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:43:23.807244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:43:23.807244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:43:23.807244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:43:23.807244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:43:23.807244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:43:23.807244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:43:23.807244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:43:23.807244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:43:23.807244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:43:23.807244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:43:23.807669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:43:23.807669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:43:23.807669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:43:23.807669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:23.807669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:43:23.807669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:23.807669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:43:23.807669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:23.807669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:43:23.807669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:23.807669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:43:23.807669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:23.807669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:43:23.807669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:43:23.807669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:23.807669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:43:23.807669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:23.807669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:43:23.807669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:43:23.808303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:23.808303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:43:23.808303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:23.808303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:43:23.812974 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:43:23.813207 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:43:23.813317 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:43:23.813463 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:43:23.813888 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:43:23.814181 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:43:23.814408 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:23.815134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:43:23.815134 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:23.815134 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:43:23.815134 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:43:23.815134 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:23.815134 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:43:23.815134 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:23.815134 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:43:23.816004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:23.816004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:43:23.816004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:43:23.816004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:43:23.816004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:43:23.816004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:43:23.816004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:43:23.816004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:23.816004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:43:23.816004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:23.816004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:43:23.816004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:23.816345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:43:23.816345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:23.816345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:43:23.816345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:23.816345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:43:23.816345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:43:23.816345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:23.816345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:43:23.816345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:23.816345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:43:23.816345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:43:23.816345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:23.816345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:43:23.816687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:23.816687 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:43:23.817045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:23.817045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:43:23.817045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:43:23.817045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:43:23.817045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:43:23.817045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:43:23.817045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:43:23.817045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:43:23.817045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:43:23.817045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:43:23.817045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:43:23.817045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:43:23.817045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:43:23.817045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:43:23.817045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:43:23.817045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:43:23.817463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:43:23.817463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:43:23.817463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:23.817463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:43:23.817463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:23.817463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:43:23.817463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:23.817463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:43:23.817463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:23.817463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:43:23.817463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:23.817463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:43:23.817463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:43:23.817463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:23.817463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:43:23.817463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:23.817463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:43:23.817463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:43:23.817463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:23.817977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:43:23.817977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:23.817977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:43:28.642207 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:43:28.643086 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:43:28.643086 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:43:28.644403 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:43:28.678767 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:43:28.679059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:28.679762 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:43:28.679762 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:28.679762 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:43:28.679762 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:43:28.679762 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:28.679762 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:43:28.679762 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:28.679762 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:43:28.680591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:28.680591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:43:28.680591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:43:28.680591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:43:28.680591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:43:28.680591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:43:28.680591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:43:28.680591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:28.680591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:43:28.680591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:28.680591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:43:28.680591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:28.681255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:43:28.681255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:28.681255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:43:28.681255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:28.681255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:43:28.681255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:43:28.681255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:28.681255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:43:28.681255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:28.681255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:43:28.681255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:43:28.681255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:28.681255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:43:28.681698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:28.681698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:43:28.681698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:28.681698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:43:28.681698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:43:28.681698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:43:28.681698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:43:28.681698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:43:28.681698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:43:28.681698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:43:28.681698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:43:28.681698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:43:28.681698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:43:28.681698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:43:28.681698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:43:28.682220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:43:28.682220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:43:28.682220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:43:28.682220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:43:28.682220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:43:28.682220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:28.682220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:43:28.682220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:28.682220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:43:28.682220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:28.682220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:43:28.682220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:28.682220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:43:28.682220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:28.682220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:43:28.682220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:43:28.682220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:28.682220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:43:28.682220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:28.683006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:43:28.683006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:43:28.683006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:28.683006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:43:28.683006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:28.683006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:43:28.788461 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:43:28.789226 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:43:28.789450 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:43:28.790869 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:43:28.791335 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:43:28.791638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:28.792791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:43:28.792791 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:28.792791 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:43:28.792791 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:43:28.792791 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:28.792791 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:43:28.792791 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:28.792791 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:43:28.794741 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:28.794741 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:43:28.794741 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:43:28.794741 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:43:28.794741 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:43:28.794741 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:43:28.794741 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:43:28.794741 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:28.794741 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:43:28.794741 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:28.794741 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:43:28.794741 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:28.795402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:43:28.795402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:28.795402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:43:28.795402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:28.795402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:43:28.795402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:43:28.795402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:28.795402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:43:28.795402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:28.795402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:43:28.795402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:43:28.795402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:28.795402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:43:28.795973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:28.795973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:43:28.796623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:28.796623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:43:28.796623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:43:28.796623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:43:28.796623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:43:28.796623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:43:28.796623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:43:28.796623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:43:28.796623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:43:28.796623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:43:28.796623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:43:28.796623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:43:28.796623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:43:28.796623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:43:28.796623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:43:28.796623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:43:28.797876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:43:28.797876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:43:28.797876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:28.797876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:43:28.797876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:28.797876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:43:28.797876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:28.797876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:43:28.797876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:28.797876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:43:28.797876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:28.797876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:43:28.797876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:43:28.797876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:28.797876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:43:28.797876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:28.797876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:43:28.797876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:43:28.797876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:28.798917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:43:28.798917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:28.798917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:43:28.798917 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:43:28.798917 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:43:28.798917 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:43:28.798917 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:43:28.798917 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:43:28.799504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:28.799504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:43:28.799504 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:28.799504 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:43:28.799504 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:43:28.799504 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:28.799504 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:43:28.799504 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:28.799504 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:43:28.799504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:28.799504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:43:28.799504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:43:28.799504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:43:28.799504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:28.799504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:43:28.799504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:28.799504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:43:28.800274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:28.800274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:43:28.800274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:28.800274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:43:28.800274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:28.800274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:43:28.800274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:43:28.800274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:28.800274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:43:28.800274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:28.800274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:43:28.800274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:43:28.800274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:28.800274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:43:28.800274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:28.800274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:43:28.800715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:28.800715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:43:28.800715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:43:28.800715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:43:28.800715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:43:28.800715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:43:28.800715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:43:28.800715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:43:28.800715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:43:28.800715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:43:28.800715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:28.800715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:43:28.801427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:28.801427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:43:28.801427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:28.801427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:43:28.801427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:28.801427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:43:28.801427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:28.801427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:43:28.801427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:43:28.801427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:28.801427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:43:28.801427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:28.801427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:43:28.801427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:43:28.802445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:28.802445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:43:28.802445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:28.802445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:43:28.802445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:28.802445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:43:28.802445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:43:28.802445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:43:28.802445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:43:28.802445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:43:28.802445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:43:28.802445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:43:28.802445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:43:28.802445 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:43:28.803528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:43:28.803528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:43:28.803528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:43:28.803528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:43:28.803528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:43:28.803528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:43:28.803528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:43:28.803528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:43:28.803528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:43:28.803528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:43:28.803528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:43:28.803528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:28.803528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:43:28.803528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:28.803528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:43:28.803528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:28.803528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:43:28.803528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:28.803528 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:43:28.804214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:28.804214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:43:28.804214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:43:28.804214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:28.804214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:43:28.804214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:28.804214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:43:28.804214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:43:28.804214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:28.804214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:43:28.804214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:28.804214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:43:28.804214 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:43:28.804214 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:43:28.804620 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:43:28.804620 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:43:28.804620 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:43:28.804620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:28.805106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:43:28.805106 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:28.805106 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:43:28.805106 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:43:28.805106 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:28.805106 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:43:28.805106 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:28.805106 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:43:28.806102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:28.806102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:43:28.806102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:43:28.806102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:43:28.806102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:28.806102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:43:28.806102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:28.806102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:43:28.806102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:28.806102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:43:28.806102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:28.806102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:43:28.806102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:28.806102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:43:28.806102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:43:28.806991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:28.806991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:43:28.806991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:28.806991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:43:28.806991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:43:28.806991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:28.806991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:43:28.806991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:28.806991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:43:28.807402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:28.807402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:43:28.807402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:43:28.807402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:43:28.807402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:43:28.807402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:43:28.807402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:43:28.807402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:43:28.807402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:43:28.807402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:28.807402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:43:28.807947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:28.807947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:43:28.807947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:28.807947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:43:28.807947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:28.807947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:43:28.807947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:28.807947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:43:28.807947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:43:28.807947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:28.807947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:43:28.807947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:28.807947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:43:28.807947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:43:28.808211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:28.808211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:43:28.808211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:28.808211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:43:28.808423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:28.808423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:43:28.808423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:43:28.808423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:43:28.808423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:43:28.808423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:43:28.808423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:43:28.808423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:43:28.808423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:43:28.808423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:43:28.808423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:43:28.808423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:43:28.808423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:43:28.808423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:43:28.808423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:43:28.808423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:43:28.808423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:43:28.809121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:43:28.809121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:43:28.809121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:43:28.809121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:28.809121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:43:28.809121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:28.809121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:43:28.809121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:28.809121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:43:28.809121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:28.809121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:43:28.809121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:28.809121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:43:28.809121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:43:28.809121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:28.809121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:43:28.809121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:28.809121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:43:28.809121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:43:28.809501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:28.809501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:43:28.809501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:28.809501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:43:28.813423 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:43:28.813795 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:43:28.813916 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:43:28.814069 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:43:28.814503 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:43:28.815020 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:43:28.815235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:28.815865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:43:28.815865 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:28.815865 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:43:28.815865 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:43:28.815865 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:28.815865 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:43:28.815865 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:28.815865 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:43:28.816725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:28.816725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:43:28.816725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:43:28.816725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:43:28.816725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:43:28.816725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:43:28.816725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:43:28.816725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:28.816725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:43:28.816725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:28.816725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:43:28.816725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:28.817102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:43:28.817102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:28.817102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:43:28.817102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:28.817102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:43:28.817102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:43:28.817102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:28.817102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:43:28.817102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:28.817102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:43:28.817102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:43:28.817102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:28.817102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:43:28.817520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:28.817520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:43:28.817798 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:28.817798 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:43:28.817798 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:43:28.817798 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:43:28.817798 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:43:28.817798 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:43:28.817798 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:43:28.817798 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:43:28.817798 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:43:28.817798 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:43:28.817798 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:43:28.817798 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:43:28.817798 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:43:28.817798 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:43:28.817798 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:43:28.817798 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:43:28.818228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:43:28.818228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:43:28.818228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:28.818228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:43:28.818228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:28.818228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:43:28.818228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:28.818228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:43:28.818228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:28.818228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:43:28.818228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:28.818228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:43:28.818228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:43:28.818228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:28.818228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:43:28.818228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:28.818228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:43:28.818228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:43:28.818228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:28.818780 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:43:28.818780 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:28.818780 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:43:33.642920 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:43:33.643583 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:43:33.643705 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:43:33.644549 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:43:33.677494 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:43:33.677494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:33.677494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:43:33.677494 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:33.677494 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:43:33.677494 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:43:33.677494 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:33.677494 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:43:33.677494 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:33.677494 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:43:33.678378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:33.678378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:43:33.678378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:43:33.678378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:43:33.678378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:43:33.678378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:43:33.678378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:43:33.678378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:33.678378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:43:33.678378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:33.678378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:43:33.678378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:33.679491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:43:33.679491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:33.679491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:43:33.679491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:33.679491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:43:33.679491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:43:33.679491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:33.679491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:43:33.679491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:33.679491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:43:33.679491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:43:33.679491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:33.679491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:43:33.681115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:33.681115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:43:33.681115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:33.681115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:43:33.681115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:43:33.681115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:43:33.681115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:43:33.681115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:43:33.681115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:43:33.681115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:43:33.681115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:43:33.681115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:43:33.681115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:43:33.681115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:43:33.681115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:43:33.682081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:43:33.682081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:43:33.682081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:43:33.682081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:43:33.682081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:43:33.682081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:33.682081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:43:33.682081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:33.682081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:43:33.682081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:33.682081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:43:33.682081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:33.682081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:43:33.682081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:33.682081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:43:33.682081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:43:33.682081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:33.682081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:43:33.682081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:33.685064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:43:33.685064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:43:33.685064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:33.685064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:43:33.685064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:33.685064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:43:33.789254 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:43:33.789909 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:43:33.790130 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:43:33.791103 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:43:33.792013 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:43:33.792013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:33.792777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:43:33.792777 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:33.792777 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:43:33.792777 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:43:33.792777 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:33.792777 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:43:33.792777 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:33.792777 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:43:33.794783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:33.794783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:43:33.794783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:43:33.794783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:43:33.794783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:43:33.794783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:43:33.794783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:43:33.794783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:33.794783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:43:33.794783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:33.794783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:43:33.794783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:33.796012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:43:33.796012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:33.796012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:43:33.796012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:33.796012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:43:33.796012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:43:33.796012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:33.796012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:43:33.796012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:33.796012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:43:33.796012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:43:33.796012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:33.796012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:43:33.796817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:33.796817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:43:33.796817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:33.796817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:43:33.796817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:43:33.796817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:43:33.796817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:43:33.796817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:43:33.796817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:43:33.796817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:43:33.796817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:43:33.796817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:43:33.796817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:43:33.796817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:43:33.796817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:43:33.797560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:43:33.797560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:43:33.797560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:43:33.797560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:43:33.797560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:43:33.797560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:33.797560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:43:33.797560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:33.797560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:43:33.797560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:33.797560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:43:33.797560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:33.797560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:43:33.797560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:33.797560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:43:33.797560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:43:33.797560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:33.797560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:43:33.797560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:33.798507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:43:33.798507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:43:33.798507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:33.798507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:43:33.798507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:33.798507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:43:33.798507 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:43:33.798507 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:43:33.798507 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:43:33.798507 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:43:33.798885 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:43:33.798885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:33.799450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:43:33.799450 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:33.799450 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:43:33.799450 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:43:33.799450 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:33.799450 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:43:33.799450 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:33.799450 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:43:33.800248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:33.800248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:43:33.800248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:43:33.800248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:43:33.800248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:33.800248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:43:33.800248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:33.800248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:43:33.800248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:33.800248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:43:33.800248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:33.800248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:43:33.800248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:33.800248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:43:33.800248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:43:33.800769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:33.800769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:43:33.800769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:33.800769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:43:33.800769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:43:33.800769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:33.800769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:43:33.800769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:33.800769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:43:33.801135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:33.801135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:43:33.801135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:43:33.801135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:43:33.801135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:43:33.801135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:43:33.801135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:43:33.801135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:43:33.801135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:43:33.801135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:43:33.801135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:33.801135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:43:33.801764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:33.801764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:43:33.801764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:33.801764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:43:33.801764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:33.801764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:43:33.801764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:33.801764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:43:33.801764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:43:33.801764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:33.801764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:43:33.801764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:33.801764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:43:33.801764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:43:33.802137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:33.802137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:43:33.802137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:33.802137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:43:33.802294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:33.802294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:43:33.802294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:43:33.802294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:43:33.802294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:43:33.802294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:43:33.802294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:43:33.802294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:43:33.802294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:43:33.802294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:43:33.802294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:43:33.802294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:43:33.802294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:43:33.802294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:43:33.802294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:43:33.802294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:43:33.802828 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:43:33.802828 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:43:33.802828 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:43:33.802828 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:43:33.802828 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:43:33.802828 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:33.802828 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:43:33.802828 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:33.802828 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:43:33.802828 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:33.802828 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:43:33.802828 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:33.802828 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:43:33.802828 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:33.802828 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:43:33.802828 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:43:33.802828 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:33.802828 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:43:33.803357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:33.803357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:43:33.803357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:43:33.803357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:33.803357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:43:33.803357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:33.803357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:43:33.803357 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:43:33.803593 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:43:33.803637 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:43:33.803775 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:43:33.804380 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:43:33.804661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:33.805352 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:43:33.805352 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:33.805352 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:43:33.805352 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:43:33.805352 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:33.805352 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:43:33.805352 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:33.805352 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:43:33.806209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:33.806209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:43:33.806209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:43:33.806209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:43:33.806209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:33.806209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:43:33.806209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:33.806209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:43:33.806209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:33.806209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:43:33.806209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:33.806209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:43:33.806209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:33.806209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:43:33.806209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:43:33.806649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:33.806649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:43:33.806649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:33.806649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:43:33.806649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:43:33.806649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:33.806649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:43:33.806649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:33.806649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:43:33.807286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:33.807286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:43:33.807286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:43:33.807286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:43:33.807286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:43:33.807286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:43:33.807286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:43:33.807286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:43:33.807286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:43:33.807286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:33.807286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:43:33.807637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:33.807637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:43:33.807637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:33.807637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:43:33.807637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:33.807637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:43:33.807637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:33.807637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:43:33.807637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:43:33.807637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:33.807637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:43:33.807637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:33.807637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:43:33.807637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:43:33.808087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:33.808087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:43:33.808087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:33.808087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:43:33.808366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:33.808366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:43:33.808366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:43:33.808366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:43:33.808366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:43:33.808366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:43:33.808366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:43:33.808366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:43:33.808366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:43:33.808366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:43:33.808366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:43:33.808366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:43:33.808366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:43:33.808366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:43:33.808366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:43:33.808366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:43:33.808366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:43:33.808897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:43:33.808897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:43:33.808897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:43:33.808897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:33.808897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:43:33.808897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:33.808897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:43:33.808897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:33.808897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:43:33.808897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:33.808897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:43:33.808897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:33.808897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:43:33.808897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:43:33.808897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:33.808897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:43:33.808897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:33.808897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:43:33.808897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:43:33.809444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:33.809444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:43:33.809444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:33.809444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:43:33.813853 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:43:33.814206 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:43:33.814329 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:43:33.814489 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:43:33.814896 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:43:33.815177 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:43:33.815333 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:33.815945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:43:33.815945 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:33.815945 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:43:33.815945 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:43:33.815945 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:33.815945 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:43:33.815945 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:33.815945 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:43:33.817367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:33.817367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:43:33.817367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:43:33.817367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:43:33.817367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:43:33.817367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:43:33.817367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:43:33.817367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:33.817367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:43:33.817367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:33.817367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:43:33.817367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:33.817807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:43:33.817807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:33.817807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:43:33.817807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:33.817807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:43:33.817807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:43:33.817807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:33.817807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:43:33.817807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:33.817807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:43:33.817807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:43:33.817807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:33.817807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:43:33.818267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:33.818267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:43:33.818998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:33.818998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:43:33.818998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:43:33.818998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:43:33.818998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:43:33.818998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:43:33.818998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:43:33.818998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:43:33.818998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:43:33.818998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:43:33.818998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:43:33.818998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:43:33.818998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:43:33.818998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:43:33.818998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:43:33.818998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:43:33.819461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:43:33.819461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:43:33.819461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:33.819461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:43:33.819461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:33.819461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:43:33.819461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:33.819461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:43:33.819461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:33.819461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:43:33.819461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:33.819461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:43:33.819461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:43:33.819461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:33.819461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:43:33.819461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:33.819461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:43:33.819461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:43:33.819461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:33.820966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:43:33.820966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:33.820966 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:43:38.644669 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:43:38.644669 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:43:38.644669 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:43:38.646257 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:43:38.690476 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:43:38.690913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:38.691933 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:43:38.691933 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:38.691933 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:43:38.691933 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:43:38.691933 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:38.691933 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:43:38.691933 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:38.691933 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:43:38.692680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.047 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:38.692680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:43:38.692680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:43:38.692680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:43:38.692680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:43:38.692680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:43:38.692680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:43:38.692680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:38.692680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:43:38.692680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:38.692680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:43:38.692680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:38.693119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:43:38.693119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:38.693119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:43:38.693119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:38.693119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:43:38.693119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:43:38.693119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:38.693119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:43:38.693119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:38.693119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:43:38.693119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:43:38.693119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:38.693119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:43:38.693545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:38.693545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:43:38.693545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:38.693545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:43:38.693545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:43:38.693545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:43:38.693545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:43:38.693545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:43:38.693545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:43:38.693545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:43:38.693545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:43:38.693545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:43:38.693545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:43:38.693545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:43:38.693545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:43:38.694152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:43:38.694152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:43:38.694152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:43:38.694152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:43:38.694152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:43:38.694152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:38.694152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:43:38.694152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:38.694152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:43:38.694152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:38.694152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:43:38.694152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:38.694152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:43:38.694152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:38.694152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:43:38.694152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:43:38.694152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:38.694152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:43:38.694152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:38.694945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:43:38.694945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:43:38.694945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:38.694945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:43:38.694945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:38.694945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:43:38.790040 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:43:38.790612 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:43:38.790880 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:43:38.791985 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:43:38.792328 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:43:38.792566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:38.793397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:43:38.793397 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:38.793397 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:43:38.793397 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:43:38.793397 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:38.793397 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:43:38.793397 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:38.793397 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:43:38.794955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:38.794955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:43:38.794955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:43:38.794955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:43:38.794955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:43:38.794955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:43:38.794955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:43:38.794955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:38.794955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:43:38.794955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:38.794955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:43:38.794955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:38.795371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:43:38.795371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:38.795371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:43:38.795371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:38.795371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:43:38.795371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:43:38.795371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:38.795371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:43:38.795371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:38.795371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:43:38.795371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:43:38.795371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:38.795371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:43:38.795713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:38.795713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:43:38.796766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:38.796766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:43:38.796766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:43:38.796766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:43:38.796766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:43:38.796766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:43:38.796766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:43:38.796766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:43:38.796766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:43:38.796766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:43:38.796766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:43:38.796766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:43:38.796766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:43:38.796766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:43:38.796766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:43:38.796766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:43:38.801537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:43:38.801537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:43:38.801537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:38.801537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:43:38.801537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:38.801537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:43:38.801537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:38.801537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:43:38.801537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:38.801537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:43:38.801537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:38.801537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:43:38.801537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:43:38.801537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:38.801537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:43:38.801537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:38.801537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:43:38.801537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:43:38.801537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:38.802057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:43:38.802057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:38.802057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:43:38.802057 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:43:38.802057 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:43:38.802057 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:43:38.802057 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:43:38.802057 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:43:38.802302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:38.802302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:43:38.802302 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:38.802302 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:43:38.802302 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:43:38.802302 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:38.802302 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:43:38.802302 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:38.802302 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:43:38.802302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:38.802302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:43:38.802302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:43:38.802302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:43:38.802302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:38.802302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:43:38.802302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:38.802302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:43:38.802828 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:38.802828 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:43:38.802828 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:38.802828 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:43:38.802828 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:38.802828 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:43:38.802828 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:43:38.802828 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:38.802828 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:43:38.802828 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:38.802828 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:43:38.802828 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:43:38.802828 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:38.802828 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:43:38.802828 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:38.802828 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:43:38.803316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:38.803316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:43:38.803316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:43:38.803316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:43:38.803316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:43:38.803316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:43:38.803316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:43:38.803316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:43:38.803316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:43:38.803316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:43:38.803316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:38.803316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:43:38.803651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:38.803651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:43:38.803651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:38.803651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:43:38.803651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:38.803651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:43:38.803651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:38.803651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:43:38.803651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:43:38.803651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:38.803651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:43:38.803651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:38.803651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:43:38.803651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:43:38.804005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:38.804005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:43:38.804005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:38.804005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:43:38.804005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:38.804005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:43:38.804005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:43:38.804005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:43:38.804005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:43:38.804005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:43:38.804005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:43:38.804005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:43:38.804005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:43:38.804005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:43:38.804372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:43:38.804372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:43:38.804372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:43:38.804372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:43:38.804372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:43:38.804372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:43:38.804372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:43:38.804372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:43:38.804372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:43:38.804372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:43:38.804372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:43:38.804372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:38.804372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:43:38.804372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:38.804372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:43:38.804372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:38.804372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:43:38.804372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:38.804372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:43:38.804872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:38.804872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:43:38.804872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:43:38.804872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:38.804872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:43:38.804872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:38.804872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:43:38.804872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:43:38.804872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:38.804872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:43:38.804872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:38.804872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:43:38.804872 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:43:38.804872 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:43:38.805228 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:43:38.805228 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:43:38.805228 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:43:38.805228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:38.805472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:43:38.805472 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:38.805472 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:43:38.805472 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:43:38.805472 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:38.805472 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:43:38.805472 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:38.805472 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:43:38.806224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:38.806224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:43:38.806224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:43:38.806224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:43:38.806224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:38.806224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:43:38.806224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:38.806224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:43:38.806224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:38.806224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:43:38.806224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:38.806224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:43:38.806224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:38.806224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:43:38.806224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:43:38.806667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:38.806667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:43:38.806667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:38.806667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:43:38.806667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:43:38.806667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:38.806667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:43:38.806667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:38.806667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:43:38.807245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:38.807245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:43:38.807245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:43:38.807245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:43:38.807245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:43:38.807245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:43:38.807245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:43:38.807245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:43:38.807245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:43:38.807245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:38.807245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:43:38.807597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:38.807597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:43:38.807597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:38.807597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:43:38.807597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:38.807597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:43:38.807597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:38.807597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:43:38.807597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:43:38.807597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:38.807597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:43:38.807597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:38.807597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:43:38.807597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:43:38.807968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:38.807968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:43:38.807968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:38.807968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:43:38.808424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:38.808424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:43:38.808424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:43:38.808424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:43:38.808424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:43:38.808424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:43:38.808424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:43:38.808424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:43:38.808424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:43:38.808424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:43:38.808424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:43:38.808424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:43:38.808424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:43:38.808424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:43:38.808424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:43:38.808424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:43:38.808424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:43:38.808918 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:43:38.808918 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:43:38.808918 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:43:38.808918 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:38.808918 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:43:38.808918 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:38.808918 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:43:38.808918 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:38.808918 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:43:38.808918 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:38.808918 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:43:38.808918 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:38.808918 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:43:38.808918 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:43:38.808918 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:38.808918 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:43:38.808918 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:38.808918 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:43:38.808918 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:43:38.809407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:38.809407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:43:38.809407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:38.809407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:43:38.814198 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:43:38.814490 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:43:38.814608 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:43:38.814796 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:43:38.815488 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:43:38.816057 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:43:38.816276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:38.816978 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:43:38.816978 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:38.816978 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:43:38.816978 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:43:38.816978 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:38.816978 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:43:38.816978 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:38.816978 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:43:38.817911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:38.817911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:43:38.817911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:43:38.817911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:43:38.817911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:43:38.817911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:43:38.817911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:43:38.817911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:38.817911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:43:38.817911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:38.817911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:43:38.817911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:38.818270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:43:38.818270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:38.818270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:43:38.818270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:38.818270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:43:38.818270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:43:38.818270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:38.818270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:43:38.818270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:38.818270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:43:38.818270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:43:38.818270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:38.818270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:43:38.818585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:38.818585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:43:38.818962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:38.818962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:43:38.818962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:43:38.818962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:43:38.818962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:43:38.818962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:43:38.818962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:43:38.818962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:43:38.818962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:43:38.818962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:43:38.818962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:43:38.818962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:43:38.818962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:43:38.818962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:43:38.818962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:43:38.818962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:43:38.819397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:43:38.819397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:43:38.819397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:38.819397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:43:38.819397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:38.819397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:43:38.819397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:38.819397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:43:38.819397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:38.819397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:43:38.819397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:38.819397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:43:38.819397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:43:38.819397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:38.819397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:43:38.819397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:38.819397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:43:38.819397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:43:38.819397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:38.819947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:43:38.819947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:38.819947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:43:43.644628 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:43:43.645358 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:43:43.645526 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:43:43.646933 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:43:43.687633 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:43:43.688128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:43.688818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:43:43.688818 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:43.688818 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:43:43.688818 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:43:43.688818 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:43.688818 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:43:43.688818 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:43.688818 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:43:43.689566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.043 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:43.689566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:43:43.689566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:43:43.689566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:43:43.689566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:43:43.689566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:43:43.689566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:43:43.689566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:43.689566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:43:43.689566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:43.689566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:43:43.689566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:43.690098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:43:43.690098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:43.690098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:43:43.690098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:43.690098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:43:43.690098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:43:43.690098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:43.690098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:43:43.690098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:43.690098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:43:43.690098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:43:43.690098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:43.690098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:43:43.690585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:43.690585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:43:43.690585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:43.690585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:43:43.690585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:43:43.690585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:43:43.690585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:43:43.690585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:43:43.690585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:43:43.690585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:43:43.690585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:43:43.690585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:43:43.690585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:43:43.690585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:43:43.690585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:43:43.691154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:43:43.691154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:43:43.691154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:43:43.691154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:43:43.691154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:43:43.691154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:43.691154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:43:43.691154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:43.691154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:43:43.691154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:43.691154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:43:43.691154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:43.691154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:43:43.691154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:43.691154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:43:43.691154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:43:43.691154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:43.691154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:43:43.691154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:43.691819 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:43:43.691819 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:43:43.691819 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:43.691819 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:43:43.691819 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:43.691819 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:43:43.790777 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:43:43.791399 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:43:43.791659 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:43:43.792702 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:43:43.793072 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:43:43.793389 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:43.794605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:43:43.794605 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:43.794605 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:43:43.794605 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:43:43.794605 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:43.794605 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:43:43.794605 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:43.794605 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:43:43.796141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:43.796141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:43:43.796141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:43:43.796141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:43:43.796141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:43:43.796141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:43:43.796141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:43:43.796141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:43.796141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:43:43.796141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:43.796141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:43:43.796141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:43.796900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:43:43.796900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:43.796900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:43:43.796900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:43.796900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:43:43.796900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:43:43.796900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:43.796900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:43:43.796900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:43.796900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:43:43.796900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:43:43.796900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:43.796900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:43:43.797473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:43.797473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:43:43.797920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:43.797920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:43:43.797920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:43:43.797920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:43:43.797920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:43:43.797920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:43:43.797920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:43:43.797920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:43:43.797920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:43:43.797920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:43:43.797920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:43:43.797920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:43:43.797920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:43:43.797920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:43:43.797920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:43:43.797920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:43:43.798639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:43:43.798639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:43:43.798639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:43.798639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:43:43.798639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:43.798639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:43:43.798639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:43.798639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:43:43.798639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:43.798639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:43:43.798639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:43.798639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:43:43.798639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:43:43.798639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:43.798639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:43:43.798639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:43.798639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:43:43.798639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:43:43.798639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:43.799509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:43:43.799509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:43.799509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:43:43.799509 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:43:43.799509 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:43:43.799509 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:43:43.799509 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:43:43.799509 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:43:43.799975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:43.800776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:43:43.800776 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:43.800776 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:43:43.800776 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:43:43.800776 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:43.800776 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:43:43.800776 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:43.800776 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:43:43.801727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:43.801727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:43:43.801727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:43:43.801727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:43:43.801727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:43.801727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:43:43.801727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:43.801727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:43:43.801727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:43.801727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:43:43.801727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:43.801727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:43:43.801727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:43.801727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:43:43.801727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:43:43.802578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:43.802578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:43:43.802578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:43.802578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:43:43.802578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:43:43.802578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:43.802578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:43:43.802578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:43.802578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:43:43.803109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:43.803109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:43:43.803109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:43:43.803109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:43:43.803109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:43:43.803109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:43:43.803109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:43:43.803109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:43:43.803109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:43:43.803109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:43:43.803109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:43.803109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:43:43.803794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:43.803794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:43:43.803794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:43.803794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:43:43.803794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:43.803794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:43:43.803794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:43.803794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:43:43.803794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:43:43.803794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:43.803794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:43:43.803794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:43.803794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:43:43.803794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:43:43.804615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:43.804615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:43:43.804615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:43.804615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:43:43.804615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:43.804615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:43:43.804615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:43:43.804615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:43:43.804615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:43:43.804615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:43:43.804615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:43:43.804615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:43:43.804615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:43:43.804615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:43:43.805462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:43:43.805462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:43:43.805462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:43:43.805462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:43:43.805462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:43:43.805462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:43:43.805462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:43:43.805462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:43:43.805462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:43:43.805462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:43:43.805462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:43:43.805462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:43.805462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:43:43.805462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:43.805462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:43:43.805462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:43.805462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:43:43.805462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:43.805462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:43:43.806465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:43.806465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:43:43.806465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:43:43.806465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:43.806465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:43:43.806465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:43.806465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:43:43.806465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:43:43.806465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:43.806465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:43:43.806465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:43.806465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:43:43.806465 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:43:43.806465 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:43:43.807165 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:43:43.807165 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:43:43.807165 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:43:43.807165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:43.807165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:43:43.807165 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:43.807165 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:43:43.807165 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:43:43.807165 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:43.807165 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:43:43.807165 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:43.807165 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:43:43.808420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:43.808420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:43:43.808420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:43:43.808420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:43:43.808420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:43.808420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:43:43.808420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:43.808420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:43:43.808420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:43.808420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:43:43.808420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:43.808420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:43:43.808420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:43.808420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:43:43.808420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:43:43.809227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:43.809227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:43:43.809227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:43.809227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:43:43.809227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:43:43.809227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:43.809227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:43:43.809227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:43.809227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:43:43.809227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:43.809227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:43:43.809227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:43:43.809227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:43:43.809227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:43:43.809979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:43:43.809979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:43:43.809979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:43:43.809979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:43:43.809979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:43.809979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:43:43.809979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:43.809979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:43:43.809979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:43.809979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:43:43.809979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:43.809979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:43:43.809979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:43.810664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:43:43.810664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:43:43.810664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:43.810664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:43:43.810664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:43.810664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:43:43.810664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:43:43.810664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:43.810664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:43:43.810664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:43.810664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:43:43.810664 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:43.811283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:43:43.811283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:43:43.811283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:43:43.811283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:43:43.811283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:43:43.811283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:43:43.811283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:43:43.811283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:43:43.811283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:43:43.811283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:43:43.811283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:43:43.811283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:43:43.811283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:43:43.811283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:43:43.811283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:43:43.811283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:43:43.811283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:43:43.811283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:43:43.811283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:43:43.812312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:43.812312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:43:43.812312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:43.812312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:43:43.812312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:43.812312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:43:43.812312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:43.812312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:43:43.812312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:43.812312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:43:43.812312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:43:43.812312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:43.812312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:43:43.812312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:43.812312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:43:43.812312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:43:43.812312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:43.812312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:43:43.812312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:43.812312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:43:43.814619 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:43:43.815187 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:43:43.815187 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:43:43.815187 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:43:43.815408 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:43:43.815706 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:43:43.815813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:43.816345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:43:43.816345 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:43.816345 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:43:43.816345 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:43:43.816345 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:43.816345 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:43:43.816345 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:43.816345 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:43:43.816997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:43.816997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:43:43.816997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:43:43.816997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:43:43.816997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:43:43.816997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:43:43.816997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:43:43.816997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:43.816997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:43:43.816997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:43.816997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:43:43.816997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:43.817277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:43:43.817277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:43.817277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:43:43.817277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:43.817277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:43:43.817277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:43:43.817277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:43.817277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:43:43.817277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:43.817277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:43:43.817277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:43:43.817277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:43.817277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:43:43.817526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:43.817526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:43:43.817839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:43.817839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:43:43.817839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:43:43.817839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:43:43.817839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:43:43.817839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:43:43.817839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:43:43.817839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:43:43.817839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:43:43.817839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:43:43.817839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:43:43.817839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:43:43.817839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:43:43.817839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:43:43.817839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:43:43.817839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:43:43.818519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:43:43.818519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:43:43.818519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:43.818519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:43:43.818519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:43.818519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:43:43.818519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:43.818519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:43:43.818519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:43.818519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:43:43.818519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:43.818519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:43:43.818519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:43:43.818519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:43.818519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:43:43.818519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:43.818519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:43:43.818519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:43:43.818519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:43.818961 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:43:43.818961 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:43.818961 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:43:48.645519 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:43:48.646604 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:43:48.646604 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:43:48.646896 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:43:48.699531 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:43:48.699935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:48.700734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:43:48.700734 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:48.700734 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:43:48.700734 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:43:48.700734 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:48.700734 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:43:48.700734 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:48.700734 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:43:48.702160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.055 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:48.702160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:43:48.702160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:43:48.702160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:43:48.702160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:43:48.702160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:43:48.702160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:43:48.702160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:48.702160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:43:48.702160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:48.702160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:43:48.702160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:48.702724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:43:48.702724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:48.702724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:43:48.702724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:48.702724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:43:48.702724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:43:48.702724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:48.702724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:43:48.702724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:48.702724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:43:48.702724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:43:48.702724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:48.702724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:43:48.703106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:48.703106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:43:48.703581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:48.703581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:43:48.703581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:43:48.703581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:43:48.703581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:43:48.703581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:43:48.703581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:43:48.703581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:43:48.703581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:43:48.703581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:43:48.703581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:43:48.703581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:43:48.703581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:43:48.703581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:43:48.703581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:43:48.703581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:43:48.704035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:43:48.704035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:43:48.704035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:48.704035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:43:48.704035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:48.704035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:43:48.704035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:48.704035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:43:48.704035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:48.704035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:43:48.704035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:48.704035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:43:48.704035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:43:48.704035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:48.704035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:43:48.704035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:48.704035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:43:48.704035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:43:48.704035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:48.704508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:43:48.704508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:48.704508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:43:48.791534 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:43:48.792239 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:43:48.792421 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:43:48.793770 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:43:48.794101 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:43:48.794300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:48.795099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:43:48.795099 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:48.795099 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:43:48.795099 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:43:48.795099 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:48.795099 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:43:48.795099 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:48.795099 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:43:48.796835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:48.796835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:43:48.796835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:43:48.796835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:43:48.796835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:43:48.796835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:43:48.796835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:43:48.796835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:48.796835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:43:48.796835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:48.796835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:43:48.796835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:48.797586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:43:48.797586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:48.797586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:43:48.797586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:48.797586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:43:48.797586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:43:48.797586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:48.797586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:43:48.797586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:48.797586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:43:48.797586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:43:48.797586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:48.797586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:43:48.798203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:48.798203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:43:48.798583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:48.798583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:43:48.798583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:43:48.798583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:43:48.798583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:43:48.798583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:43:48.798583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:43:48.798583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:43:48.798583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:43:48.798583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:43:48.798583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:43:48.798583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:43:48.798583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:43:48.798583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:43:48.798583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:43:48.798583 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:43:48.799165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:43:48.799165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:43:48.799165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:48.799165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:43:48.799165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:48.799165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:43:48.799165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:48.799165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:43:48.799165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:48.799165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:43:48.799165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:48.799165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:43:48.799165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:43:48.799165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:48.799165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:43:48.799165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:48.799165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:43:48.799165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:43:48.799165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:48.799704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:43:48.799704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:48.799704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:43:48.799704 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:43:48.799704 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:43:48.799704 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:43:48.799704 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:43:48.800240 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:43:48.800450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:48.801084 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:43:48.801084 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:48.801084 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:43:48.801084 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:43:48.801084 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:48.801084 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:43:48.801084 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:48.801084 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:43:48.801922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:48.801922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:43:48.801922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:43:48.801922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:43:48.801922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:48.801922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:43:48.801922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:48.801922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:43:48.801922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:48.801922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:43:48.801922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:48.801922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:43:48.801922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:48.801922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:43:48.801922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:43:48.802474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:48.802474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:43:48.802474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:48.802474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:43:48.802474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:43:48.802474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:48.802474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:43:48.802474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:48.802474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:43:48.802814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:48.802814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:43:48.802814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:43:48.802814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:43:48.802814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:43:48.802814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:43:48.802814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:43:48.802814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:43:48.802814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:43:48.802814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:43:48.802814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:48.802814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:43:48.803201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:48.803201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:43:48.803201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:48.803201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:43:48.803201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:48.803201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:43:48.803201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:48.803201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:43:48.803201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:43:48.803201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:48.803201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:43:48.803201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:48.803201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:43:48.803201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:43:48.803598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:48.803598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:43:48.803598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:48.803598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:43:48.803939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:48.803939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:43:48.803939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:43:48.803939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:43:48.803939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:43:48.803939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:43:48.803939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:43:48.803939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:43:48.803939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:43:48.803939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:43:48.803939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:43:48.803939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:43:48.803939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:43:48.803939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:43:48.803939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:43:48.803939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:43:48.804419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:43:48.804419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:43:48.804419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:43:48.804419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:43:48.804419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:43:48.804419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:48.804419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:43:48.804419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:48.804419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:43:48.804419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:48.804419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:43:48.804419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:48.804419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:43:48.804419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:48.804419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:43:48.804419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:43:48.804419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:48.804419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:43:48.805040 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:48.805040 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:43:48.805040 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:43:48.805040 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:48.805040 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:43:48.805040 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:48.805040 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:43:48.805040 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:43:48.805403 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:43:48.805509 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:43:48.805705 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:43:48.806172 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:43:48.806428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:48.807111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:43:48.807111 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:48.807111 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:43:48.807111 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:43:48.807111 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:48.807111 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:43:48.807111 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:48.807111 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:43:48.808035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:48.808035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:43:48.808035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:43:48.808035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:43:48.808035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:48.808035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:43:48.808035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:48.808035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:43:48.808035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:48.808035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:43:48.808035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:48.808035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:43:48.808035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:48.808035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:43:48.808035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:43:48.808691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:48.808691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:43:48.808691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:48.808691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:43:48.808691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:43:48.808691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:48.808691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:43:48.808691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:48.808691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:43:48.809511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:48.809511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:43:48.809511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:43:48.809511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:43:48.809511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:43:48.809511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:43:48.809511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:43:48.809511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:43:48.809511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:43:48.809511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:48.809511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:43:48.810013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:48.810013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:43:48.810013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:48.810013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:43:48.810013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:48.810013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:43:48.810013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:48.810013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:43:48.810013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:43:48.810013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:48.810013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:43:48.810013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:48.810013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:43:48.810013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:43:48.810412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:48.810412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:43:48.810412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:48.810412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:43:48.810863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:48.810863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:43:48.810863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:43:48.810863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:43:48.810863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:43:48.810863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:43:48.810863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:43:48.810863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:43:48.810863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:43:48.810863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:43:48.810863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:43:48.810863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:43:48.810863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:43:48.810863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:43:48.810863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:43:48.810863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:43:48.810863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:43:48.811412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:43:48.811412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:43:48.811412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:43:48.811412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:48.811412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:43:48.811412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:48.811412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:43:48.811412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:48.811412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:43:48.811412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:48.811412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:43:48.811412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:48.811412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:43:48.811412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:43:48.811412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:48.811412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:43:48.811412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:48.811412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:43:48.811412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:43:48.812504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:48.812504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:43:48.812504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:48.812504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:43:48.815139 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:43:48.815554 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:43:48.815705 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:43:48.815966 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:43:48.816581 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:43:48.816991 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:43:48.817156 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:48.817867 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:43:48.817867 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:48.817867 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:43:48.817867 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:43:48.817867 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:48.817867 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:43:48.817867 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:48.817867 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:43:48.819123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:48.819123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:43:48.819123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:43:48.819123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:43:48.819123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:43:48.819123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:43:48.819123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:43:48.819123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:48.819123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:43:48.819123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:48.819123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:43:48.819123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:48.819683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:43:48.819683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:48.819683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:43:48.819683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:48.819683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:43:48.819683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:43:48.819683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:48.819683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:43:48.819683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:48.819683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:43:48.819683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:43:48.819683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:48.819683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:43:48.820293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:48.820293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:43:48.820383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:48.820383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:43:48.820383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:43:48.820383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:43:48.820383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:43:48.820383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:43:48.820383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:43:48.820383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:43:48.820383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:43:48.820383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:43:48.820383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:43:48.820383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:43:48.820383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:43:48.820383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:43:48.820383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:43:48.820383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:43:48.821046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:43:48.821046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:43:48.821046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:48.821046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:43:48.821046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:48.821046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:43:48.821046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:48.821046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:43:48.821046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:48.821046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:43:48.821046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:48.821046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:43:48.821046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:43:48.821046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:48.821046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:43:48.821046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:48.821046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:43:48.821046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:43:48.821046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:48.821765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:43:48.821765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:48.821765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:43:53.647305 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:43:53.647305 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:43:53.647305 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:43:53.649261 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:43:53.691253 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:43:53.691562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:53.692516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:43:53.692516 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:53.692516 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:43:53.692516 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:43:53.692516 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:53.692516 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:43:53.692516 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:53.692516 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:43:53.693626 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.045 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:53.693626 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:43:53.693626 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:43:53.693626 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:43:53.693626 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:43:53.693626 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:43:53.693626 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:43:53.693626 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:53.693626 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:43:53.693626 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:53.693626 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:43:53.693626 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:53.694151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:43:53.694151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:53.694151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:43:53.694151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:53.694151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:43:53.694151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:43:53.694151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:53.694151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:43:53.694151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:53.694151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:43:53.694151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:43:53.694151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:53.694151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:43:53.694946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:53.694946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:43:53.694946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:53.694946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:43:53.694946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:43:53.694946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:43:53.694946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:43:53.694946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:43:53.694946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:43:53.694946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:43:53.694946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:43:53.694946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:43:53.694946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:43:53.694946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:43:53.694946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:43:53.696343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:43:53.696343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:43:53.696343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:43:53.696343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:43:53.696343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:43:53.696343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:53.696343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:43:53.696343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:53.696343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:43:53.696343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:53.696343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:43:53.696343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:53.696343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:43:53.696343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:53.696343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:43:53.696343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:43:53.696343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:53.696343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:43:53.696343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:53.697631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:43:53.697631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:43:53.697631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:53.697631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:43:53.697631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:53.697631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:43:53.792271 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:43:53.792904 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:43:53.793040 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:43:53.794477 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:43:53.795055 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:43:53.795344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:53.796416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:43:53.796416 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:53.796416 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:43:53.796416 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:43:53.796416 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:53.796416 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:43:53.796416 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:53.796416 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:43:53.797635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:53.797635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:43:53.797635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:43:53.797635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:43:53.797635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:43:53.797635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:43:53.797635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:43:53.797635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:53.797635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:43:53.797635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:53.797635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:43:53.797635 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:53.798302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:43:53.798302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:53.798302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:43:53.798302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:53.798302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:43:53.798302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:43:53.798302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:53.798302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:43:53.798302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:53.798302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:43:53.798302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:43:53.798302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:53.798302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:43:53.799269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:53.799269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:43:53.799269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:53.799269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:43:53.799269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:43:53.799269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:43:53.799269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:43:53.799269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:43:53.799269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:43:53.799269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:43:53.799269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:43:53.799269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:43:53.799269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:43:53.799269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:43:53.799269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:43:53.800181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:43:53.800181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:43:53.800181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:43:53.800181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:43:53.800181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:43:53.800181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:53.800181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:43:53.800181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:53.800181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:43:53.800181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:53.800181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:43:53.800181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:53.800181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:43:53.800181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:53.800181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:43:53.800181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:43:53.800181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:53.800181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:43:53.800181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:53.801503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:43:53.801503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:43:53.801503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:53.801503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:43:53.801503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:53.801503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:43:53.801503 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:43:53.801503 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:43:53.801503 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:43:53.801503 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:43:53.802107 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:43:53.802107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:53.802107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:43:53.802107 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:53.802107 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:43:53.802107 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:43:53.802107 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:53.802107 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:43:53.802107 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:53.802107 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:43:53.802616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:53.802616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:43:53.802616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:43:53.802616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:43:53.802616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:53.802616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:43:53.802616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:53.802616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:43:53.802616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:53.802616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:43:53.802616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:53.802616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:43:53.802616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:53.802616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:43:53.802616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:43:53.803487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:53.803487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:43:53.803487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:53.803487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:43:53.803487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:43:53.803487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:53.803487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:43:53.803487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:53.803487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:43:53.803487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:53.803487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:43:53.803487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:43:53.803487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:43:53.803487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:43:53.804319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:43:53.804319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:43:53.804319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:43:53.804319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:43:53.804319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:43:53.804319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:53.804319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:43:53.804319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:53.804319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:43:53.804319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:53.804319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:43:53.804319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:53.804319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:43:53.805053 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:53.805053 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:43:53.805053 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:43:53.805053 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:53.805053 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:43:53.805053 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:53.805053 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:43:53.805053 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:43:53.805053 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:53.805053 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:43:53.805053 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:53.805053 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:43:53.805665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:53.805665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:43:53.805665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:43:53.805665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:43:53.805665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:43:53.805665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:43:53.805665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:43:53.805665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:43:53.805665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:43:53.805665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:43:53.805665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:43:53.805665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:43:53.805665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:43:53.805665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:43:53.805665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:43:53.805665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:43:53.806525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:43:53.806525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:43:53.806525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:43:53.806525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:43:53.806525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:43:53.806525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:53.806525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:43:53.806525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:53.806525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:43:53.806525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:53.806525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:43:53.806525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:53.806525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:43:53.806525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:53.806525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:43:53.806525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:43:53.806525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:53.806525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:43:53.807544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:53.807544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:43:53.807544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:43:53.807544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:53.807544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:43:53.807544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:53.807544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:43:53.807544 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:43:53.807544 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:43:53.807544 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:43:53.808180 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:43:53.808180 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:43:53.808180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:53.808180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:43:53.808180 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:53.808180 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:43:53.808180 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:43:53.808180 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:53.808180 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:43:53.808180 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:53.808180 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:43:53.809241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:53.809241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:43:53.809241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:43:53.809241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:43:53.809241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:53.809241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:43:53.809241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:53.809241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:43:53.809241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:53.809241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:43:53.809241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:53.809241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:43:53.809241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:53.809241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:43:53.809241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:43:53.809540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:53.809540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:43:53.809540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:53.809540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:43:53.809540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:43:53.809540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:53.809540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:43:53.809540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:53.809540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:43:53.810439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:53.810439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:43:53.810439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:43:53.810439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:43:53.810439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:43:53.810439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:43:53.810439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:43:53.810439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:43:53.810439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:43:53.810439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:53.810439 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:43:53.810768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:53.810768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:43:53.810768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:53.810768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:43:53.810768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:53.810768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:43:53.810768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:53.810768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:43:53.810768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:43:53.810768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:53.810768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:43:53.810768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:53.810768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:43:53.810768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:43:53.811359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:53.811359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:43:53.811359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:53.811359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:43:53.812093 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:53.812093 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:43:53.812093 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:43:53.812093 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:43:53.812093 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:43:53.812093 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:43:53.812093 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:43:53.812093 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:43:53.812093 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:43:53.812093 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:43:53.812093 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:43:53.812093 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:43:53.812093 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:43:53.812093 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:43:53.812093 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:43:53.812093 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:43:53.812093 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:43:53.812459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:43:53.812459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:43:53.812459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:43:53.812459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:53.812459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:43:53.812459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:53.812459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:43:53.812459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:53.812459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:43:53.812459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:53.812459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:43:53.812459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:53.812459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:43:53.812459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:43:53.812459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:53.812459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:43:53.812459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:53.812459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:43:53.812459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:43:53.812876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:53.812876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:43:53.812876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:53.812876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:43:53.815559 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:43:53.815838 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:43:53.816087 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:43:53.816376 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:43:53.816769 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:43:53.817480 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:43:53.817786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:53.818188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:43:53.818188 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:53.818188 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:43:53.818188 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:43:53.818188 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:53.818188 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:43:53.818188 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:53.818188 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:43:53.818877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:53.818877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:43:53.818877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:43:53.818877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:43:53.818877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:43:53.818877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:43:53.818877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:43:53.818877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:53.818877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:43:53.818877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:53.818877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:43:53.818877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:53.819558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:43:53.819558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:53.819558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:43:53.819558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:53.819558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:43:53.819558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:43:53.819558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:53.819558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:43:53.819558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:53.819558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:43:53.819558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:43:53.819558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:53.819558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:43:53.820349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:53.820349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:43:53.820349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:53.820349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:43:53.820349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:43:53.820349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:43:53.820349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:43:53.820349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:43:53.820349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:43:53.820349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:43:53.820349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:43:53.820349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:43:53.820349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:43:53.820349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:43:53.820349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:43:53.821218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:43:53.821218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:43:53.821218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:43:53.821218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:43:53.821218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:43:53.821218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:53.821218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:43:53.821218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:53.821218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:43:53.821218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:53.821218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:43:53.821218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:53.821218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:43:53.821218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:53.821218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:43:53.821218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:43:53.821218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:53.821218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:43:53.821218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:53.822415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:43:53.822415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:43:53.822415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:53.822415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:43:53.822415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:53.822415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:43:58.647003 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:43:58.648621 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:43:58.648862 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:43:58.650198 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:43:58.652404 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG dbcounter [-] [106314] Writing DB stats neutron:SELECT=240 {{(pid=106314) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 09 09:43:58.683725 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:43:58.684026 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:58.684942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:43:58.684942 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:58.684942 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:43:58.684942 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:43:58.684942 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:58.684942 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:43:58.684942 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:58.684942 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:43:58.685791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:58.685791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:43:58.685791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:43:58.685791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:43:58.685791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:43:58.685791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:43:58.685791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:43:58.685791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:58.685791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:43:58.685791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:58.685791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:43:58.685791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:58.686416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:43:58.686416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:58.686416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:43:58.686416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:58.686416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:43:58.686416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:43:58.686416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:58.686416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:43:58.686416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:58.686416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:43:58.686416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:43:58.686416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:58.686416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:43:58.686974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:58.686974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:43:58.686974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:58.686974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:43:58.686974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:43:58.686974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:43:58.686974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:43:58.686974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:43:58.686974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:43:58.686974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:43:58.686974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:43:58.686974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:43:58.686974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:43:58.686974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:43:58.686974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:43:58.687596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:43:58.687596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:43:58.687596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:43:58.687596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:43:58.687596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:43:58.687596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:58.687596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:43:58.687596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:58.687596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:43:58.687596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:58.687596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:43:58.687596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:58.687596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:43:58.687596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:58.687596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:43:58.687596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:43:58.687596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:58.687596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:43:58.687596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:58.688387 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:43:58.688387 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:43:58.688387 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:58.688387 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:43:58.688387 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:58.688387 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:43:58.793371 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:43:58.794105 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:43:58.794322 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:43:58.795543 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:43:58.795849 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:43:58.796069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:58.796897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:43:58.796897 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:58.796897 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:43:58.796897 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:43:58.796897 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:58.796897 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:43:58.796897 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:58.796897 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:43:58.798614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:58.798614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:43:58.798614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:43:58.798614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:43:58.798614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:43:58.798614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:43:58.798614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:43:58.798614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:58.798614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:43:58.798614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:58.798614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:43:58.798614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:58.800023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:43:58.800023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:58.800023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:43:58.800023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:58.800023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:43:58.800023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:43:58.800023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:58.800023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:43:58.800023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:58.800023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:43:58.800023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:43:58.800023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:58.800023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:43:58.801842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:58.801842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:43:58.801842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:58.801842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:43:58.801842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:43:58.801842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:43:58.801842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:43:58.801842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:43:58.801842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:43:58.801842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:43:58.801842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:43:58.801842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:43:58.801842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:43:58.801842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:43:58.801842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:43:58.802532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:43:58.802532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:43:58.802532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:43:58.802532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:43:58.802532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:43:58.802532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:58.802532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:43:58.802532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:58.802532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:43:58.802532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:58.802532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:43:58.802532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:58.802532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:43:58.802532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:58.802532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:43:58.802532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:43:58.802532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:58.802532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:43:58.802532 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:58.803308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:43:58.803308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:43:58.803308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:58.803308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:43:58.803308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:58.803308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:43:58.803308 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:43:58.803308 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:43:58.803308 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:43:58.803308 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:43:58.803796 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:43:58.803796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:58.803796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:43:58.803796 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:58.803796 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:43:58.803796 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:43:58.803796 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:58.803796 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:43:58.803796 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:58.803796 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:43:58.804384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:58.804384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:43:58.804384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:43:58.804384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:43:58.804384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:58.804384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:43:58.804384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:58.804384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:43:58.804384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:58.804384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:43:58.804384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:58.804384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:43:58.804384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:58.804384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:43:58.804384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:43:58.805214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:58.805214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:43:58.805214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:58.805214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:43:58.805214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:43:58.805214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:58.805214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:43:58.805214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:58.805214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:43:58.805214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:58.805214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:43:58.805214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:43:58.805214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:43:58.805214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:43:58.806096 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:43:58.806096 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:43:58.806096 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:43:58.806096 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:43:58.806096 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:43:58.806096 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:58.806096 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:43:58.806096 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:58.806096 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:43:58.806096 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:58.806096 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:43:58.806096 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:58.806096 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:43:58.807214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:58.807214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:43:58.807214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:43:58.807214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:58.807214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:43:58.807214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:58.807214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:43:58.807214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:43:58.807214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:58.807214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:43:58.807214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:58.807214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:43:58.807727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:58.807727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:43:58.807727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:43:58.807727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:43:58.807727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:43:58.807727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:43:58.807727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:43:58.807727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:43:58.807727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:43:58.807727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:43:58.807727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:43:58.807727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:43:58.807727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:43:58.807727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:43:58.807727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:43:58.807727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:43:58.808187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:43:58.808187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:43:58.808187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:43:58.808187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:43:58.808187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:43:58.808187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:58.808187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:43:58.808187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:58.808187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:43:58.808187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:58.808187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:43:58.808187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:58.808187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:43:58.808187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:58.808187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:43:58.808187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:43:58.808187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:58.808187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:43:58.809514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:58.809514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:43:58.809514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:43:58.809514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:58.809514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:43:58.809514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:58.809514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:43:58.809514 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:43:58.809514 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:43:58.809514 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:43:58.810155 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:43:58.810155 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:43:58.810155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:58.810155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:43:58.810155 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:58.810155 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:43:58.810155 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:43:58.810155 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:58.810155 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:43:58.810155 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:58.810155 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:43:58.810155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:58.810880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:43:58.810880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:43:58.810880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:43:58.810880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:58.810880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:43:58.810880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:58.810880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:43:58.810880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:58.810880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:43:58.810880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:58.810880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:43:58.810880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:58.810880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:43:58.810880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:43:58.810880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:58.810880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:43:58.810880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:58.811321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:43:58.811321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:43:58.811321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:58.811321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:43:58.811321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:58.811321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:43:58.811321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:58.811321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:43:58.811321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:43:58.811321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:43:58.811321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:43:58.811321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:43:58.811321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:43:58.811850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:43:58.811850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:43:58.811850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:58.811850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:43:58.811850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:58.811850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:43:58.811850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:58.811850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:43:58.811850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:58.811850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:43:58.811850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:58.811850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:43:58.811850 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:43:58.812201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:58.812201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:43:58.812201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:58.812201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:43:58.812201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:43:58.812201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:58.812201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:43:58.812201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:58.812201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:43:58.812201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:58.812201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:43:58.812201 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:43:58.812585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:43:58.812585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:43:58.812585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:43:58.812585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:43:58.812585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:43:58.812585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:43:58.812585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:43:58.812585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:43:58.812585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:43:58.812585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:43:58.812585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:43:58.812585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:43:58.812585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:43:58.812585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:43:58.812585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:43:58.812585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:43:58.812585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:43:58.812585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:58.812585 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:43:58.813285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:58.813285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:43:58.813285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:58.813285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:43:58.813285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:58.813285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:43:58.813285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:58.813285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:43:58.813285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:43:58.813285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:58.813285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:43:58.813285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:58.813285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:43:58.813285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:43:58.813285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:58.813285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:43:58.813285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:58.813285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:43:58.815979 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:43:58.816257 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:43:58.816419 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:43:58.816618 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:43:58.816987 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:43:58.817309 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:43:58.817628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:58.818573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:43:58.818573 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:58.818573 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:43:58.818573 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:43:58.818573 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:58.818573 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:43:58.818573 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:58.818573 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:43:58.819534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:58.819534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:43:58.819534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:43:58.819534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:43:58.819534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:43:58.819534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:43:58.819534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:43:58.819534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:58.819534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:43:58.819534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:58.819534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:43:58.819534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:58.819817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:43:58.819817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:58.819817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:43:58.819817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:58.819817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:43:58.819817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:43:58.819817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:58.819817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:43:58.819817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:58.819817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:43:58.819817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:43:58.819817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:58.819817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:43:58.820100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:58.820100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:43:58.820759 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:58.820759 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:43:58.820759 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:43:58.820759 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:43:58.820759 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:43:58.820759 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:43:58.820759 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:43:58.820759 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:43:58.820759 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:43:58.820759 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:43:58.820759 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:43:58.820759 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:43:58.820759 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:43:58.820759 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:43:58.820759 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:43:58.820759 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:43:58.821106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:43:58.821106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:43:58.821106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:58.821106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:43:58.821106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:43:58.821106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:43:58.821106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:43:58.821106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:43:58.821106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:43:58.821106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:43:58.821106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:43:58.821106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:43:58.821106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:43:58.821106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:43:58.821106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:43:58.821106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:43:58.821106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:43:58.821106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:43:58.821106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:43:58.821498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:43:58.821498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:43:58.821498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:44:03.648710 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:44:03.649655 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:44:03.649655 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:44:03.650936 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:44:03.688439 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:44:03.688854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:03.689781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:44:03.689781 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:03.689781 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:44:03.689781 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:44:03.689781 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:03.689781 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:44:03.689781 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:03.689781 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:44:03.690525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:03.690525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:44:03.690525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:44:03.690525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:44:03.690525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:44:03.690525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:44:03.690525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:44:03.690525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:03.690525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:44:03.690525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:03.690525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:44:03.690525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:03.691259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:44:03.691259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:03.691259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:44:03.691259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:03.691259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:44:03.691259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:44:03.691259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:03.691259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:44:03.691259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:03.691259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:44:03.691259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:44:03.691259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:03.691259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:44:03.692097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:03.692097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:44:03.692097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:03.692097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:44:03.692097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:44:03.692097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:44:03.692097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:44:03.692097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:44:03.692097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:44:03.692097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:44:03.692097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:44:03.692097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:44:03.692097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:44:03.692097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:44:03.692097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:44:03.692899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:44:03.692899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:44:03.692899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:44:03.692899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:44:03.692899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:44:03.692899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:03.692899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:44:03.692899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:03.692899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:44:03.692899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:03.692899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:44:03.692899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:03.692899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:44:03.692899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:03.692899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:44:03.692899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:44:03.692899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:03.692899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:44:03.692899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:03.693904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:44:03.693904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:44:03.693904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:03.693904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:44:03.693904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:03.693904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:44:03.794074 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:44:03.795042 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:44:03.795321 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:44:03.796498 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:44:03.797059 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:44:03.797279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:03.798211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:44:03.798211 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:03.798211 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:44:03.798211 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:44:03.798211 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:03.798211 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:44:03.798211 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:03.798211 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:44:03.800248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:03.800248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:44:03.800248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:44:03.800248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:44:03.800248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:44:03.800248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:44:03.800248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:44:03.800248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:03.800248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:44:03.800248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:03.800248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:44:03.800248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:03.801284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:44:03.801284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:03.801284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:44:03.801284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:03.801284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:44:03.801284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:44:03.801284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:03.801284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:44:03.801284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:03.801284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:44:03.801284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:44:03.801284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:03.801284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:44:03.802636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:03.802636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:44:03.803401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:03.803401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:44:03.803401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:44:03.803401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:44:03.803401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:44:03.803401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:44:03.803401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:44:03.803401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:44:03.803401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:44:03.803401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:44:03.803401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:44:03.803401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:44:03.803401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:44:03.803401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:44:03.803401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:44:03.803401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:44:03.804347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:44:03.804347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:44:03.804347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:03.804347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:44:03.804347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:03.804347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:44:03.804347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:03.804347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:44:03.804347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:03.804347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:44:03.804347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:03.804347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:44:03.804347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:44:03.804347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:03.804347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:44:03.804347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:03.804347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:44:03.804347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:44:03.804347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:03.805899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:44:03.805899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:03.805899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:44:03.805899 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:44:03.805899 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:44:03.805899 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:44:03.805899 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:44:03.805899 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:44:03.806526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:03.806526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:44:03.806526 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:03.806526 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:44:03.806526 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:44:03.806526 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:03.806526 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:44:03.806526 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:03.806526 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:44:03.807642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:03.807642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:44:03.807642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:44:03.807642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:44:03.807642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:03.807642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:44:03.807642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:03.807642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:44:03.807642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:03.807642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:44:03.807642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:03.807642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:44:03.807642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:03.807642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:44:03.807642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:44:03.808919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:03.808919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:44:03.808919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:03.808919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:44:03.808919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:44:03.808919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:03.808919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:44:03.808919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:03.808919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:44:03.808919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:03.808919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:44:03.808919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:44:03.808919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:44:03.808919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:44:03.809946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:44:03.809946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:44:03.809946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:44:03.809946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:44:03.809946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:44:03.809946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:03.809946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:44:03.809946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:03.809946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:44:03.809946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:03.809946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:44:03.809946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:03.809946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:44:03.810440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:03.810440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:44:03.810440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:44:03.810440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:03.810440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:44:03.810440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:03.810440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:44:03.810440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:44:03.810440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:03.810440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:44:03.810440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:03.810440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:44:03.810952 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:44:03.810952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:03.810952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:44:03.810952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:44:03.810952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:44:03.810952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:44:03.810952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:44:03.810952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:44:03.810952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:44:03.810952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:44:03.810952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:44:03.810952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:44:03.810952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:44:03.810952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:44:03.811486 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:44:03.811486 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:44:03.811486 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:44:03.811486 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:44:03.811486 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:44:03.811486 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:44:03.811486 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:44:03.811486 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:44:03.811486 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:03.811486 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:44:03.811486 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:03.811486 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:44:03.811486 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:03.811486 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:44:03.811486 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:03.811486 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:44:03.811486 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:03.811486 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:44:03.811486 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:44:03.812216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:03.812216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:44:03.812216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:03.812216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:44:03.812216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:44:03.812216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:03.812216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:44:03.812216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:03.812216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:44:03.812216 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:44:03.812216 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:44:03.812216 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:44:03.812710 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:44:03.812710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:03.812710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:44:03.812710 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:03.812710 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:44:03.812710 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:44:03.812710 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:03.812710 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:44:03.812710 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:03.812710 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:44:03.812710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:03.812710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:44:03.812710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:44:03.812710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:44:03.813279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:03.813279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:44:03.813279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:03.813279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:44:03.813279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:03.813279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:44:03.813279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:03.813279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:44:03.813279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:03.813279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:44:03.813279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:44:03.813279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:03.813279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:44:03.813279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:03.813279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:44:03.813279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:44:03.813279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:03.813279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:44:03.813841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:03.813841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:44:03.813841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:03.813841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:44:03.813841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:44:03.813841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:44:03.813841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:44:03.813841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:44:03.813841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:44:03.813841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:44:03.813841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:44:03.813841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:03.814191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:44:03.814191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:03.814191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:44:03.814191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:03.814191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:44:03.814191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:03.814191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:44:03.814191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:03.814191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:44:03.814191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:44:03.814191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:03.814191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:44:03.814191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:03.814191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:44:03.814545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:44:03.814545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:03.814545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:44:03.814545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:03.814545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:44:03.814545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:03.814545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:44:03.814545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:44:03.814545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:44:03.814545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:44:03.814545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:44:03.814545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:44:03.814545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:44:03.814545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:44:03.814545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:44:03.815003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:44:03.815003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:44:03.815003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:44:03.815003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:44:03.815003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:44:03.815003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:44:03.815003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:44:03.815003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:44:03.815003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:44:03.815003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:44:03.815003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:03.815003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:44:03.815003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:03.815003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:44:03.815003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:03.815003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:44:03.815003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:03.815003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:44:03.815003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:03.815655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:44:03.815655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:44:03.815655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:03.815655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:44:03.815655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:03.815655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:44:03.815655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:44:03.815655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:03.815655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:44:03.815655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:03.815655 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:44:03.816427 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:44:03.816770 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:44:03.816967 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:44:03.817192 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:44:03.817606 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:44:03.817975 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:44:03.818284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:03.818989 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:44:03.818989 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:03.818989 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:44:03.818989 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:44:03.818989 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:03.818989 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:44:03.818989 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:03.818989 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:44:03.819915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:03.819915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:44:03.819915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:44:03.819915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:44:03.819915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:44:03.819915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:44:03.819915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:44:03.819915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:03.819915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:44:03.819915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:03.819915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:44:03.819915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:03.820905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:44:03.820905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:03.820905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:44:03.820905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:03.820905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:44:03.820905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:44:03.820905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:03.820905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:44:03.820905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:03.820905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:44:03.820905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:44:03.820905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:03.820905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:44:03.821760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:03.821760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:44:03.821760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:03.821760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:44:03.821760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:44:03.821760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:44:03.821760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:44:03.821760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:44:03.821760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:44:03.821760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:44:03.821760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:44:03.821760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:44:03.821760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:44:03.821760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:44:03.821760 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:44:03.822340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:44:03.822340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:44:03.822340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:44:03.822340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:44:03.822340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:44:03.822340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:03.822340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:44:03.822340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:03.822340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:44:03.822340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:03.822340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:44:03.822340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:03.822340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:44:03.822340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:03.822340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:44:03.822340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:44:03.822340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:03.822340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:44:03.822340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:03.823081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:44:03.823081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:44:03.823081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:03.823081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:44:03.823081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:03.823081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:44:08.649377 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:44:08.650310 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:44:08.650310 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:44:08.651766 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:44:08.701141 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:44:08.701560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:08.702503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:44:08.702503 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:08.702503 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:44:08.702503 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:44:08.702503 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:08.702503 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:44:08.702503 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:08.702503 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:44:08.703683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.052 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:08.703683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:44:08.703683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:44:08.703683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:44:08.703683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:44:08.703683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:44:08.703683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:44:08.703683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:08.703683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:44:08.703683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:08.703683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:44:08.703683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:08.704398 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:44:08.704398 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:08.704398 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:44:08.704398 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:08.704398 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:44:08.704398 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:44:08.704398 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:08.704398 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:44:08.704398 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:08.704398 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:44:08.704398 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:44:08.704398 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:08.704398 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:44:08.705087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:08.705087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:44:08.705209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:08.705209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:44:08.705209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:44:08.705209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:44:08.705209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:44:08.705209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:44:08.705209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:44:08.705209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:44:08.705209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:44:08.705209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:44:08.705209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:44:08.705209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:44:08.705209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:44:08.705209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:44:08.705209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:44:08.705209 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:44:08.706191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:44:08.706191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:44:08.706191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:08.706191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:44:08.706191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:08.706191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:44:08.706191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:08.706191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:44:08.706191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:08.706191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:44:08.706191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:08.706191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:44:08.706191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:44:08.706191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:08.706191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:44:08.706191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:08.706191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:44:08.706191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:44:08.706191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:08.707327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:44:08.707327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:08.707327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:44:08.794890 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:44:08.795452 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:44:08.795613 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:44:08.796688 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:44:08.797028 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:44:08.797331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:08.798062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:44:08.798062 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:08.798062 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:44:08.798062 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:44:08.798062 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:08.798062 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:44:08.798062 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:08.798062 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:44:08.799489 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:08.799489 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:44:08.799489 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:44:08.799489 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:44:08.799489 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:44:08.799489 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:44:08.799489 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:44:08.799489 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:08.799489 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:44:08.799489 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:08.799489 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:44:08.799489 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:08.800434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:44:08.800434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:08.800434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:44:08.800434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:08.800434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:44:08.800434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:44:08.800434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:08.800434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:44:08.800434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:08.800434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:44:08.800434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:44:08.800434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:08.800434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:44:08.801191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:08.801191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:44:08.801337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:08.801337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:44:08.801337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:44:08.801337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:44:08.801337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:44:08.801337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:44:08.801337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:44:08.801337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:44:08.801337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:44:08.801337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:44:08.801337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:44:08.801337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:44:08.801337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:44:08.801337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:44:08.801337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:44:08.801337 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:44:08.802153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:44:08.802153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:44:08.802153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:08.802153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:44:08.802153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:08.802153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:44:08.802153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:08.802153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:44:08.802153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:08.802153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:44:08.802153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:08.802153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:44:08.802153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:44:08.802153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:08.802153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:44:08.802153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:08.802153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:44:08.802153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:44:08.802153 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:08.803522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:44:08.803522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:08.803522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:44:08.804321 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:44:08.804692 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:44:08.804814 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:44:08.804976 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:44:08.805658 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:44:08.806131 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:08.807116 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:44:08.807116 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:08.807116 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:44:08.807116 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:44:08.807116 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:08.807116 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:44:08.807116 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:08.807116 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:44:08.807882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:08.807882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:44:08.807882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:44:08.807882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:44:08.807882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:08.807882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:44:08.807882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:08.807882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:44:08.807882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:08.807882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:44:08.807882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:08.807882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:44:08.807882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:08.807882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:44:08.807882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:44:08.808356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:08.808356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:44:08.808356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:08.808356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:44:08.808356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:44:08.808356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:08.808356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:44:08.808356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:08.808356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:44:08.808757 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:08.808757 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:44:08.808757 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:44:08.808757 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:44:08.808757 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:44:08.808757 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:44:08.808757 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:44:08.808757 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:44:08.808757 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:44:08.808757 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:44:08.808757 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:08.808757 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:44:08.809148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:08.809148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:44:08.809148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:08.809148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:44:08.809148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:08.809148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:44:08.809148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:08.809148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:44:08.809148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:44:08.809148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:08.809148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:44:08.809148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:08.809148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:44:08.809148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:44:08.809509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:08.809509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:44:08.809509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:08.809509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:44:08.809509 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:44:08.810069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:08.810069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:44:08.810069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:44:08.810069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:44:08.810069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:44:08.810069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:44:08.810069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:44:08.810069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:44:08.810069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:44:08.810069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:44:08.810069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:44:08.810069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:44:08.810069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:44:08.810069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:44:08.810069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:44:08.810069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:44:08.810561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:44:08.810561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:44:08.810561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:44:08.810561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:44:08.810561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:44:08.810561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:08.810561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:44:08.810561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:08.810561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:44:08.810561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:08.810561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:44:08.810561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:08.810561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:44:08.810561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:08.810561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:44:08.810561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:44:08.810561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:08.810561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:44:08.811206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:08.811206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:44:08.811206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:44:08.811206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:08.811206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:44:08.811206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:08.811206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:44:08.811206 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:44:08.811206 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:44:08.811206 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:44:08.811567 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:44:08.811567 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:08.811684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:44:08.811684 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:08.811684 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:44:08.811684 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:44:08.811684 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:08.811684 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:44:08.811684 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:08.811684 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:44:08.812415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:08.812415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:44:08.812415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:44:08.812415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:44:08.812415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:08.812415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:44:08.812415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:08.812415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:44:08.812415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:08.812415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:44:08.812415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:08.812415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:44:08.812415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:08.812415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:44:08.812415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:44:08.812855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:08.812855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:44:08.812855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:08.812855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:44:08.812855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:44:08.812855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:08.812855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:44:08.812855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:08.812855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:44:08.813285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:08.813285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:44:08.813285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:44:08.813285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:44:08.813285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:44:08.813285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:44:08.813285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:44:08.813285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:44:08.813285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:44:08.813285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:08.813285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:44:08.813626 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:08.813626 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:44:08.813626 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:08.813626 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:44:08.813626 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:08.813626 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:44:08.813626 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:08.813626 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:44:08.813626 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:44:08.813626 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:08.813626 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:44:08.813626 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:08.813626 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:44:08.813626 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:44:08.814011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:08.814011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:44:08.814011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:08.814011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:44:08.814343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:08.814343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:44:08.814343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:44:08.814343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:44:08.814343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:44:08.814343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:44:08.814343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:44:08.814343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:44:08.814343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:44:08.814343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:44:08.814343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:44:08.814343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:44:08.814343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:44:08.814343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:44:08.814343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:44:08.814343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:44:08.814343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:44:08.815039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:44:08.815039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:44:08.815039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:44:08.815039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:08.815039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:44:08.815039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:08.815039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:44:08.815039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:08.815039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:44:08.815039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:08.815039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:44:08.815039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:08.815039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:44:08.815039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:44:08.815039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:08.815039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:44:08.815039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:08.815039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:44:08.815039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:44:08.815610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:08.815610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:44:08.815610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:08.815610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:44:08.816815 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:44:08.817039 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:44:08.817167 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:44:08.817304 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:44:08.817640 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:44:08.817902 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:44:08.818053 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:08.818520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:44:08.818520 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:08.818520 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:44:08.818520 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:44:08.818520 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:08.818520 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:44:08.818520 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:08.818520 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:44:08.819387 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:08.819387 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:44:08.819387 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:44:08.819387 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:44:08.819387 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:44:08.819387 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:44:08.819387 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:44:08.819387 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:08.819387 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:44:08.819387 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:08.819387 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:44:08.819387 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:08.819815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:44:08.819815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:08.819815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:44:08.819815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:08.819815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:44:08.819815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:44:08.819815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:08.819815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:44:08.819815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:08.819815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:44:08.819815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:44:08.819815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:08.819815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:44:08.820155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:08.820155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:44:08.820358 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:08.820358 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:44:08.820358 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:44:08.820358 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:44:08.820358 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:44:08.820358 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:44:08.820358 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:44:08.820358 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:44:08.820358 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:44:08.820358 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:44:08.820358 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:44:08.820358 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:44:08.820358 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:44:08.820358 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:44:08.820358 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:44:08.820358 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:44:08.820872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:44:08.820872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:44:08.820872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:08.820872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:44:08.820872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:08.820872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:44:08.820872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:08.820872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:44:08.820872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:08.820872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:44:08.820872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:08.820872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:44:08.820872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:44:08.820872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:08.820872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:44:08.820872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:08.820872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:44:08.820872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:44:08.820872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:08.821395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:44:08.821395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:08.821395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:44:13.650287 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:44:13.651045 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:44:13.651301 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:44:13.652656 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:44:13.682620 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:44:13.682970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:13.683711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:44:13.683711 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:13.683711 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:44:13.683711 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:44:13.683711 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:13.683711 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:44:13.683711 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:13.683711 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:44:13.684613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.032 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:13.684613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:44:13.684613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:44:13.684613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:44:13.684613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:44:13.684613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:44:13.684613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:44:13.684613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:13.684613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:44:13.684613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:13.684613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:44:13.684613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:13.685081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:44:13.685081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:13.685081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:44:13.685081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:13.685081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:44:13.685081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:44:13.685081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:13.685081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:44:13.685081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:13.685081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:44:13.685081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:44:13.685081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:13.685081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:44:13.685554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:13.685554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:44:13.685554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:13.685554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:44:13.685554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:44:13.685554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:44:13.685554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:44:13.685554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:44:13.685554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:44:13.685554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:44:13.685554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:44:13.685554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:44:13.685554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:44:13.685554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:44:13.685554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:44:13.686253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:44:13.686253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:44:13.686253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:44:13.686253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:44:13.686253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:44:13.686253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:13.686253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:44:13.686253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:13.686253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:44:13.686253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:13.686253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:44:13.686253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:13.686253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:44:13.686253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:13.686253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:44:13.686253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:44:13.686253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:13.686253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:44:13.686253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:13.687101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:44:13.687101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:44:13.687101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:13.687101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:44:13.687101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:13.687101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:44:13.795588 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:44:13.796050 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:44:13.796247 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:44:13.797582 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:44:13.797983 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:44:13.798331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:13.799298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:44:13.799298 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:13.799298 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:44:13.799298 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:44:13.799298 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:13.799298 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:44:13.799298 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:13.799298 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:44:13.800866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:13.800866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:44:13.800866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:44:13.800866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:44:13.800866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:44:13.800866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:44:13.800866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:44:13.800866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:13.800866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:44:13.800866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:13.800866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:44:13.800866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:13.801518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:44:13.801518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:13.801518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:44:13.801518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:13.801518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:44:13.801518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:44:13.801518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:13.801518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:44:13.801518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:13.801518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:44:13.801518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:44:13.801518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:13.801518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:44:13.802136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:13.802136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:44:13.802505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:13.802505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:44:13.802505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:44:13.802505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:44:13.802505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:44:13.802505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:44:13.802505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:44:13.802505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:44:13.802505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:44:13.802505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:44:13.802505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:44:13.802505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:44:13.802505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:44:13.802505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:44:13.802505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:44:13.802505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:44:13.803315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:44:13.803315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:44:13.803315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:13.803315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:44:13.803315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:13.803315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:44:13.803315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:13.803315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:44:13.803315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:13.803315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:44:13.803315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:13.803315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:44:13.803315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:44:13.803315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:13.803315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:44:13.803315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:13.803315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:44:13.803315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:44:13.803315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:13.804143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:44:13.804143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:13.804143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:44:13.804971 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:44:13.805255 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:44:13.805342 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:44:13.805471 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:44:13.805864 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:44:13.806077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:13.806577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:44:13.806577 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:13.806577 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:44:13.806577 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:44:13.806577 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:13.806577 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:44:13.806577 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:13.806577 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:44:13.807298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:13.807298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:44:13.807298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:44:13.807298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:44:13.807298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:13.807298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:44:13.807298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:13.807298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:44:13.807298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:13.807298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:44:13.807298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:13.807298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:44:13.807298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:13.807298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:44:13.807298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:44:13.808160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:13.808160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:44:13.808160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:13.808160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:44:13.808160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:44:13.808160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:13.808160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:44:13.808160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:13.808160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:44:13.808564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:13.808564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:44:13.808564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:44:13.808564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:44:13.808564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:44:13.808564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:44:13.808564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:44:13.808564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:44:13.808564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:44:13.808564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:44:13.808564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:13.808564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:44:13.808960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:13.808960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:44:13.808960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:13.808960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:44:13.808960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:13.808960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:44:13.808960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:13.808960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:44:13.808960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:44:13.808960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:13.808960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:44:13.808960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:13.808960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:44:13.808960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:44:13.809344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:13.809344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:44:13.809344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:13.809344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:44:13.809344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:13.809344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:44:13.809344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:44:13.809344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:44:13.809344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:44:13.809344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:44:13.809344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:44:13.809344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:44:13.809344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:44:13.809344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:44:13.809813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:44:13.809813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:44:13.809813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:44:13.809813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:44:13.809813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:44:13.809813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:44:13.809813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:44:13.809813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:44:13.809813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:44:13.809813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:44:13.809813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:44:13.809813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:13.809813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:44:13.809813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:13.809813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:44:13.809813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:13.809813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:44:13.809813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:13.809813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:44:13.810386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:13.810386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:44:13.810386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:44:13.810386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:13.810386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:44:13.810386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:13.810386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:44:13.810386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:44:13.810386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:13.810386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:44:13.810386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:13.810386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:44:13.810386 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:44:13.810800 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:44:13.810847 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:44:13.810977 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:44:13.811352 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:44:13.811648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:13.812224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:44:13.812224 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:13.812224 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:44:13.812224 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:44:13.812224 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:13.812224 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:44:13.812224 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:13.812224 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:44:13.813043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:13.813043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:44:13.813043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:44:13.813043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:44:13.813043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:13.813043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:44:13.813043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:13.813043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:44:13.813043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:13.813043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:44:13.813043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:13.813043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:44:13.813043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:13.813043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:44:13.813043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:44:13.813428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:13.813428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:44:13.813428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:13.813428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:44:13.813428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:44:13.813428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:13.813428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:44:13.813428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:13.813428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:44:13.813913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:13.813913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:44:13.813913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:44:13.813913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:44:13.813913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:44:13.813913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:44:13.813913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:44:13.813913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:44:13.813913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:44:13.813913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:13.813913 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:44:13.814255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:13.814255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:44:13.814255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:13.814255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:44:13.814255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:13.814255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:44:13.814255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:13.814255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:44:13.814255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:44:13.814255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:13.814255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:44:13.814255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:13.814255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:44:13.814255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:44:13.814691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:13.814691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:44:13.814691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:13.814691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:44:13.815015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:13.815015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:44:13.815015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:44:13.815015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:44:13.815015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:44:13.815015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:44:13.815015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:44:13.815015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:44:13.815015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:44:13.815015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:44:13.815015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:44:13.815015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:44:13.815015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:44:13.815015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:44:13.815015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:44:13.815015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:44:13.815015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:44:13.815650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:44:13.815650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:44:13.815650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:44:13.815650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:13.815650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:44:13.815650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:13.815650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:44:13.815650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:13.815650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:44:13.815650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:13.815650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:44:13.815650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:13.815650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:44:13.815650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:44:13.815650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:13.815650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:44:13.815650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:13.815650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:44:13.815650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:44:13.816261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:13.816261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:44:13.816261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:13.816261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:44:13.817275 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:44:13.817522 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:44:13.817658 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:44:13.817823 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:44:13.818341 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:44:13.818846 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:44:13.819104 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:13.819872 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:44:13.819872 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:13.819872 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:44:13.819872 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:44:13.819872 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:13.819872 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:44:13.819872 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:13.819872 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:44:13.820738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:13.820738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:44:13.820738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:44:13.820738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:44:13.820738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:44:13.820738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:44:13.820738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:44:13.820738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:13.820738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:44:13.820738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:13.820738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:44:13.820738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:13.821292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:44:13.821292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:13.821292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:44:13.821292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:13.821292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:44:13.821292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:44:13.821292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:13.821292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:44:13.821292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:13.821292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:44:13.821292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:44:13.821292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:13.821292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:44:13.821725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:13.821725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:44:13.821725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:13.821725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:44:13.821725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:44:13.821725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:44:13.821725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:44:13.821725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:44:13.821725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:44:13.821725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:44:13.821725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:44:13.821725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:44:13.821725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:44:13.821725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:44:13.821725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:44:13.822197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:44:13.822197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:44:13.822197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:44:13.822197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:44:13.822197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:44:13.822197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:13.822197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:44:13.822197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:13.822197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:44:13.822197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:13.822197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:44:13.822197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:13.822197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:44:13.822197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:13.822197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:44:13.822197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:44:13.822197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:13.822197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:44:13.822197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:13.822787 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:44:13.822787 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:44:13.822787 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:13.822787 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:44:13.822787 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:13.822787 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:44:18.651352 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:44:18.652455 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:44:18.652455 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:44:18.653513 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:44:18.695410 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:44:18.695410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:18.696694 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:44:18.696694 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:18.696694 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:44:18.696694 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:44:18.696694 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:18.696694 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:44:18.696694 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:18.696694 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:44:18.697214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.044 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:18.697214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:44:18.697214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:44:18.697214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:44:18.697214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:44:18.697214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:44:18.697214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:44:18.697214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:18.697214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:44:18.697214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:18.697214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:44:18.697214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:18.697690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:44:18.697690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:18.697690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:44:18.697690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:18.697690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:44:18.697690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:44:18.697690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:18.697690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:44:18.697690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:18.697690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:44:18.697690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:44:18.697690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:18.697690 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:44:18.698266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:18.698266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:44:18.698266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:18.698266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:44:18.698266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:44:18.698266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:44:18.698266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:44:18.698266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:44:18.698266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:44:18.698266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:44:18.698266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:44:18.698266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:44:18.698266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:44:18.698266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:44:18.698266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:44:18.698806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:44:18.698806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:44:18.698806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:44:18.698806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:44:18.698806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:44:18.698806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:18.698806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:44:18.698806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:18.698806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:44:18.698806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:18.698806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:44:18.698806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:18.698806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:44:18.698806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:18.698806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:44:18.698806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:44:18.698806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:18.698806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:44:18.698806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:18.699456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:44:18.699456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:44:18.699456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:18.699456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:44:18.699456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:18.699456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:44:18.796222 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:44:18.796544 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:44:18.796696 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:44:18.797895 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:44:18.798191 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:44:18.798384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:18.799361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:44:18.799361 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:18.799361 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:44:18.799361 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:44:18.799361 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:18.799361 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:44:18.799361 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:18.799361 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:44:18.801309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:18.801309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:44:18.801309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:44:18.801309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:44:18.801309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:44:18.801309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:44:18.801309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:44:18.801309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:18.801309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:44:18.801309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:18.801309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:44:18.801309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:18.802062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:44:18.802062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:18.802062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:44:18.802062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:18.802062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:44:18.802062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:44:18.802062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:18.802062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:44:18.802062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:18.802062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:44:18.802062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:44:18.802062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:18.802062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:44:18.802680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:18.802680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:44:18.803391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:18.803391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:44:18.803391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:44:18.803391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:44:18.803391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:44:18.803391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:44:18.803391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:44:18.803391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:44:18.803391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:44:18.803391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:44:18.803391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:44:18.803391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:44:18.803391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:44:18.803391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:44:18.803391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:44:18.803391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:44:18.803921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:44:18.803921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:44:18.803921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:18.803921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:44:18.803921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:18.803921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:44:18.803921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:18.803921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:44:18.803921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:18.803921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:44:18.803921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:18.803921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:44:18.803921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:44:18.803921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:18.803921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:44:18.803921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:18.803921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:44:18.803921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:44:18.803921 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:18.804434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:44:18.804434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:18.804434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:44:18.805585 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:44:18.806056 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:44:18.806369 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:44:18.806549 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:44:18.806970 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:44:18.807177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:18.808011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:44:18.808011 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:18.808011 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:44:18.808011 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:44:18.808011 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:18.808011 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:44:18.808011 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:18.808011 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:44:18.809066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:18.809066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:44:18.809066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:44:18.809066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:44:18.809066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:18.809066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:44:18.809066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:18.809066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:44:18.809066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:18.809066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:44:18.809066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:18.809066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:44:18.809066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:18.809066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:44:18.809066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:44:18.809782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:18.809782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:44:18.809782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:18.809782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:44:18.809782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:44:18.809782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:18.809782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:44:18.809782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:18.809782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:44:18.810258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:18.810258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:44:18.810258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:44:18.810258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:44:18.810258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:44:18.810258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:44:18.810258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:44:18.810258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:44:18.810258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:44:18.810258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:44:18.810258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:18.810258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:44:18.810789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:18.810789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:44:18.810789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:18.810789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:44:18.810789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:18.810789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:44:18.810789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:18.810789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:44:18.810789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:44:18.810789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:18.810789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:44:18.810789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:18.810789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:44:18.810789 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:44:18.811266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:18.811266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:44:18.811266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:18.811266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:44:18.811774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:18.811774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:44:18.811774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:44:18.811774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:44:18.811774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:44:18.811774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:44:18.811774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:44:18.811774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:44:18.811774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:44:18.811774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:44:18.811774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:44:18.811774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:44:18.811774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:44:18.811774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:44:18.811774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:44:18.811774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:44:18.812502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:44:18.812502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:44:18.812502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:44:18.812502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:44:18.812502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:44:18.812502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:18.812502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:44:18.812502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:18.812502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:44:18.812502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:18.812502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:44:18.812502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:18.812502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:44:18.812502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:18.812502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:44:18.812502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:44:18.812502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:18.812502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:44:18.813248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:18.813248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:44:18.813248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:44:18.813248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:18.813248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:44:18.813248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:18.813248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:44:18.813248 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:44:18.813248 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:44:18.813248 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:44:18.813692 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:44:18.813692 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:44:18.813692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:18.814324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:44:18.814324 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:18.814324 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:44:18.814324 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:44:18.814324 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:18.814324 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:44:18.814324 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:18.814324 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:44:18.815496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:18.815496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:44:18.815496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:44:18.815496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:44:18.815496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:18.815496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:44:18.815496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:18.815496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:44:18.815496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:18.815496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:44:18.815496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:18.815496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:44:18.815496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:18.815496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:44:18.815496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:44:18.816034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:18.816034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:44:18.816034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:18.816034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:44:18.816034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:44:18.816034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:18.816034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:44:18.816034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:18.816034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:44:18.816504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:18.816504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:44:18.816504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:44:18.816504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:44:18.816504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:44:18.816504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:44:18.816504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:44:18.816504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:44:18.816504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:44:18.816504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:18.816504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:44:18.816982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:18.816982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:44:18.816982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:18.816982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:44:18.816982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:18.816982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:44:18.816982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:18.816982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:44:18.816982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:44:18.816982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:18.816982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:44:18.816982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:18.816982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:44:18.816982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:44:18.817474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:18.817474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:44:18.817474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:18.817474 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:44:18.817995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:18.817995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:44:18.817995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:44:18.817995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:44:18.817995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:44:18.817995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:44:18.817995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:44:18.817995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:44:18.817995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:44:18.817995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:44:18.817995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:44:18.817995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:44:18.817995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:44:18.817995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:44:18.817995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:44:18.817995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:44:18.817995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:44:18.818679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:44:18.818679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:44:18.818679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:44:18.818679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:18.818679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:44:18.818679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:18.818679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:44:18.818679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:18.818679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:44:18.818679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:18.818679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:44:18.818679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:18.818679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:44:18.818679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:44:18.818679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:18.818679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:44:18.818679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:18.818679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:44:18.818679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:44:18.819412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:18.819412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:44:18.819412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:18.819412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:44:18.819412 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:44:18.819412 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:44:18.819412 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:44:18.819412 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:44:18.819772 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:44:18.820372 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:44:18.820507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:18.821255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:44:18.821255 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:18.821255 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:44:18.821255 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:44:18.821255 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:18.821255 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:44:18.821255 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:18.821255 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:44:18.822441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:18.822441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:44:18.822441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:44:18.822441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:44:18.822441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:44:18.822441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:44:18.822441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:44:18.822441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:18.822441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:44:18.822441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:18.822441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:44:18.822441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:18.822842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:44:18.822842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:18.822842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:44:18.822842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:18.822842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:44:18.822842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:44:18.822842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:18.822842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:44:18.822842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:18.822842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:44:18.822842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:44:18.822842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:18.822842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:44:18.823166 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:18.823166 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:44:18.823491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:18.823491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:44:18.823491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:44:18.823491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:44:18.823491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:44:18.823491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:44:18.823491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:44:18.823491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:44:18.823491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:44:18.823491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:44:18.823491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:44:18.823491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:44:18.823491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:44:18.823491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:44:18.823491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:44:18.823491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:44:18.823940 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:44:18.823940 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:44:18.823940 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:18.823940 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:44:18.823940 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:18.823940 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:44:18.823940 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:18.823940 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:44:18.823940 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:18.823940 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:44:18.823940 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:18.823940 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:44:18.823940 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:44:18.823940 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:18.823940 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:44:18.823940 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:18.823940 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:44:18.823940 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:44:18.823940 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:18.824442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:44:18.824442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:18.824442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:44:23.653101 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:44:23.653101 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:44:23.653101 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:44:23.653941 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:44:23.688084 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:44:23.688559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:23.690138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:44:23.690138 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:23.690138 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:44:23.690138 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:44:23.690138 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:23.690138 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:44:23.690138 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:23.690138 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:44:23.690997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:23.690997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:44:23.690997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:44:23.690997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:44:23.690997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:44:23.690997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:44:23.690997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:44:23.690997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:23.690997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:44:23.690997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:23.690997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:44:23.690997 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:23.691409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:44:23.691409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:23.691409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:44:23.691409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:23.691409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:44:23.691409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:44:23.691409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:23.691409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:44:23.691409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:23.691409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:44:23.691409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:44:23.691409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:23.691409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:44:23.691802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:23.691802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:44:23.691885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:23.691885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:44:23.691885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:44:23.691885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:44:23.691885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:44:23.691885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:44:23.691885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:44:23.691885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:44:23.691885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:44:23.691885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:44:23.691885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:44:23.691885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:44:23.691885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:44:23.691885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:44:23.691885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:44:23.691885 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:44:23.692388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:44:23.692388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:44:23.692388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:23.692388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:44:23.692388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:23.692388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:44:23.692388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:23.692388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:44:23.692388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:23.692388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:44:23.692388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:23.692388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:44:23.692388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:44:23.692388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:23.692388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:44:23.692388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:23.692388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:44:23.692388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:44:23.692388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:23.692962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:44:23.692962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:23.692962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:44:23.796903 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:44:23.797582 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:44:23.797879 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:44:23.799214 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:44:23.799618 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:44:23.800108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:23.801227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:44:23.801227 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:23.801227 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:44:23.801227 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:44:23.801227 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:23.801227 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:44:23.801227 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:23.801227 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:44:23.803248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:23.803248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:44:23.803248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:44:23.803248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:44:23.803248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:44:23.803248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:44:23.803248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:44:23.803248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:23.803248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:44:23.803248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:23.803248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:44:23.803248 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:23.803832 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:44:23.803832 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:23.803832 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:44:23.803832 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:23.803832 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:44:23.803832 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:44:23.803832 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:23.803832 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:44:23.803832 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:23.803832 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:44:23.803832 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:44:23.803832 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:23.803832 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:44:23.804466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:23.804466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:44:23.805080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:23.805080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:44:23.805080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:44:23.805080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:44:23.805080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:44:23.805080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:44:23.805080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:44:23.805080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:44:23.805080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:44:23.805080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:44:23.805080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:44:23.805080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:44:23.805080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:44:23.805080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:44:23.805080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:44:23.805080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:44:23.805774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:44:23.805774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:44:23.805774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:23.805774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:44:23.805774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:23.805774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:44:23.805774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:23.805774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:44:23.805774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:23.805774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:44:23.805774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:23.805774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:44:23.805774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:44:23.805774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:23.805774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:44:23.805774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:23.805774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:44:23.805774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:44:23.805774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:23.806550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:44:23.806550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:23.806550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:44:23.806550 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:44:23.806550 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:44:23.806827 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:44:23.806883 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:44:23.807435 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:44:23.807851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:23.809037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:44:23.809037 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:23.809037 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:44:23.809037 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:44:23.809037 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:23.809037 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:44:23.809037 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:23.809037 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:44:23.810501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:23.810501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:44:23.810501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:44:23.810501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:44:23.810501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:23.810501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:44:23.810501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:23.810501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:44:23.810501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:23.810501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:44:23.810501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:23.810501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:44:23.810501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:23.810501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:44:23.810501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:44:23.811441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:23.811441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:44:23.811441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:23.811441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:44:23.811441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:44:23.811441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:23.811441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:44:23.811441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:23.811441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:44:23.812279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:23.812279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:44:23.812279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:44:23.812279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:44:23.812279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:44:23.812279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:44:23.812279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:44:23.812279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:44:23.812279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:44:23.812279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:44:23.812279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:23.812279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:44:23.812945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:23.812945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:44:23.812945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:23.812945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:44:23.812945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:23.812945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:44:23.812945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:23.812945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:44:23.812945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:44:23.812945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:23.812945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:44:23.812945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:23.812945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:44:23.812945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:44:23.813501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:23.813501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:44:23.813501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:23.813501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:44:23.813670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:23.813670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:44:23.813670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:44:23.813670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:44:23.813670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:44:23.813670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:44:23.813670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:44:23.813670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:44:23.813670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:44:23.813670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:44:23.813670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:44:23.813670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:44:23.813670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:44:23.813670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:44:23.813670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:44:23.813670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:44:23.814312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:44:23.814312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:44:23.814312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:44:23.814312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:44:23.814312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:44:23.814312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:23.814312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:44:23.814312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:23.814312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:44:23.814312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:23.814312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:44:23.814312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:23.814312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:44:23.814312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:23.814312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:44:23.814312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:44:23.814312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:23.814312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:44:23.815037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:23.815037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:44:23.815037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:44:23.815037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:23.815037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:44:23.815037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:23.815037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:44:23.815037 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:44:23.815037 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:44:23.815037 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:44:23.815452 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:44:23.815452 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:44:23.815452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:23.815827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:44:23.815827 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:23.815827 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:44:23.815827 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:44:23.815827 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:23.815827 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:44:23.815827 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:23.815827 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:44:23.816618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:23.816618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:44:23.816618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:44:23.816618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:44:23.816618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:23.816618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:44:23.816618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:23.816618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:44:23.816618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:23.816618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:44:23.816618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:23.816618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:44:23.816618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:23.816618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:44:23.816618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:44:23.817176 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:23.817176 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:44:23.817176 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:23.817176 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:44:23.817176 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:44:23.817176 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:23.817176 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:44:23.817176 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:23.817176 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:44:23.817515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:23.817515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:44:23.817515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:44:23.817515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:44:23.817515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:44:23.817515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:44:23.817515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:44:23.817515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:44:23.817515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:44:23.817515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:23.817515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:44:23.818017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:23.818017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:44:23.818017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:23.818017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:44:23.818017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:23.818017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:44:23.818017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:23.818017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:44:23.818017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:44:23.818017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:23.818017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:44:23.818017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:23.818017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:44:23.818017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:44:23.818567 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:23.818567 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:44:23.818567 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:23.818567 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:44:23.818768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:23.818768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:44:23.818768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:44:23.818768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:44:23.818768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:44:23.818768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:44:23.818768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:44:23.818768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:44:23.818768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:44:23.818768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:44:23.818768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:44:23.818768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:44:23.818768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:44:23.818768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:44:23.818768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:44:23.818768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:44:23.818768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:44:23.819447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:44:23.819447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:44:23.819447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:44:23.819447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:23.819447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:44:23.819447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:23.819447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:44:23.819447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:23.819447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:44:23.819447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:23.819447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:44:23.819447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:23.819447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:44:23.819447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:44:23.819447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:23.819447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:44:23.819447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:23.819447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:44:23.819447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:44:23.820220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:23.820220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:44:23.820220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:23.820220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:44:23.820220 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:44:23.820220 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:44:23.820220 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:44:23.820220 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:44:23.820584 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:44:23.820584 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:44:23.820584 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:23.820952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:44:23.820952 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:23.820952 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:44:23.820952 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:44:23.820952 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:23.820952 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:44:23.820952 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:23.820952 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:44:23.821846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:23.821846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:44:23.821846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:44:23.821846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:44:23.821846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:44:23.821846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:44:23.821846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:44:23.821846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:23.821846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:44:23.821846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:23.821846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:44:23.821846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:23.822194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:44:23.822194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:23.822194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:44:23.822194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:23.822194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:44:23.822194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:44:23.822194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:23.822194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:44:23.822194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:23.822194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:44:23.822194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:44:23.822194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:23.822194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:44:23.822537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:23.822537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:44:23.822915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:23.822915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:44:23.822915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:44:23.822915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:44:23.822915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:44:23.822915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:44:23.822915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:44:23.822915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:44:23.822915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:44:23.822915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:44:23.822915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:44:23.822915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:44:23.822915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:44:23.822915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:44:23.822915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:44:23.822915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:44:23.823319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:44:23.823319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:44:23.823319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:23.823319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:44:23.823319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:23.823319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:44:23.823319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:23.823319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:44:23.823319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:23.823319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:44:23.823319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:23.823319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:44:23.823319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:44:23.823319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:23.823319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:44:23.823319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:23.823319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:44:23.823319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:44:23.823319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:23.823821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:44:23.823821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:23.823821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:44:28.652737 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:44:28.653404 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:44:28.653404 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:44:28.654843 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:44:28.689824 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:44:28.690403 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:28.691629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:44:28.691629 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:28.691629 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:44:28.691629 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:44:28.691629 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:28.691629 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:44:28.691629 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:28.691629 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:44:28.692943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:28.692943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:44:28.692943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:44:28.692943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:44:28.692943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:44:28.692943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:44:28.692943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:44:28.692943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:28.692943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:44:28.692943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:28.692943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:44:28.692943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:28.693383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:44:28.693383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:28.693383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:44:28.693383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:28.693383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:44:28.693383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:44:28.693383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:28.693383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:44:28.693383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:28.693383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:44:28.693383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:44:28.693383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:28.693383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:44:28.693842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:28.693842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:44:28.694371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:28.694371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:44:28.694371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:44:28.694371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:44:28.694371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:44:28.694371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:44:28.694371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:44:28.694371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:44:28.694371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:44:28.694371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:44:28.694371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:44:28.694371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:44:28.694371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:44:28.694371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:44:28.694371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:44:28.694371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:44:28.694956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:44:28.694956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:44:28.694956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:28.694956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:44:28.694956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:28.694956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:44:28.694956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:28.694956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:44:28.694956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:28.694956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:44:28.694956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:28.694956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:44:28.694956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:44:28.694956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:28.694956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:44:28.694956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:28.694956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:44:28.694956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:44:28.694956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:28.695558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:44:28.695558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:28.695558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:44:28.797695 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:44:28.798351 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:44:28.798483 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:44:28.799630 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:44:28.799996 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:44:28.800228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:28.801210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:44:28.801210 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:28.801210 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:44:28.801210 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:44:28.801210 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:28.801210 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:44:28.801210 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:28.801210 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:44:28.802867 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:28.802867 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:44:28.802867 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:44:28.802867 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:44:28.802867 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:44:28.802867 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:44:28.802867 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:44:28.802867 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:28.802867 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:44:28.802867 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:28.802867 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:44:28.802867 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:28.803267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:44:28.803267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:28.803267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:44:28.803267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:28.803267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:44:28.803267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:44:28.803267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:28.803267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:44:28.803267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:28.803267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:44:28.803267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:44:28.803267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:28.803267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:44:28.803646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:28.803646 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:44:28.804727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:28.804727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:44:28.804727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:44:28.804727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:44:28.804727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:44:28.804727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:44:28.804727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:44:28.804727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:44:28.804727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:44:28.804727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:44:28.804727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:44:28.804727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:44:28.804727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:44:28.804727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:44:28.804727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:44:28.804727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:44:28.805454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:44:28.805454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:44:28.805454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:28.805454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:44:28.805454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:28.805454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:44:28.805454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:28.805454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:44:28.805454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:28.805454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:44:28.805454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:28.805454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:44:28.805454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:44:28.805454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:28.805454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:44:28.805454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:28.805454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:44:28.805454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:44:28.805454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:28.806072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:44:28.806072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:28.806072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:44:28.806589 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:44:28.806935 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:44:28.807076 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:44:28.807246 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:44:28.807831 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:44:28.808146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:28.808946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:44:28.808946 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:28.808946 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:44:28.808946 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:44:28.808946 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:28.808946 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:44:28.808946 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:28.808946 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:44:28.809880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:28.809880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:44:28.809880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:44:28.809880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:44:28.809880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:28.809880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:44:28.809880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:28.809880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:44:28.809880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:28.809880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:44:28.809880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:28.809880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:44:28.809880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:28.809880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:44:28.809880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:44:28.810391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:28.810391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:44:28.810391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:28.810391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:44:28.810391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:44:28.810391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:28.810391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:44:28.810391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:28.810391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:44:28.810970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:28.810970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:44:28.810970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:44:28.810970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:44:28.810970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:44:28.810970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:44:28.810970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:44:28.810970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:44:28.810970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:44:28.810970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:44:28.810970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:28.810970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:44:28.811483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:28.811483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:44:28.811483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:28.811483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:44:28.811483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:28.811483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:44:28.811483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:28.811483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:44:28.811483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:44:28.811483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:28.811483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:44:28.811483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:28.811483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:44:28.811483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:44:28.811938 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:28.811938 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:44:28.811938 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:28.811938 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:44:28.812354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:28.812354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:44:28.812354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:44:28.812354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:44:28.812354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:44:28.812354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:44:28.812354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:44:28.812354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:44:28.812354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:44:28.812354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:44:28.812354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:44:28.812354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:44:28.812354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:44:28.812354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:44:28.812354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:44:28.812354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:44:28.813108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:44:28.813108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:44:28.813108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:44:28.813108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:44:28.813108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:44:28.813108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:28.813108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:44:28.813108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:28.813108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:44:28.813108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:28.813108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:44:28.813108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:28.813108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:44:28.813108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:28.813108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:44:28.813108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:44:28.813108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:28.813108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:44:28.813916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:28.813916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:44:28.813916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:44:28.813916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:28.813916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:44:28.813916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:28.813916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:44:28.814545 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:44:28.814856 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:44:28.814943 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:44:28.815090 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:44:28.815599 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:44:28.816064 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:28.816912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:44:28.816912 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:28.816912 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:44:28.816912 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:44:28.816912 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:28.816912 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:44:28.816912 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:28.816912 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:44:28.817904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:28.817904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:44:28.817904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:44:28.817904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:44:28.817904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:28.817904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:44:28.817904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:28.817904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:44:28.817904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:28.817904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:44:28.817904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:28.817904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:44:28.817904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:28.817904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:44:28.817904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:44:28.818600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:28.818600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:44:28.818600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:28.818600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:44:28.818600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:44:28.818600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:28.818600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:44:28.818600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:28.818600 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:44:28.819087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:28.819087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:44:28.819087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:44:28.819087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:44:28.819087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:44:28.819087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:44:28.819087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:44:28.819087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:44:28.819087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:44:28.819087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:28.819087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:44:28.819619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:28.819619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:44:28.819619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:28.819619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:44:28.819619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:28.819619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:44:28.819619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:28.819619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:44:28.819619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:44:28.819619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:28.819619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:44:28.819619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:28.819619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:44:28.819619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:44:28.820177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:28.820177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:44:28.820177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:28.820177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:44:28.820707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:28.820707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:44:28.820707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:44:28.820707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:44:28.820707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:44:28.820707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:44:28.820707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:44:28.820707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:44:28.820707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:44:28.820707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:44:28.820707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:44:28.820707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:44:28.820707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:44:28.820707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:44:28.820707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:44:28.820707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:44:28.820707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:44:28.821459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:44:28.821459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:44:28.821459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:44:28.821459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:28.821459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:44:28.821459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:28.821459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:44:28.821459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:28.821459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:44:28.821459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:28.821459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:44:28.821459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:28.821459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:44:28.821459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:44:28.821459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:28.821459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:44:28.821459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:28.821459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:44:28.821459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:44:28.822274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:28.822274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:44:28.822274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:28.822274 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:44:28.822274 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:44:28.822274 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:44:28.822274 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:44:28.822274 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:44:28.822642 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:44:28.822711 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:44:28.822954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:28.823514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:44:28.823514 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:28.823514 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:44:28.823514 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:44:28.823514 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:28.823514 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:44:28.823514 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:28.823514 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:44:28.824434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:28.824434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:44:28.824434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:44:28.824434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:44:28.824434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:44:28.824434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:44:28.824434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:44:28.824434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:28.824434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:44:28.824434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:28.824434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:44:28.824434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:28.824977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:44:28.824977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:28.824977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:44:28.824977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:28.824977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:44:28.824977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:44:28.824977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:28.824977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:44:28.824977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:28.824977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:44:28.824977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:44:28.824977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:28.824977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:44:28.825436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:28.825436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:44:28.825798 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:28.825798 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:44:28.825798 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:44:28.825798 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:44:28.825798 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:44:28.825798 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:44:28.825798 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:44:28.825798 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:44:28.825798 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:44:28.825798 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:44:28.825798 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:44:28.825798 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:44:28.825798 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:44:28.825798 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:44:28.825798 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:44:28.825798 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:44:28.826370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:44:28.826370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:44:28.826370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:28.826370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:44:28.826370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:28.826370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:44:28.826370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:28.826370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:44:28.826370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:28.826370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:44:28.826370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:28.826370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:44:28.826370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:44:28.826370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:28.826370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:44:28.826370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:28.826370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:44:28.826370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:44:28.826370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:28.827369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:44:28.827369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:28.827369 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:44:33.653634 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:44:33.654319 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:44:33.654427 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:44:33.655588 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:44:33.687223 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:44:33.688807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:33.688807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:44:33.688807 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:33.688807 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:44:33.688807 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:44:33.688807 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:33.688807 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:44:33.688807 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:33.688807 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:44:33.690046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:33.690046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:44:33.690046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:44:33.690046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:44:33.690046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:44:33.690046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:44:33.690046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:44:33.690046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:33.690046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:44:33.690046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:33.690046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:44:33.690046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:33.690461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:44:33.690461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:33.690461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:44:33.690461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:33.690461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:44:33.690461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:44:33.690461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:33.690461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:44:33.690461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:33.690461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:44:33.690461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:44:33.690461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:33.690461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:44:33.690905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:33.690905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:44:33.691844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:33.691844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:44:33.691844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:44:33.691844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:44:33.691844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:44:33.691844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:44:33.691844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:44:33.691844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:44:33.691844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:44:33.691844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:44:33.691844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:44:33.691844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:44:33.691844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:44:33.691844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:44:33.691844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:44:33.691844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:44:33.692171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:44:33.692171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:44:33.692171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:33.692171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:44:33.692171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:33.692171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:44:33.692171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:33.692171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:44:33.692171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:33.692171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:44:33.692171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:33.692171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:44:33.692171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:44:33.692171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:33.692171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:44:33.692171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:33.692171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:44:33.692171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:44:33.692171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:33.692512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:44:33.692512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:33.692512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:44:33.798415 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:44:33.799441 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:44:33.799561 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:44:33.800277 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:44:33.800470 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:44:33.800609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:33.801281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:44:33.801281 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:33.801281 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:44:33.801281 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:44:33.801281 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:33.801281 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:44:33.801281 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:33.801281 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:44:33.802281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:33.802281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:44:33.802281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:44:33.802281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:44:33.802281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:44:33.802281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:44:33.802281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:44:33.802281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:33.802281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:44:33.802281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:33.802281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:44:33.802281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:33.804191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:44:33.804191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:33.804191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:44:33.804191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:33.804191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:44:33.804191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:44:33.804191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:33.804191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:44:33.804191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:33.804191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:44:33.804191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:44:33.804191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:33.804191 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:44:33.804536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:33.804536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:44:33.804536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:33.804536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:44:33.804536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:44:33.804536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:44:33.804536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:44:33.804536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:44:33.804536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:44:33.804536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:44:33.804536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:44:33.804536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:44:33.804536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:44:33.804536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:44:33.804536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:44:33.804983 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:44:33.804983 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:44:33.804983 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:44:33.804983 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:44:33.804983 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:44:33.804983 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:33.804983 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:44:33.804983 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:33.804983 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:44:33.804983 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:33.804983 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:44:33.804983 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:33.804983 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:44:33.804983 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:33.804983 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:44:33.804983 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:44:33.804983 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:33.804983 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:44:33.804983 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:33.805468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:44:33.805468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:44:33.805468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:33.805468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:44:33.805468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:33.805468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:44:33.807201 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:44:33.807561 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:44:33.807725 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:44:33.807933 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:44:33.808402 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:44:33.808619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:33.809205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:44:33.809205 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:33.809205 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:44:33.809205 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:44:33.809205 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:33.809205 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:44:33.809205 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:33.809205 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:44:33.810014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:33.810014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:44:33.810014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:44:33.810014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:44:33.810014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:33.810014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:44:33.810014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:33.810014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:44:33.810014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:33.810014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:44:33.810014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:33.810014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:44:33.810014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:33.810014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:44:33.810014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:44:33.810981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:33.810981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:44:33.810981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:33.810981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:44:33.810981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:44:33.810981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:33.810981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:44:33.810981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:33.810981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:44:33.810981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:33.810981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:44:33.810981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:44:33.810981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:44:33.810981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:44:33.811847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:44:33.811847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:44:33.811847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:44:33.811847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:44:33.811847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:44:33.811847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:33.811847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:44:33.811847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:33.811847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:44:33.811847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:33.811847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:44:33.811847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:33.811847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:44:33.812518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:33.812518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:44:33.812518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:44:33.812518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:33.812518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:44:33.812518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:33.812518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:44:33.812518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:44:33.812518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:33.812518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:44:33.812518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:33.812518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:44:33.813146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:33.813146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:44:33.813146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:44:33.813146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:44:33.813146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:44:33.813146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:44:33.813146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:44:33.813146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:44:33.813146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:44:33.813146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:44:33.813146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:44:33.813146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:44:33.813146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:44:33.813146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:44:33.813146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:44:33.813146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:44:33.814088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:44:33.814088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:44:33.814088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:44:33.814088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:44:33.814088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:44:33.814088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:33.814088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:44:33.814088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:33.814088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:44:33.814088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:33.814088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:44:33.814088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:33.814088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:44:33.814088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:33.814088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:44:33.814088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:44:33.814088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:33.814088 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:44:33.815059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:33.815059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:44:33.815059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:44:33.815059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:33.815059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:44:33.815059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:33.815059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:44:33.815059 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:44:33.815514 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:44:33.815514 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:44:33.815643 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:44:33.815863 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:44:33.816082 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:33.816644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:44:33.816644 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:33.816644 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:44:33.816644 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:44:33.816644 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:33.816644 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:44:33.816644 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:33.816644 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:44:33.817354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:33.817354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:44:33.817354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:44:33.817354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:44:33.817354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:33.817354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:44:33.817354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:33.817354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:44:33.817354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:33.817354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:44:33.817354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:33.817354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:44:33.817354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:33.817354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:44:33.817354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:44:33.817627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:33.817627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:44:33.817627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:33.817627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:44:33.817627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:44:33.817627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:33.817627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:44:33.817627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:33.817627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:44:33.818415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:33.818415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:44:33.818415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:44:33.818415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:44:33.818415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:44:33.818415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:44:33.818415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:44:33.818415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:44:33.818415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:44:33.818415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:33.818415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:44:33.818683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:33.818683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:44:33.818683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:33.818683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:44:33.818683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:33.818683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:44:33.818683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:33.818683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:44:33.818683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:44:33.818683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:33.818683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:44:33.818683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:33.818683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:44:33.818683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:44:33.819003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:33.819003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:44:33.819003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:33.819003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:44:33.819580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:33.819580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:44:33.819580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:44:33.819580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:44:33.819580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:44:33.819580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:44:33.819580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:44:33.819580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:44:33.819580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:44:33.819580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:44:33.819580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:44:33.819580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:44:33.819580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:44:33.819580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:44:33.819580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:44:33.819580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:44:33.819580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:44:33.819927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:44:33.819927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:44:33.819927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:44:33.819927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:33.819927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:44:33.819927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:33.819927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:44:33.819927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:33.819927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:44:33.819927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:33.819927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:44:33.819927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:33.819927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:44:33.819927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:44:33.819927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:33.819927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:44:33.819927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:33.819927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:44:33.819927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:44:33.820295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:33.820295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:44:33.820295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:33.820295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:44:33.821829 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:44:33.822184 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:44:33.822343 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:44:33.822511 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:44:33.822783 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:44:33.823031 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:44:33.823256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:33.823698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:44:33.823698 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:33.823698 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:44:33.823698 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:44:33.823698 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:33.823698 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:44:33.823698 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:33.823698 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:44:33.824710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:33.824710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:44:33.824710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:44:33.824710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:44:33.824710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:44:33.824710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:44:33.824710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:44:33.824710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:33.824710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:44:33.824710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:33.824710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:44:33.824710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:33.825051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:44:33.825051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:33.825051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:44:33.825051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:33.825051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:44:33.825051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:44:33.825051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:33.825051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:44:33.825051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:33.825051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:44:33.825051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:44:33.825051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:33.825051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:44:33.825302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:33.825302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:44:33.825899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:33.825899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:44:33.825899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:44:33.825899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:44:33.825899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:44:33.825899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:44:33.825899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:44:33.825899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:44:33.825899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:44:33.825899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:44:33.825899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:44:33.825899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:44:33.825899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:44:33.825899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:44:33.825899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:44:33.825899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:44:33.826564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:44:33.826564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:44:33.826564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:33.826564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:44:33.826564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:33.826564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:44:33.826564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:33.826564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:44:33.826564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:33.826564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:44:33.826564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:33.826564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:44:33.826564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:44:33.826564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:33.826564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:44:33.826564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:33.826564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:44:33.826564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:44:33.826564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:33.827300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:44:33.827300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:33.827300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:44:38.654601 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:44:38.655427 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:44:38.655427 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:44:38.656592 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:44:38.698793 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:44:38.698986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:38.699402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:44:38.699402 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:38.699402 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:44:38.699402 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:44:38.699402 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:38.699402 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:44:38.699402 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:38.699402 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:44:38.700197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.044 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:38.700197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:44:38.700197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:44:38.700197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:44:38.700197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:44:38.700197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:44:38.700197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:44:38.700197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:38.700197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:44:38.700197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:38.700197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:44:38.700197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:38.700569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:44:38.700569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:38.700569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:44:38.700569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:38.700569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:44:38.700569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:44:38.700569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:38.700569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:44:38.700569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:38.700569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:44:38.700569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:44:38.700569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:38.700569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:44:38.700944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:38.700944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:44:38.701258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:38.701258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:44:38.701258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:44:38.701258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:44:38.701258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:44:38.701258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:44:38.701258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:44:38.701258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:44:38.701258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:44:38.701258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:44:38.701258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:44:38.701258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:44:38.701258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:44:38.701258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:44:38.701258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:44:38.701258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:44:38.702354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:44:38.702354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:44:38.702354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:38.702354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:44:38.702354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:38.702354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:44:38.702354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:38.702354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:44:38.702354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:38.702354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:44:38.702354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:38.702354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:44:38.702354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:44:38.702354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:38.702354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:44:38.702354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:38.702354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:44:38.702354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:44:38.702354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:38.703194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:44:38.703194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:38.703194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:44:38.799634 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:44:38.800301 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:44:38.800492 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:44:38.801641 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:44:38.801982 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:44:38.802204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:38.803052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:44:38.803052 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:38.803052 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:44:38.803052 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:44:38.803052 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:38.803052 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:44:38.803052 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:38.803052 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:44:38.804700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:38.804700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:44:38.804700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:44:38.804700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:44:38.804700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:44:38.804700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:44:38.804700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:44:38.804700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:38.804700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:44:38.804700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:38.804700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:44:38.804700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:38.805060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:44:38.805060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:38.805060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:44:38.805060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:38.805060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:44:38.805060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:44:38.805060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:38.805060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:44:38.805060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:38.805060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:44:38.805060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:44:38.805060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:38.805060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:44:38.805341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:38.805341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:44:38.806794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:38.806794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:44:38.806794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:44:38.806794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:44:38.806794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:44:38.806794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:44:38.806794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:44:38.806794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:44:38.806794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:44:38.806794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:44:38.806794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:44:38.806794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:44:38.806794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:44:38.806794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:44:38.806794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:44:38.806794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:44:38.807179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:44:38.807179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:44:38.807179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:38.807179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:44:38.807179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:38.807179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:44:38.807179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:38.807179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:44:38.807179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:38.807179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:44:38.807179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:38.807179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:44:38.807179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:44:38.807179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:38.807179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:44:38.807179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:38.807179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:44:38.807179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:44:38.807179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:38.807611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:44:38.807611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:38.807611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:44:38.807685 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:44:38.808055 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:44:38.808158 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:44:38.808509 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:44:38.808875 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:44:38.809087 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:38.809631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:44:38.809631 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:38.809631 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:44:38.809631 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:44:38.809631 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:38.809631 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:44:38.809631 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:38.809631 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:44:38.810449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:38.810449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:44:38.810449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:44:38.810449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:44:38.810449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:38.810449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:44:38.810449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:38.810449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:44:38.810449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:38.810449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:44:38.810449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:38.810449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:44:38.810449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:38.810449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:44:38.810449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:44:38.811355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:38.811355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:44:38.811355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:38.811355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:44:38.811355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:44:38.811355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:38.811355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:44:38.811355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:38.811355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:44:38.811871 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:38.811871 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:44:38.811871 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:44:38.811871 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:44:38.811871 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:44:38.811871 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:44:38.811871 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:44:38.811871 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:44:38.811871 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:44:38.811871 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:44:38.811871 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:38.811871 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:44:38.812189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:38.812189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:44:38.812189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:38.812189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:44:38.812189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:38.812189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:44:38.812189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:38.812189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:44:38.812189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:44:38.812189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:38.812189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:44:38.812189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:38.812189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:44:38.812189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:44:38.812693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:38.812693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:44:38.812693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:38.812693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:44:38.813117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:38.813117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:44:38.813117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:44:38.813117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:44:38.813117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:44:38.813117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:44:38.813117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:44:38.813117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:44:38.813117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:44:38.813117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:44:38.813117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:44:38.813117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:44:38.813117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:44:38.813117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:44:38.813117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:44:38.813117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:44:38.814147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:44:38.814147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:44:38.814147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:44:38.814147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:44:38.814147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:44:38.814147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:38.814147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:44:38.814147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:38.814147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:44:38.814147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:38.814147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:44:38.814147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:38.814147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:44:38.814147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:38.814147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:44:38.814147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:44:38.814147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:38.814147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:44:38.814941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:38.814941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:44:38.814941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:44:38.814941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:38.814941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:44:38.814941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:38.814941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:44:38.815493 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:44:38.815858 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:44:38.815965 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:44:38.816116 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:44:38.816644 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:44:38.816954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:38.817598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:44:38.817598 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:38.817598 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:44:38.817598 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:44:38.817598 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:38.817598 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:44:38.817598 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:38.817598 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:44:38.818512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:38.818512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:44:38.818512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:44:38.818512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:44:38.818512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:38.818512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:44:38.818512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:38.818512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:44:38.818512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:38.818512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:44:38.818512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:38.818512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:44:38.818512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:38.818512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:44:38.818512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:44:38.818877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:38.818877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:44:38.818877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:38.818877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:44:38.818877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:44:38.818877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:38.818877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:44:38.818877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:38.818877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:44:38.819476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:38.819476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:44:38.819476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:44:38.819476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:44:38.819476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:44:38.819476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:44:38.819476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:44:38.819476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:44:38.819476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:44:38.819476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:38.819476 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:44:38.819798 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:38.819798 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:44:38.819798 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:38.819798 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:44:38.819798 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:38.819798 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:44:38.819798 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:38.819798 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:44:38.819798 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:44:38.819798 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:38.819798 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:44:38.819798 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:38.819798 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:44:38.819798 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:44:38.820103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:38.820103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:44:38.820103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:38.820103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:44:38.821065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:38.821065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:44:38.821065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:44:38.821065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:44:38.821065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:44:38.821065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:44:38.821065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:44:38.821065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:44:38.821065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:44:38.821065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:44:38.821065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:44:38.821065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:44:38.821065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:44:38.821065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:44:38.821065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:44:38.821065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:44:38.821065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:44:38.821478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:44:38.821478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:44:38.821478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:44:38.821478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:38.821478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:44:38.821478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:38.821478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:44:38.821478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:38.821478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:44:38.821478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:38.821478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:44:38.821478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:38.821478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:44:38.821478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:44:38.821478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:38.821478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:44:38.821478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:38.821478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:44:38.821478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:44:38.821955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:38.821955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:44:38.821955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:38.821955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:44:38.822391 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:44:38.822752 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:44:38.822883 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:44:38.823059 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:44:38.823354 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:44:38.823906 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:44:38.824063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:38.824776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:44:38.824776 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:38.824776 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:44:38.824776 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:44:38.824776 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:38.824776 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:44:38.824776 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:38.824776 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:44:38.825633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:38.825633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:44:38.825633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:44:38.825633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:44:38.825633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:44:38.825633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:44:38.825633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:44:38.825633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:38.825633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:44:38.825633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:38.825633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:44:38.825633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:38.826002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:44:38.826002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:38.826002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:44:38.826002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:38.826002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:44:38.826002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:44:38.826002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:38.826002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:44:38.826002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:38.826002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:44:38.826002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:44:38.826002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:38.826002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:44:38.826282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:38.826282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:44:38.826781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:38.826781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:44:38.826781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:44:38.826781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:44:38.826781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:44:38.826781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:44:38.826781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:44:38.826781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:44:38.826781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:44:38.826781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:44:38.826781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:44:38.826781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:44:38.826781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:44:38.826781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:44:38.826781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:44:38.826781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:44:38.827170 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:44:38.827170 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:44:38.827170 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:38.827170 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:44:38.827170 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:38.827170 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:44:38.827170 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:38.827170 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:44:38.827170 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:38.827170 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:44:38.827170 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:38.827170 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:44:38.827170 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:44:38.827170 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:38.827170 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:44:38.827170 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:38.827170 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:44:38.827170 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:44:38.827170 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:38.827604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:44:38.827604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:38.827604 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:44:43.655366 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:44:43.656438 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:44:43.656438 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:44:43.657674 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:44:43.685189 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:44:43.685397 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:43.685804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:44:43.685804 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:43.685804 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:44:43.685804 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:44:43.685804 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:43.685804 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:44:43.685804 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:43.685804 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:44:43.686484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.030 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:43.686484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:44:43.686484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:44:43.686484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:44:43.686484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:44:43.686484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:44:43.686484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:44:43.686484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:43.686484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:44:43.686484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:43.686484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:44:43.686484 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:43.686884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:44:43.686884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:43.686884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:44:43.686884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:43.686884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:44:43.686884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:44:43.686884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:43.686884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:44:43.686884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:43.686884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:44:43.686884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:44:43.686884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:43.686884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:44:43.687200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:43.687200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:44:43.687462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:43.687462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:44:43.687462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:44:43.687462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:44:43.687462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:44:43.687462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:44:43.687462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:44:43.687462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:44:43.687462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:44:43.687462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:44:43.687462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:44:43.687462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:44:43.687462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:44:43.687462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:44:43.687462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:44:43.687462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:44:43.687839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:44:43.687839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:44:43.687839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:43.687839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:44:43.687839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:43.687839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:44:43.687839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:43.687839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:44:43.687839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:43.687839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:44:43.687839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:43.687839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:44:43.687839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:44:43.687839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:43.687839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:44:43.687839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:43.687839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:44:43.687839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:44:43.687839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:43.688232 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:44:43.688232 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:43.688232 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:44:43.800359 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:44:43.801123 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:44:43.801380 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:44:43.802560 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:44:43.803033 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:44:43.803380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:43.804782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:44:43.804782 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:43.804782 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:44:43.804782 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:44:43.804782 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:43.804782 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:44:43.804782 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:43.804782 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:44:43.807915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:43.807915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:44:43.807915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:44:43.807915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:44:43.807915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:44:43.807915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:44:43.807915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:44:43.807915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:43.807915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:44:43.807915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:43.807915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:44:43.807915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:43.808982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:44:43.808982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:43.808982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:44:43.808982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:43.808982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:44:43.808982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:44:43.808982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:43.808982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:44:43.808982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:43.808982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:44:43.808982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:44:43.808982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:43.808982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:44:43.809594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:43.809594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:44:43.809884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:43.809884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:44:43.809884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:44:43.809884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:44:43.809884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:44:43.809884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:44:43.809884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:44:43.809884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:44:43.809884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:44:43.809884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:44:43.809884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:44:43.809884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:44:43.809884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:44:43.809884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:44:43.809884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:44:43.809884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:44:43.810796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:44:43.810796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:44:43.810796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:43.810796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:44:43.810796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:43.810796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:44:43.810796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:43.810796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:44:43.810796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:43.810796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:44:43.810796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:43.810796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:44:43.810796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:44:43.810796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:43.810796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:44:43.810796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:43.810796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:44:43.810796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:44:43.810796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:43.811814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:44:43.811814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:43.811814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:44:43.811814 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:44:43.811814 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:44:43.811814 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:44:43.811814 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:44:43.811814 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:44:43.812339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:43.812452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:44:43.812452 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:43.812452 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:44:43.812452 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:44:43.812452 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:43.812452 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:44:43.812452 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:43.812452 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:44:43.813416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:43.813416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:44:43.813416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:44:43.813416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:44:43.813416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:43.813416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:44:43.813416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:43.813416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:44:43.813416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:43.813416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:44:43.813416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:43.813416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:44:43.813416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:43.813416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:44:43.813416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:44:43.814319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:43.814319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:44:43.814319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:43.814319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:44:43.814319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:44:43.814319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:43.814319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:44:43.814319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:43.814319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:44:43.814880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:43.814880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:44:43.814880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:44:43.814880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:44:43.814880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:44:43.814880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:44:43.814880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:44:43.814880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:44:43.814880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:44:43.814880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:44:43.814880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:43.814880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:44:43.815566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:43.815566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:44:43.815566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:43.815566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:44:43.815566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:43.815566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:44:43.815566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:43.815566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:44:43.815566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:44:43.815566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:43.815566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:44:43.815566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:43.815566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:44:43.815566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:44:43.816300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:43.816300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:44:43.816300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:43.816300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:44:43.816300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:43.816300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:44:43.816300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:44:43.816300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:44:43.816300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:44:43.816300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:44:43.816300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:44:43.816300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:44:43.816300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:44:43.816300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:44:43.817112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:44:43.817112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:44:43.817112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:44:43.817112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:44:43.817112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:44:43.817112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:44:43.817112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:44:43.817112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:44:43.817112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:44:43.817112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:44:43.817112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:44:43.817112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:43.817112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:44:43.817112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:43.817112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:44:43.817112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:43.817112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:44:43.817112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:43.817112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:44:43.817985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:43.817985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:44:43.817985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:44:43.817985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:43.817985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:44:43.817985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:43.817985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:44:43.817985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:44:43.817985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:43.817985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:44:43.817985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:43.817985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:44:43.817985 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:44:43.817985 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:44:43.818636 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:44:43.818636 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:44:43.818636 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:44:43.818636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:43.818636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:44:43.818636 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:43.818636 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:44:43.818636 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:44:43.818636 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:43.818636 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:44:43.818636 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:43.818636 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:44:43.819307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:43.819307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:44:43.819307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:44:43.819307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:44:43.819307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:43.819307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:44:43.819307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:43.819307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:44:43.819307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:43.819307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:44:43.819307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:43.819307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:44:43.819307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:43.819307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:44:43.819307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:44:43.819938 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:43.819938 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:44:43.819938 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:43.819938 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:44:43.819938 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:44:43.819938 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:43.819938 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:44:43.819938 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:43.819938 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:44:43.820120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:43.820120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:44:43.820120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:44:43.820120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:44:43.820120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:44:43.820120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:44:43.820120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:44:43.820120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:44:43.820120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:44:43.820120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:43.820120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:44:43.820382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:43.820382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:44:43.820382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:43.820382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:44:43.820382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:43.820382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:44:43.820382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:43.820382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:44:43.820382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:44:43.820382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:43.820382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:44:43.820382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:43.820382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:44:43.820382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:44:43.820668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:43.820668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:44:43.820668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:43.820668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:44:43.821052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:43.821052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:44:43.821052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:44:43.821052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:44:43.821052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:44:43.821052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:44:43.821052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:44:43.821052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:44:43.821052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:44:43.821052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:44:43.821052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:44:43.821052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:44:43.821052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:44:43.821052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:44:43.821052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:44:43.821052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:44:43.821052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:44:43.821501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:44:43.821501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:44:43.821501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:44:43.821501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:43.821501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:44:43.821501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:43.821501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:44:43.821501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:43.821501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:44:43.821501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:43.821501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:44:43.821501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:43.821501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:44:43.821501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:44:43.821501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:43.821501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:44:43.821501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:43.821501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:44:43.821501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:44:43.821914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:43.821914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:44:43.821914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:43.821914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:44:43.822977 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:44:43.823238 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:44:43.823318 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:44:43.823439 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:44:43.823625 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:44:43.823819 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:44:43.823926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:43.824729 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:44:43.824729 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:43.824729 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:44:43.824729 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:44:43.824729 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:43.824729 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:44:43.824729 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:43.824729 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:44:43.826573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:43.826573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:44:43.826573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:44:43.826573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:44:43.826573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:44:43.826573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:44:43.826573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:44:43.826573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:43.826573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:44:43.826573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:43.826573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:44:43.826573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:43.826887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:44:43.826887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:43.826887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:44:43.826887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:43.826887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:44:43.826887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:44:43.826887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:43.826887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:44:43.826887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:43.826887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:44:43.826887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:44:43.826887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:43.826887 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:44:43.827149 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:43.827149 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:44:43.828218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:43.828218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:44:43.828218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:44:43.828218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:44:43.828218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:44:43.828218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:44:43.828218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:44:43.828218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:44:43.828218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:44:43.828218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:44:43.828218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:44:43.828218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:44:43.828218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:44:43.828218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:44:43.828218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:44:43.828218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:44:43.828546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:44:43.828546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:44:43.828546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:43.828546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:44:43.828546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:43.828546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:44:43.828546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:43.828546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:44:43.828546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:43.828546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:44:43.828546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:43.828546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:44:43.828546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:44:43.828546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:43.828546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:44:43.828546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:43.828546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:44:43.828546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:44:43.828546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:43.828962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:44:43.828962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:43.828962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:44:47.984379 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_for_inconsistencies' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:44:47.985288 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_for_inconsistencies {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:44:47.985288 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Checking Neutron and OVN revision consistency. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:44:47.993064 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Maintenance task: Synchronizing Neutron and OVN databases started {{(pid=106314) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:440}}
May 09 09:44:47.993186 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Maintenance task: Number of inconsistencies found at create/update: networks=4, routers=1, router_ports=1 {{(pid=106314) _log /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:419}}
May 09 09:44:47.993308 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Maintenance task: Fixing resource f8a38fcb-e8a2-421a-a340-bf7f2f5cc72b (type: networks) at create/update {{(pid=106314) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 09 09:44:48.030572 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-f8a38fcb-e8a2-421a-a340-bf7f2f5cc72b, resource={'id': 'f8a38fcb-e8a2-421a-a340-bf7f2f5cc72b', 'name': 'private', 'tenant_id': 'f4d54dfe001d43fbb344093a1ccabd6a', 'project_id': 'f4d54dfe001d43fbb344093a1ccabd6a', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['96e72a03-97b8-412a-b2b9-1bf048ce1066', 'bbd6957a-953f-4fc1-8761-5d93128564d9'], 'standard_attr_id': 12, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-05-09T09:04:34Z', 'updated_at': '2026-05-09T09:04:37Z', 'revision_number': 3, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 3730}, resource_type=networks, if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:44:48.030872 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-f8a38fcb-e8a2-421a-a340-bf7f2f5cc72b, col_values=(('external_ids', {'neutron:network_name': 'private', 'neutron:mtu': '1372', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:44:48.031185 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-78a32c7c-7637-4b83-b0ab-62aee8bb3038, external_ids_update=None, columns={'tag': 3730}, if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:44:48.031373 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:48.032277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:44:48.032277 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:48.032277 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:44:48.032277 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:44:48.032277 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:48.032277 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:44:48.032277 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:48.032277 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:44:48.033890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Maintenance task: Failed to fix resource f8a38fcb-e8a2-421a-a340-bf7f2f5cc72b (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:48.033890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:44:48.033890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 09 09:44:48.033890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 09 09:44:48.033890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 09 09:44:48.033890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 09 09:44:48.033890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2332, in update_network
May 09 09:44:48.033890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:44:48.033890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:48.033890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:44:48.033890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:48.034483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:44:48.034483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:48.034483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:44:48.034483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:48.034483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:44:48.034483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:48.034483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:44:48.034483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:44:48.034483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:48.034483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:44:48.034483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:48.034483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:44:48.034483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:44:48.034483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:48.035235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:44:48.035235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:48.035235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:44:48.035235 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Maintenance task: Fixing resource d18a9fac-2ebf-4d1e-a86c-0f33fba8eeda (type: networks) at create/update {{(pid=106314) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 09 09:44:48.062549 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-d18a9fac-2ebf-4d1e-a86c-0f33fba8eeda, resource={'id': 'd18a9fac-2ebf-4d1e-a86c-0f33fba8eeda', 'name': 'public', 'tenant_id': 'f4d54dfe001d43fbb344093a1ccabd6a', 'project_id': 'f4d54dfe001d43fbb344093a1ccabd6a', 'admin_state_up': True, 'mtu': 1430, 'status': 'ACTIVE', 'subnets': ['088bbd0f-7e81-4a76-8399-24d15c984009', 'bbd6a8cc-80e6-4336-a70b-1c962c561590'], 'standard_attr_id': 18, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': True, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'is_default': True, 'tags': [], 'created_at': '2026-05-09T09:04:40Z', 'updated_at': '2026-05-09T09:04:50Z', 'revision_number': 3, 'provider:network_type': 'flat', 'provider:physical_network': 'public', 'provider:segmentation_id': None}, resource_type=networks, if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:44:48.063096 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-d18a9fac-2ebf-4d1e-a86c-0f33fba8eeda, col_values=(('external_ids', {'neutron:network_name': 'public', 'neutron:mtu': '1430', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'flat', 'neutron:provnet-physical-network': 'public'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false', 'fdb_age_threshold': '300', 'broadcast-arps-to-all-routers': 'true'})), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:44:48.063713 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-1163f525-70cd-4b84-ae9b-7d56378305e9, external_ids_update=None, columns={'tag': []}, if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:44:48.064109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:48.064801 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:44:48.064801 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:48.064801 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:44:48.064801 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:44:48.064801 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:48.064801 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:44:48.064801 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:48.064801 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:44:48.065626 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Maintenance task: Failed to fix resource d18a9fac-2ebf-4d1e-a86c-0f33fba8eeda (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:48.065626 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:44:48.065626 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 09 09:44:48.065626 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 09 09:44:48.065626 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 09 09:44:48.065626 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 09 09:44:48.065626 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2332, in update_network
May 09 09:44:48.065626 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:44:48.065626 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:48.065626 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:44:48.065626 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:48.066443 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:44:48.066443 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:48.066443 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:44:48.066443 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:48.066443 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:44:48.066443 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:48.066443 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:44:48.066443 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:44:48.066443 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:48.066443 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:44:48.066443 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:48.066443 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:44:48.066443 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:44:48.066443 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:48.067279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:44:48.067279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:48.067279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:44:48.067279 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Maintenance task: Fixing resource 417da935-2882-452f-8ba7-9076d68fcd2b (type: networks) at create/update {{(pid=106314) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 09 09:44:48.093277 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-417da935-2882-452f-8ba7-9076d68fcd2b, resource={'id': '417da935-2882-452f-8ba7-9076d68fcd2b', 'name': 'admin_net', 'tenant_id': '38e536cfc9c94b6cbf529caf55cc7739', 'project_id': '38e536cfc9c94b6cbf529caf55cc7739', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['d3c53d14-f490-4615-a986-3715a315f827'], 'standard_attr_id': 42, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-05-09T09:10:38Z', 'updated_at': '2026-05-09T09:10:39Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 32823}, resource_type=networks, if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:44:48.093598 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-417da935-2882-452f-8ba7-9076d68fcd2b, col_values=(('external_ids', {'neutron:network_name': 'admin_net', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:44:48.094274 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-5738ab2a-b471-447b-9292-7f8c6bc97168, external_ids_update=None, columns={'tag': 32823}, if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:44:48.094486 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:48.095413 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:44:48.095413 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:48.095413 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:44:48.095413 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:44:48.095413 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:48.095413 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:44:48.095413 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:48.095413 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:44:48.096504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Maintenance task: Failed to fix resource 417da935-2882-452f-8ba7-9076d68fcd2b (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:48.096504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:44:48.096504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 09 09:44:48.096504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 09 09:44:48.096504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 09 09:44:48.096504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 09 09:44:48.096504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2332, in update_network
May 09 09:44:48.096504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:44:48.096504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:48.096504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:44:48.096504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:48.097186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:44:48.097186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:48.097186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:44:48.097186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:48.097186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:44:48.097186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:48.097186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:44:48.097186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:44:48.097186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:48.097186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:44:48.097186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:48.097186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:44:48.097186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:44:48.097186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:48.098034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:44:48.098034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:48.098034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:44:48.098034 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Maintenance task: Fixing resource 433617e8-cdf8-414a-8cc6-63a7b5d0c678 (type: networks) at create/update {{(pid=106314) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 09 09:44:48.128101 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-433617e8-cdf8-414a-8cc6-63a7b5d0c678, resource={'id': '433617e8-cdf8-414a-8cc6-63a7b5d0c678', 'name': 'k8s-clusterapi-cluster-magnum-system-kube-whtb8', 'tenant_id': '3e013d45a44544c3b50eb0e83e4b1efc', 'project_id': '3e013d45a44544c3b50eb0e83e4b1efc', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['9187f065-9264-427a-8df8-672a6981191d'], 'standard_attr_id': 52, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-05-09T09:11:20Z', 'updated_at': '2026-05-09T09:11:20Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 32644}, resource_type=networks, if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:44:48.128343 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-433617e8-cdf8-414a-8cc6-63a7b5d0c678, col_values=(('external_ids', {'neutron:network_name': 'k8s-clusterapi-cluster-magnum-system-kube-whtb8', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:44:48.128717 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-8c3fe586-c182-47ba-9845-d7608113cb71, external_ids_update=None, columns={'tag': 32644}, if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:44:48.128879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:48.129458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:44:48.129458 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:48.129458 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:44:48.129458 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:44:48.129458 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:48.129458 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:44:48.129458 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:48.129458 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:44:48.130368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Maintenance task: Failed to fix resource 433617e8-cdf8-414a-8cc6-63a7b5d0c678 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:48.130368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:44:48.130368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 09 09:44:48.130368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 09 09:44:48.130368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 09 09:44:48.130368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 09 09:44:48.130368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2332, in update_network
May 09 09:44:48.130368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:44:48.130368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:48.130368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:44:48.130368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:48.130920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:44:48.130920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:48.130920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:44:48.130920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:48.130920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:44:48.130920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:48.130920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:44:48.130920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:44:48.130920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:48.130920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:44:48.130920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:48.130920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:44:48.130920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:44:48.130920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:48.131538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:44:48.131538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:48.131538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:44:48.131538 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Maintenance task: Fixing resource cab10384-14c5-492c-8bd0-061f5838310c (type: routers) at create/update {{(pid=106314) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 09 09:44:48.205375 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-cab10384-14c5-492c-8bd0-061f5838310c, resource={'id': 'cab10384-14c5-492c-8bd0-061f5838310c', 'name': 'k8s-clusterapi-cluster-magnum-system-kube-whtb8', 'project_id': '3e013d45a44544c3b50eb0e83e4b1efc', 'admin_state_up': True, 'status': 'ACTIVE', 'enable_snat': True, 'external_gateway_info': {'network_id': 'd18a9fac-2ebf-4d1e-a86c-0f33fba8eeda', 'external_fixed_ips': [{'subnet_id': 'bbd6a8cc-80e6-4336-a70b-1c962c561590', 'ip_address': '172.24.5.116'}, {'subnet_id': '088bbd0f-7e81-4a76-8399-24d15c984009', 'ip_address': '2001:db8::1b7'}], 'enable_snat': True}, 'gw_port_id': 'af118f11-dc01-429a-b20a-e97b77b8d5ec', 'description': 'Created by cluster-api-provider-openstack cluster magnum-system-kube-whtb8', 'availability_zones': [], 'distributed': False, 'ha': True, 'ha_vr_id': 0, 'availability_zone_hints': [], 'enable_default_route_ecmp': False, 'enable_default_route_bfd': False, 'external_gateways': [{'network_id': 'd18a9fac-2ebf-4d1e-a86c-0f33fba8eeda', 'external_fixed_ips': [{'ip_address': '172.24.5.116', 'subnet_id': 'bbd6a8cc-80e6-4336-a70b-1c962c561590'}, {'ip_address': '2001:db8::1b7', 'subnet_id': '088bbd0f-7e81-4a76-8399-24d15c984009'}]}], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2026-05-09T09:11:21Z', 'updated_at': '2026-05-09T09:11:24Z', 'revision_number': 3, 'tenant_id': '3e013d45a44544c3b50eb0e83e4b1efc'}, resource_type=routers, if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:44:48.205609 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterCommand(_result=None, name=neutron-cab10384-14c5-492c-8bd0-061f5838310c, columns={'external_ids': {'neutron:router_name': 'k8s-clusterapi-cluster-magnum-system-kube-whtb8', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': ''}, 'enabled': True}, if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:44:48.205938 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): QoSDelCommand(_result=None, switch=neutron-d18a9fac-2ebf-4d1e-a86c-0f33fba8eeda, conditions=[('direction', '=', 'from-lport'), ('priority', '=', 2002), ('match', '=', 'inport == "af118f11-dc01-429a-b20a-e97b77b8d5ec"')], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:44:48.206092 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): QoSDelCommand(_result=None, switch=neutron-d18a9fac-2ebf-4d1e-a86c-0f33fba8eeda, conditions=[('direction', '=', 'to-lport'), ('priority', '=', 2002), ('match', '=', 'outport == "af118f11-dc01-429a-b20a-e97b77b8d5ec"')], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:44:48.206232 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:48.206801 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:44:48.206801 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:48.206801 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:44:48.206801 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:44:48.206801 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:48.206801 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:44:48.206801 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:48.206801 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:44:48.207033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_client [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Unable to update router cab10384-14c5-492c-8bd0-061f5838310c. Error: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:48.208127 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Maintenance task: Failed to fix resource cab10384-14c5-492c-8bd0-061f5838310c (type: routers): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:48.208127 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:44:48.208127 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 09 09:44:48.208127 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 09 09:44:48.208127 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 09 09:44:48.208127 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 09 09:44:48.208127 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1635, in update_router
May 09 09:44:48.208127 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with excutils.save_and_reraise_exception():
May 09 09:44:48.208127 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 271, in __exit__
May 09 09:44:48.208127 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.force_reraise()
May 09 09:44:48.208127 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 233, in force_reraise
May 09 09:44:48.208472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise self.value
May 09 09:44:48.208472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1569, in update_router
May 09 09:44:48.208472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:44:48.208472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:48.208472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:44:48.208472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:48.208472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:44:48.208472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:48.208472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:44:48.208472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:48.208472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:44:48.208472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:48.208472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:44:48.208472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:44:48.208865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:48.208865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:44:48.208865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:48.208865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:44:48.208865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:44:48.208865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:48.208865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:44:48.208865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:48.208865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:44:48.208865 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Maintenance task: Fixing resource af118f11-dc01-429a-b20a-e97b77b8d5ec (type: router_ports) at create/update {{(pid=106314) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
May 09 09:44:48.329293 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=lrp-af118f11-dc01-429a-b20a-e97b77b8d5ec, resource={'id': 'af118f11-dc01-429a-b20a-e97b77b8d5ec', 'name': '', 'network_id': 'd18a9fac-2ebf-4d1e-a86c-0f33fba8eeda', 'project_id': '', 'mac_address': 'fa:16:3e:51:04:94', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'cab10384-14c5-492c-8bd0-061f5838310c', 'device_owner': 'network:router_gateway', 'standard_attr_id': 57, 'fixed_ips': [{'subnet_id': 'bbd6a8cc-80e6-4336-a70b-1c962c561590', 'ip_address': '172.24.5.116'}, {'subnet_id': '088bbd0f-7e81-4a76-8399-24d15c984009', 'ip_address': '2001:db8::1b7'}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'np0000173033', 'binding:vif_type': 'ovs', 'binding:vif_details': {'port_filter': True, 'connectivity': 'l2', 'ovs_create_tap': False, 'bridge_name': 'br-int', 'datapath_type': 'system', 'bound_drivers': {'0': 'ovn'}}, 'port_security_enabled': False, 'tags': [], 'created_at': '2026-05-09T09:11:21Z', 'updated_at': '2026-05-09T09:11:24Z', 'revision_number': 5, 'tenant_id': ''}, resource_type=router_ports, if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:44:48.329782 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterPortCommand(_result=None, name=lrp-af118f11-dc01-429a-b20a-e97b77b8d5ec, columns={'external_ids': {'neutron:revision_number': '5', 'neutron:subnet_ids': 'bbd6a8cc-80e6-4336-a70b-1c962c561590 088bbd0f-7e81-4a76-8399-24d15c984009', 'neutron:network_name': 'neutron-d18a9fac-2ebf-4d1e-a86c-0f33fba8eeda', 'neutron:is_ext_gw': 'True', 'neutron:router_name': 'neutron-cab10384-14c5-492c-8bd0-061f5838310c'}, 'options': {'reside-on-redirect-chassis': 'true', 'gateway_mtu': '1372'}, 'networks': ['172.24.5.116/24', '2001:db8::1b7/64'], 'ipv6_ra_configs': {}}, if_exists=False) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:44:48.330349 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLRouterPortInLSwitchPortCommand(_result=None, lswitch_port=af118f11-dc01-429a-b20a-e97b77b8d5ec, lrouter_port=lrp-af118f11-dc01-429a-b20a-e97b77b8d5ec, is_gw_port=True, if_exists=True, lsp_address=router) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:44:48.330730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:48.331674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:44:48.331674 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:48.331674 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:44:48.331674 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:44:48.331674 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:48.331674 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:44:48.331674 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:48.331674 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:44:48.332849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Maintenance task: Failed to fix resource af118f11-dc01-429a-b20a-e97b77b8d5ec (type: router_ports): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:48.332849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:44:48.332849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
May 09 09:44:48.332849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
May 09 09:44:48.332849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
May 09 09:44:48.332849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
May 09 09:44:48.332849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1976, in update_router_port
May 09 09:44:48.332849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:44:48.332849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:48.332849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:44:48.332849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:48.333383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:44:48.333383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:48.333383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:44:48.333383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:48.333383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:44:48.333383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:48.333383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:44:48.333383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:44:48.333383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:48.333383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:44:48.333383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:48.333383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:44:48.333383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:44:48.333383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:48.334061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:44:48.334061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:48.334061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:44:48.334061 np0000173033 neutron-ovn-maintenance-worker[106314]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_for_inconsistencies finished in 0.348 seconds
May 09 09:44:48.656073 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:44:48.656906 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:44:48.657011 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:44:48.658065 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:44:48.700420 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:44:48.700804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:48.701725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:44:48.701725 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:48.701725 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:44:48.701725 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:44:48.701725 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:48.701725 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:44:48.701725 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:48.701725 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:44:48.702611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.045 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:48.702611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:44:48.702611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:44:48.702611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:44:48.702611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:44:48.702611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:44:48.702611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:44:48.702611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:48.702611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:44:48.702611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:48.702611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:44:48.702611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:48.702943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:44:48.702943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:48.702943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:44:48.702943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:48.702943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:44:48.702943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:44:48.702943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:48.702943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:44:48.702943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:48.702943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:44:48.702943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:44:48.702943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:48.702943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:44:48.703178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:48.703178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:44:48.703441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:48.703441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:44:48.703441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:44:48.703441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:44:48.703441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:44:48.703441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:44:48.703441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:44:48.703441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:44:48.703441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:44:48.703441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:44:48.703441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:44:48.703441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:44:48.703441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:44:48.703441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:44:48.703441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:44:48.703441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:44:48.703890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:44:48.703890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:44:48.703890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:48.703890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:44:48.703890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:48.703890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:44:48.703890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:48.703890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:44:48.703890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:48.703890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:44:48.703890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:48.703890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:44:48.703890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:44:48.703890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:48.703890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:44:48.703890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:48.703890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:44:48.703890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:44:48.703890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:48.704253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:44:48.704253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:48.704253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:44:48.801104 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:44:48.801895 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:44:48.802090 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:44:48.803333 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:44:48.803643 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:44:48.803812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:48.804609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:44:48.804609 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:48.804609 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:44:48.804609 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:44:48.804609 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:48.804609 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:44:48.804609 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:48.804609 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:44:48.806065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:48.806065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:44:48.806065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:44:48.806065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:44:48.806065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:44:48.806065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:44:48.806065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:44:48.806065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:48.806065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:44:48.806065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:48.806065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:44:48.806065 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:48.806891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:44:48.806891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:48.806891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:44:48.806891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:48.806891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:44:48.806891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:44:48.806891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:48.806891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:44:48.806891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:48.806891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:44:48.806891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:44:48.806891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:48.806891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:44:48.807772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:48.807772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:44:48.807772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:48.807772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:44:48.807772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:44:48.807772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:44:48.807772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:44:48.807772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:44:48.807772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:44:48.807772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:44:48.807772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:44:48.807772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:44:48.807772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:44:48.807772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:44:48.807772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:44:48.809098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:44:48.809098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:44:48.809098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:44:48.809098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:44:48.809098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:44:48.809098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:48.809098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:44:48.809098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:48.809098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:44:48.809098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:48.809098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:44:48.809098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:48.809098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:44:48.809098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:48.809098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:44:48.809098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:44:48.809098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:48.809098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:44:48.809098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:48.809618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:44:48.809618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:44:48.809618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:48.809618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:44:48.809618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:48.809618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:44:48.810847 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:44:48.811213 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:44:48.811349 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:44:48.811480 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:44:48.812109 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:44:48.812496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:48.813519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:44:48.813519 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:48.813519 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:44:48.813519 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:44:48.813519 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:48.813519 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:44:48.813519 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:48.813519 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:44:48.814377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:48.814377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:44:48.814377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:44:48.814377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:44:48.814377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:48.814377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:44:48.814377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:48.814377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:44:48.814377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:48.814377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:44:48.814377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:48.814377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:44:48.814377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:48.814377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:44:48.814377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:44:48.814899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:48.814899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:44:48.814899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:48.814899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:44:48.814899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:44:48.814899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:48.814899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:44:48.814899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:48.814899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:44:48.815293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:48.815293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:44:48.815293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:44:48.815293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:44:48.815293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:44:48.815293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:44:48.815293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:44:48.815293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:44:48.815293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:44:48.815293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:44:48.815293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:48.815293 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:44:48.815613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:48.815613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:44:48.815613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:48.815613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:44:48.815613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:48.815613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:44:48.815613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:48.815613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:44:48.815613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:44:48.815613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:48.815613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:44:48.815613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:48.815613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:44:48.815613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:44:48.815988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:48.815988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:44:48.815988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:48.815988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:44:48.816392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:48.816392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:44:48.816392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:44:48.816392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:44:48.816392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:44:48.816392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:44:48.816392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:44:48.816392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:44:48.816392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:44:48.816392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:44:48.816392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:44:48.816392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:44:48.816392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:44:48.816392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:44:48.816392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:44:48.816392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:44:48.816834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:44:48.816834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:44:48.816834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:44:48.816834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:44:48.816834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:44:48.816834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:48.816834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:44:48.816834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:48.816834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:44:48.816834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:48.816834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:44:48.816834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:48.816834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:44:48.816834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:48.816834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:44:48.816834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:44:48.816834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:48.816834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:44:48.817303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:48.817303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:44:48.817303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:44:48.817303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:48.817303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:44:48.817303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:48.817303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:44:48.817303 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:44:48.817303 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:44:48.817303 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:44:48.817600 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:44:48.818103 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:44:48.818407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:48.819184 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:44:48.819184 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:48.819184 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:44:48.819184 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:44:48.819184 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:48.819184 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:44:48.819184 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:48.819184 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:44:48.820166 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:48.820166 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:44:48.820166 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:44:48.820166 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:44:48.820166 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:48.820166 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:44:48.820166 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:48.820166 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:44:48.820166 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:48.820166 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:44:48.820166 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:48.820166 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:44:48.820166 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:48.820166 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:44:48.820166 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:44:48.820597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:48.820597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:44:48.820597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:48.820597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:44:48.820597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:44:48.820597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:48.820597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:44:48.820597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:48.820597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:44:48.821106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:48.821106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:44:48.821106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:44:48.821106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:44:48.821106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:44:48.821106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:44:48.821106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:44:48.821106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:44:48.821106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:44:48.821106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:48.821106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:44:48.821505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:48.821505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:44:48.821505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:48.821505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:44:48.821505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:48.821505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:44:48.821505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:48.821505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:44:48.821505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:44:48.821505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:48.821505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:44:48.821505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:48.821505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:44:48.821505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:44:48.821891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:48.821891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:44:48.821891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:48.821891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:44:48.822310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:48.822310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:44:48.822310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:44:48.822310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:44:48.822310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:44:48.822310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:44:48.822310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:44:48.822310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:44:48.822310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:44:48.822310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:44:48.822310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:44:48.822310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:44:48.822310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:44:48.822310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:44:48.822310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:44:48.822310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:44:48.822310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:44:48.822868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:44:48.822868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:44:48.822868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:44:48.822868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:48.822868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:44:48.822868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:48.822868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:44:48.822868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:48.822868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:44:48.822868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:48.822868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:44:48.822868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:48.822868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:44:48.822868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:44:48.822868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:48.822868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:44:48.822868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:48.822868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:44:48.822868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:44:48.823391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:48.823391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:44:48.823391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:48.823391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:44:48.823615 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:44:48.823950 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:44:48.824068 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:44:48.824236 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:44:48.824677 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:44:48.825007 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:44:48.825236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:48.825868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:44:48.825868 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:48.825868 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:44:48.825868 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:44:48.825868 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:48.825868 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:44:48.825868 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:48.825868 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:44:48.826985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:48.826985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:44:48.826985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:44:48.826985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:44:48.826985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:44:48.826985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:44:48.826985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:44:48.826985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:48.826985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:44:48.826985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:48.826985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:44:48.826985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:48.827346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:44:48.827346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:48.827346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:44:48.827346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:48.827346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:44:48.827346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:44:48.827346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:48.827346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:44:48.827346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:48.827346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:44:48.827346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:44:48.827346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:48.827346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:44:48.827671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:48.827671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:44:48.828049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:48.828049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:44:48.828049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:44:48.828049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:44:48.828049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:44:48.828049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:44:48.828049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:44:48.828049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:44:48.828049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:44:48.828049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:44:48.828049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:44:48.828049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:44:48.828049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:44:48.828049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:44:48.828049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:44:48.828049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:44:48.828512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:44:48.828512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:44:48.828512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:48.828512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:44:48.828512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:48.828512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:44:48.828512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:48.828512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:44:48.828512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:48.828512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:44:48.828512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:48.828512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:44:48.828512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:44:48.828512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:48.828512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:44:48.828512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:48.828512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:44:48.828512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:44:48.828512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:48.829050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:44:48.829050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:48.829050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:44:53.656977 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:44:53.657698 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:44:53.657698 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:44:53.658785 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:44:53.702739 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:44:53.702739 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:53.703078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:44:53.703078 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:53.703078 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:44:53.703078 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:44:53.703078 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:53.703078 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:44:53.703078 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:53.703078 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:44:53.704172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.045 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:53.704172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:44:53.704172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:44:53.704172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:44:53.704172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:44:53.704172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:44:53.704172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:44:53.704172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:53.704172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:44:53.704172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:53.704172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:44:53.704172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:53.704973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:44:53.704973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:53.704973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:44:53.704973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:53.704973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:44:53.704973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:44:53.704973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:53.704973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:44:53.704973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:53.704973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:44:53.704973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:44:53.704973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:53.704973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:44:53.705705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:53.705705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:44:53.705705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:53.705705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:44:53.705705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:44:53.705705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:44:53.705705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:44:53.705705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:44:53.705705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:44:53.705705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:44:53.705705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:44:53.705705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:44:53.705705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:44:53.705705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:44:53.705705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:44:53.706627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:44:53.706627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:44:53.706627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:44:53.706627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:44:53.706627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:44:53.706627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:53.706627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:44:53.706627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:53.706627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:44:53.706627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:53.706627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:44:53.706627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:53.706627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:44:53.706627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:53.706627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:44:53.706627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:44:53.706627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:53.706627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:44:53.706627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:53.707652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:44:53.707652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:44:53.707652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:53.707652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:44:53.707652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:53.707652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:44:53.801897 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:44:53.802589 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:44:53.803121 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:44:53.804141 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:44:53.804477 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:44:53.804982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:53.805860 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:44:53.805860 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:53.805860 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:44:53.805860 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:44:53.805860 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:53.805860 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:44:53.805860 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:53.805860 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:44:53.806974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:53.806974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:44:53.806974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:44:53.806974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:44:53.806974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:44:53.806974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:44:53.806974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:44:53.806974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:53.806974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:44:53.806974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:53.806974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:44:53.806974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:53.807933 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:44:53.807933 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:53.807933 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:44:53.807933 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:53.807933 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:44:53.807933 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:44:53.807933 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:53.807933 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:44:53.807933 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:53.807933 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:44:53.807933 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:44:53.807933 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:53.807933 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:44:53.808620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:53.808620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:44:53.808620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:53.808620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:44:53.808620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:44:53.808620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:44:53.808620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:44:53.808620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:44:53.808620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:44:53.808620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:44:53.808620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:44:53.808620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:44:53.808620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:44:53.808620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:44:53.808620 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:44:53.809401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:44:53.809401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:44:53.809401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:44:53.809401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:44:53.809401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:44:53.809401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:53.809401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:44:53.809401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:53.809401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:44:53.809401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:53.809401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:44:53.809401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:53.809401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:44:53.809401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:53.809401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:44:53.809401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:44:53.809401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:53.809401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:44:53.809401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:53.810326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:44:53.810326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:44:53.810326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:53.810326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:44:53.810326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:53.810326 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:44:53.811272 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:44:53.811534 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:44:53.811642 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:44:53.811781 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:44:53.812272 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:44:53.812631 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:53.813468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:44:53.813468 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:53.813468 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:44:53.813468 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:44:53.813468 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:53.813468 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:44:53.813468 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:53.813468 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:44:53.814777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:53.814777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:44:53.814777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:44:53.814777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:44:53.814777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:53.814777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:44:53.814777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:53.814777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:44:53.814777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:53.814777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:44:53.814777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:53.814777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:44:53.814777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:53.814777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:44:53.814777 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:44:53.815659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:53.815659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:44:53.815659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:53.815659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:44:53.815659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:44:53.815659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:53.815659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:44:53.815659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:53.815659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:44:53.816488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:53.816488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:44:53.816488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:44:53.816488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:44:53.816488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:44:53.816488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:44:53.816488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:44:53.816488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:44:53.816488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:44:53.816488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:44:53.816488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:53.816488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:44:53.817260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:53.817260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:44:53.817260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:53.817260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:44:53.817260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:53.817260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:44:53.817260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:53.817260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:44:53.817260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:44:53.817260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:53.817260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:44:53.817260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:53.817260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:44:53.817260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:44:53.817895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:53.817895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:44:53.817895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:53.817895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:44:53.818598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:53.818598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:44:53.818598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:44:53.818598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:44:53.818598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:44:53.818598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:44:53.818598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:44:53.818598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:44:53.818598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:44:53.818598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:44:53.818598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:44:53.818598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:44:53.818598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:44:53.818598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:44:53.818598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:44:53.818598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:44:53.819402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:44:53.819402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:44:53.819402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:44:53.819402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:44:53.819402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:44:53.819402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:53.819402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:44:53.819402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:53.819402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:44:53.819402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:53.819402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:44:53.819402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:53.819402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:44:53.819402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:53.819402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:44:53.819402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:44:53.819402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:53.819402 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:44:53.820233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:53.820233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:44:53.820233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:44:53.820233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:53.820233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:44:53.820233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:53.820233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:44:53.820233 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:44:53.820233 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:44:53.820233 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:44:53.820597 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:44:53.820597 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:44:53.820811 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:53.821699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:44:53.821699 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:53.821699 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:44:53.821699 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:44:53.821699 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:53.821699 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:44:53.821699 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:53.821699 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:44:53.822821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:53.822821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:44:53.822821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:44:53.822821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:44:53.822821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:53.822821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:44:53.822821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:53.822821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:44:53.822821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:53.822821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:44:53.822821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:53.822821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:44:53.822821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:53.822821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:44:53.822821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:44:53.823636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:53.823636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:44:53.823636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:53.823636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:44:53.823636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:44:53.823636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:53.823636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:44:53.823636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:53.823636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:44:53.823636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:53.823636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:44:53.823636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:44:53.823636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:44:53.823636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:44:53.824483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:44:53.824483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:44:53.824483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:44:53.824483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:44:53.824483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:53.824483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:44:53.824483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:53.824483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:44:53.824483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:53.824483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:44:53.824483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:53.824483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:44:53.824483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:53.825414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:44:53.825414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:44:53.825414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:53.825414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:44:53.825414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:53.825414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:44:53.825414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:44:53.825414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:53.825414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:44:53.825414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:53.825414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:44:53.825414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:53.826267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:44:53.826267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:44:53.826267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:44:53.826267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:44:53.826267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:44:53.826267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:44:53.826267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:44:53.826267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:44:53.826267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:44:53.826267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:44:53.826267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:44:53.826267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:44:53.826267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:44:53.826267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:44:53.826267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:44:53.826267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:44:53.826267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:44:53.826267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:44:53.826267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:44:53.827285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:53.827285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:44:53.827285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:53.827285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:44:53.827285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:53.827285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:44:53.827285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:53.827285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:44:53.827285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:53.827285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:44:53.827285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:44:53.827285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:53.827285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:44:53.827285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:53.827285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:44:53.827285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:44:53.827285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:53.827285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:44:53.827285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:53.827285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:44:53.828270 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:44:53.828270 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:44:53.828270 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:44:53.828270 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:44:53.828270 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:44:53.828270 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:44:53.828698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:53.828698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:44:53.828698 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:53.828698 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:44:53.828698 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:44:53.828698 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:53.828698 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:44:53.828698 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:53.828698 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:44:53.829520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:53.829520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:44:53.829520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:44:53.829520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:44:53.829520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:44:53.829520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:44:53.829520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:44:53.829520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:53.829520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:44:53.829520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:53.829520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:44:53.829520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:53.830361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:44:53.830361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:53.830361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:44:53.830361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:53.830361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:44:53.830361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:44:53.830361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:53.830361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:44:53.830361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:53.830361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:44:53.830361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:44:53.830361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:53.830361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:44:53.831155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:53.831155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:44:53.831155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:53.831155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:44:53.831155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:44:53.831155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:44:53.831155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:44:53.831155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:44:53.831155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:44:53.831155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:44:53.831155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:44:53.831155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:44:53.831155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:44:53.831155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:44:53.831155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:44:53.832025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:44:53.832025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:44:53.832025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:44:53.832025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:44:53.832025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:44:53.832025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:53.832025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:44:53.832025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:53.832025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:44:53.832025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:53.832025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:44:53.832025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:53.832025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:44:53.832025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:53.832025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:44:53.832025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:44:53.832025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:53.832025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:44:53.832025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:53.833154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:44:53.833154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:44:53.833154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:53.833154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:44:53.833154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:53.833154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:44:58.658502 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:44:58.661911 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:44:58.661911 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:44:58.661911 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:44:58.663081 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG dbcounter [-] [106314] Writing DB stats neutron:SELECT=434 {{(pid=106314) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 09 09:44:58.703098 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:44:58.703312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:58.704013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:44:58.704013 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:58.704013 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:44:58.704013 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:44:58.704013 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:58.704013 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:44:58.704013 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:58.704013 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:44:58.705162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.044 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:58.705162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:44:58.705162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:44:58.705162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:44:58.705162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:44:58.705162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:44:58.705162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:44:58.705162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:58.705162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:44:58.705162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:58.705162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:44:58.705162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:58.705446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:44:58.705446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:58.705446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:44:58.705446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:58.705446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:44:58.705446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:44:58.705446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:58.705446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:44:58.705446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:58.705446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:44:58.705446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:44:58.705446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:58.705446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:44:58.705680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:58.705680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:44:58.706288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:58.706288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:44:58.706288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:44:58.706288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:44:58.706288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:44:58.706288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:44:58.706288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:44:58.706288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:44:58.706288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:44:58.706288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:44:58.706288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:44:58.706288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:44:58.706288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:44:58.706288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:44:58.706288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:44:58.706288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:44:58.706621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:44:58.706621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:44:58.706621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:58.706621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:44:58.706621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:58.706621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:44:58.706621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:58.706621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:44:58.706621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:58.706621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:44:58.706621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:58.706621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:44:58.706621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:44:58.706621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:58.706621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:44:58.706621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:58.706621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:44:58.706621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:44:58.706621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:58.707091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:44:58.707091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:58.707091 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:44:58.802573 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:44:58.803314 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:44:58.803578 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:44:58.804925 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:44:58.805159 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:44:58.805285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:58.806119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:44:58.806119 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:58.806119 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:44:58.806119 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:44:58.806119 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:58.806119 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:44:58.806119 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:58.806119 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:44:58.807772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:58.807772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:44:58.807772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:44:58.807772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:44:58.807772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:44:58.807772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:44:58.807772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:44:58.807772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:58.807772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:44:58.807772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:58.807772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:44:58.807772 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:58.808529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:44:58.808529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:58.808529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:44:58.808529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:58.808529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:44:58.808529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:44:58.808529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:58.808529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:44:58.808529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:58.808529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:44:58.808529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:44:58.808529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:58.808529 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:44:58.809178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:58.809178 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:44:58.810239 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:58.810239 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:44:58.810239 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:44:58.810239 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:44:58.810239 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:44:58.810239 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:44:58.810239 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:44:58.810239 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:44:58.810239 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:44:58.810239 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:44:58.810239 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:44:58.810239 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:44:58.810239 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:44:58.810239 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:44:58.810239 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:44:58.810239 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:44:58.811123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:44:58.811123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:44:58.811123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:58.811123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:44:58.811123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:58.811123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:44:58.811123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:58.811123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:44:58.811123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:58.811123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:44:58.811123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:58.811123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:44:58.811123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:44:58.811123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:58.811123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:44:58.811123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:58.811123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:44:58.811123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:44:58.811123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:58.811991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:44:58.811991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:58.811991 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:44:58.811991 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:44:58.812433 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:44:58.812773 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:44:58.812927 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:44:58.813465 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:44:58.813762 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:58.814591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:44:58.814591 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:58.814591 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:44:58.814591 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:44:58.814591 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:58.814591 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:44:58.814591 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:58.814591 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:44:58.815509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:58.815509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:44:58.815509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:44:58.815509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:44:58.815509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:58.815509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:44:58.815509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:58.815509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:44:58.815509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:58.815509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:44:58.815509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:58.815509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:44:58.815509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:58.815509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:44:58.815509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:44:58.815965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:58.815965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:44:58.815965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:58.815965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:44:58.815965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:44:58.815965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:58.815965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:44:58.815965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:58.815965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:44:58.816509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:58.816509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:44:58.816509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:44:58.816509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:44:58.816509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:44:58.816509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:44:58.816509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:44:58.816509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:44:58.816509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:44:58.816509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:44:58.816509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:58.816509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:44:58.816955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:58.816955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:44:58.816955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:58.816955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:44:58.816955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:58.816955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:44:58.816955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:58.816955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:44:58.816955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:44:58.816955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:58.816955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:44:58.816955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:58.816955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:44:58.816955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:44:58.817357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:58.817357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:44:58.817357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:58.817357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:44:58.820213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:58.820213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:44:58.820213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:44:58.820213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:44:58.820213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:44:58.820213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:44:58.820213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:44:58.820213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:44:58.820213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:44:58.820213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:44:58.820213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:44:58.820213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:44:58.820213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:44:58.820213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:44:58.820213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:44:58.820213 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:44:58.820727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:44:58.820727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:44:58.820727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:44:58.820727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:44:58.820727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:44:58.820727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:58.820727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:44:58.820727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:58.820727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:44:58.820727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:58.820727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:44:58.820727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:58.820727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:44:58.820727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:58.820727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:44:58.820727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:44:58.820727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:58.820727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:44:58.821216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:58.821216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:44:58.821216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:44:58.821216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:58.821216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:44:58.821216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:58.821216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:44:58.821216 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:44:58.821216 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:44:58.821216 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:44:58.821522 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:44:58.821522 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:44:58.821770 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:58.822390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:44:58.822390 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:58.822390 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:44:58.822390 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:44:58.822390 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:58.822390 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:44:58.822390 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:58.822390 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:44:58.823157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:58.823157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:44:58.823157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:44:58.823157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:44:58.823157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:58.823157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:44:58.823157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:58.823157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:44:58.823157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:58.823157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:44:58.823157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:58.823157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:44:58.823157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:58.823157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:44:58.823157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:44:58.823574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:58.823574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:44:58.823574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:58.823574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:44:58.823574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:44:58.823574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:58.823574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:44:58.823574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:58.823574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:44:58.824045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:58.824045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:44:58.824045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:44:58.824045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:44:58.824045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:44:58.824045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:44:58.824045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:44:58.824045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:44:58.824045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:44:58.824045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:58.824045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:44:58.824495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:58.824495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:44:58.824495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:58.824495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:44:58.824495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:58.824495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:44:58.824495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:58.824495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:44:58.824495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:44:58.824495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:58.824495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:44:58.824495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:58.824495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:44:58.824495 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:44:58.824942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:58.824942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:44:58.824942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:58.824942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:44:58.825261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:58.825261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:44:58.825261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:44:58.825261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:44:58.825261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:44:58.825261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:44:58.825261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:44:58.825261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:44:58.825261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:44:58.825261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:44:58.825261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:44:58.825261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:44:58.825261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:44:58.825261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:44:58.825261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:44:58.825261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:44:58.825261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:44:58.825784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:44:58.825784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:44:58.825784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:44:58.825784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:58.825784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:44:58.825784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:58.825784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:44:58.825784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:58.825784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:44:58.825784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:58.825784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:44:58.825784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:58.825784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:44:58.825784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:44:58.825784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:58.825784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:44:58.825784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:58.825784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:44:58.825784 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:44:58.826279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:58.826279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:44:58.826279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:58.826279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:44:58.826279 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:44:58.826279 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:44:58.826478 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:44:58.826518 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:44:58.827226 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:44:58.827589 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:44:58.827975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:58.828881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:44:58.828881 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:58.828881 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:44:58.828881 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:44:58.828881 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:58.828881 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:44:58.828881 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:58.828881 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:44:58.829783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:58.829783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:44:58.829783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:44:58.829783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:44:58.829783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:44:58.829783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:44:58.829783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:44:58.829783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:58.829783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:44:58.829783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:58.829783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:44:58.829783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:58.830489 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:44:58.830489 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:58.830489 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:44:58.830489 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:58.830489 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:44:58.830489 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:44:58.830489 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:58.830489 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:44:58.830489 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:58.830489 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:44:58.830489 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:44:58.830489 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:58.830489 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:44:58.831104 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:58.831104 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:44:58.831104 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:58.831104 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:44:58.831104 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:44:58.831104 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:44:58.831104 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:44:58.831104 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:44:58.831104 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:44:58.831104 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:44:58.831104 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:44:58.831104 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:44:58.831104 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:44:58.831104 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:44:58.831104 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:44:58.831925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:44:58.831925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:44:58.831925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:44:58.831925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:44:58.831925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:44:58.831925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:58.831925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:44:58.831925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:44:58.831925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:44:58.831925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:44:58.831925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:44:58.831925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:44:58.831925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:44:58.831925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:44:58.831925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:44:58.831925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:44:58.831925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:44:58.831925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:44:58.831925 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:44:58.833276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:44:58.833276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:44:58.833276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:44:58.833276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:44:58.833276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:44:58.833276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:45:03.660272 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:45:03.660949 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:45:03.660949 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:45:03.661911 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:45:03.699477 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:45:03.699864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:03.700545 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:45:03.700545 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:03.700545 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:45:03.700545 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:45:03.700545 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:03.700545 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:45:03.700545 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:03.700545 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:45:03.701420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:03.701420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:45:03.701420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:45:03.701420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:45:03.701420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:45:03.701420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:45:03.701420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:45:03.701420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:03.701420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:45:03.701420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:03.701420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:45:03.701420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:03.701877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:45:03.701877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:03.701877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:45:03.701877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:03.701877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:45:03.701877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:45:03.701877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:03.701877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:45:03.701877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:03.701877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:45:03.701877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:45:03.701877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:03.701877 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:45:03.702280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:03.702280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:45:03.702280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:03.702280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:45:03.702280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:45:03.702280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:45:03.702280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:45:03.702280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:45:03.702280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:45:03.702280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:45:03.702280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:45:03.702280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:45:03.702280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:45:03.702280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:45:03.702280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:45:03.702786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:45:03.702786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:45:03.702786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:45:03.702786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:45:03.702786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:45:03.702786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:03.702786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:45:03.702786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:03.702786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:45:03.702786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:03.702786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:45:03.702786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:03.702786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:45:03.702786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:03.702786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:45:03.702786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:45:03.702786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:03.702786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:45:03.702786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:03.703393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:45:03.703393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:45:03.703393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:03.703393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:45:03.703393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:03.703393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:45:03.803287 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:45:03.803853 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:45:03.803950 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:45:03.805261 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:45:03.805593 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:45:03.805875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:03.807078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:45:03.807078 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:03.807078 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:45:03.807078 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:45:03.807078 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:03.807078 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:45:03.807078 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:03.807078 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:45:03.808367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:03.808367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:45:03.808367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:45:03.808367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:45:03.808367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:45:03.808367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:45:03.808367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:45:03.808367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:03.808367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:45:03.808367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:03.808367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:45:03.808367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:03.809190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:45:03.809190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:03.809190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:45:03.809190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:03.809190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:45:03.809190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:45:03.809190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:03.809190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:45:03.809190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:03.809190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:45:03.809190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:45:03.809190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:03.809190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:45:03.809669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:03.809669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:45:03.809669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:03.809669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:45:03.809669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:45:03.809669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:45:03.809669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:45:03.809669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:45:03.809669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:45:03.809669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:45:03.809669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:45:03.809669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:45:03.809669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:45:03.809669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:45:03.809669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:45:03.810264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:45:03.810264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:45:03.810264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:45:03.810264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:45:03.810264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:45:03.810264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:03.810264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:45:03.810264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:03.810264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:45:03.810264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:03.810264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:45:03.810264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:03.810264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:45:03.810264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:03.810264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:45:03.810264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:45:03.810264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:03.810264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:45:03.810264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:03.810975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:45:03.810975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:45:03.810975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:03.810975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:45:03.810975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:03.810975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:45:03.812542 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:45:03.813179 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:45:03.813543 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:45:03.813729 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:45:03.814230 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:45:03.814496 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:03.815302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:45:03.815302 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:03.815302 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:45:03.815302 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:45:03.815302 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:03.815302 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:45:03.815302 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:03.815302 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:45:03.817134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:03.817134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:45:03.817134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:45:03.817134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:45:03.817134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:03.817134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:45:03.817134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:03.817134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:45:03.817134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:03.817134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:45:03.817134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:03.817134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:45:03.817134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:03.817134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:45:03.817134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:45:03.817616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:03.817616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:45:03.817616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:03.817616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:45:03.817616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:45:03.817616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:03.817616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:45:03.817616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:03.817616 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:45:03.818169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:03.818169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:45:03.818169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:45:03.818169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:45:03.818169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:45:03.818169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:45:03.818169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:45:03.818169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:45:03.818169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:45:03.818169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:45:03.818169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:03.818169 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:45:03.818459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:03.818459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:45:03.818459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:03.818459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:45:03.818459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:03.818459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:45:03.818459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:03.818459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:45:03.818459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:45:03.818459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:03.818459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:45:03.818459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:03.818459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:45:03.818459 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:45:03.818799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:03.818799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:45:03.818799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:03.818799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:45:03.819359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:03.819359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:45:03.819359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:45:03.819359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:45:03.819359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:45:03.819359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:45:03.819359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:45:03.819359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:45:03.819359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:45:03.819359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:45:03.819359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:45:03.819359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:45:03.819359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:45:03.819359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:45:03.819359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:45:03.819359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:45:03.819764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:45:03.819764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:45:03.819764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:45:03.819764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:45:03.819764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:45:03.819764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:03.819764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:45:03.819764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:03.819764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:45:03.819764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:03.819764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:45:03.819764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:03.819764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:45:03.819764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:03.819764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:45:03.819764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:45:03.819764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:03.819764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:45:03.820140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:03.820140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:45:03.820140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:45:03.820140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:03.820140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:45:03.820140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:03.820140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:45:03.820764 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:45:03.821109 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:45:03.821230 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:45:03.821368 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:45:03.822217 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:45:03.822601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:03.823308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:45:03.823308 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:03.823308 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:45:03.823308 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:45:03.823308 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:03.823308 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:45:03.823308 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:03.823308 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:45:03.824121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:03.824121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:45:03.824121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:45:03.824121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:45:03.824121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:03.824121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:45:03.824121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:03.824121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:45:03.824121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:03.824121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:45:03.824121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:03.824121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:45:03.824121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:03.824121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:45:03.824121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:45:03.824525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:03.824525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:45:03.824525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:03.824525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:45:03.824525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:45:03.824525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:03.824525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:45:03.824525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:03.824525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:45:03.825237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:03.825237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:45:03.825237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:45:03.825237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:45:03.825237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:45:03.825237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:45:03.825237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:45:03.825237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:45:03.825237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:45:03.825237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:03.825237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:45:03.825573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:03.825573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:45:03.825573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:03.825573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:45:03.825573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:03.825573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:45:03.825573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:03.825573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:45:03.825573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:45:03.825573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:03.825573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:45:03.825573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:03.825573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:45:03.825573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:45:03.825974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:03.825974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:45:03.825974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:03.825974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:45:03.826501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:03.826501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:45:03.826501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:45:03.826501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:45:03.826501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:45:03.826501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:45:03.826501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:45:03.826501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:45:03.826501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:45:03.826501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:45:03.826501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:45:03.826501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:45:03.826501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:45:03.826501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:45:03.826501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:45:03.826501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:45:03.826501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:45:03.826969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:45:03.826969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:45:03.826969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:45:03.826969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:03.826969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:45:03.826969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:03.826969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:45:03.826969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:03.826969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:45:03.826969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:03.826969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:45:03.826969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:03.826969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:45:03.826969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:45:03.826969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:03.826969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:45:03.826969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:03.826969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:45:03.826969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:45:03.827504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:03.827504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:45:03.827504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:03.827504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:45:03.827504 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:45:03.827504 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:45:03.827504 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:45:03.827720 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:45:03.827980 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:45:03.828220 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:45:03.828382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:03.828961 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:45:03.828961 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:03.828961 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:45:03.828961 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:45:03.828961 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:03.828961 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:45:03.828961 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:03.828961 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:45:03.829846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:03.829846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:45:03.829846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:45:03.829846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:45:03.829846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:45:03.829846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:45:03.829846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:45:03.829846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:03.829846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:45:03.829846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:03.829846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:45:03.829846 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:03.830261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:45:03.830261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:03.830261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:45:03.830261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:03.830261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:45:03.830261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:45:03.830261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:03.830261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:45:03.830261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:03.830261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:45:03.830261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:45:03.830261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:03.830261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:45:03.830639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:03.830639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:45:03.830897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:03.830897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:45:03.830897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:45:03.830897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:45:03.830897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:45:03.830897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:45:03.830897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:45:03.830897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:45:03.830897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:45:03.830897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:45:03.830897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:45:03.830897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:45:03.830897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:45:03.830897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:45:03.830897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:45:03.830897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:45:03.831404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:45:03.831404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:45:03.831404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:03.831404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:45:03.831404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:03.831404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:45:03.831404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:03.831404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:45:03.831404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:03.831404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:45:03.831404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:03.831404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:45:03.831404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:45:03.831404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:03.831404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:45:03.831404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:03.831404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:45:03.831404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:45:03.831404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:03.831943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:45:03.831943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:03.831943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:45:08.661133 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:45:08.662123 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:45:08.662123 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:45:08.662927 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:45:08.697317 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:45:08.697494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:08.698017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:45:08.698017 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:08.698017 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:45:08.698017 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:45:08.698017 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:08.698017 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:45:08.698017 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:08.698017 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:45:08.698915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:08.698915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:45:08.698915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:45:08.698915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:45:08.698915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:45:08.698915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:45:08.698915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:45:08.698915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:08.698915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:45:08.698915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:08.698915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:45:08.698915 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:08.699324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:45:08.699324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:08.699324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:45:08.699324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:08.699324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:45:08.699324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:45:08.699324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:08.699324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:45:08.699324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:08.699324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:45:08.699324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:45:08.699324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:08.699324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:45:08.699643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:08.699643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:45:08.700014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:08.700014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:45:08.700014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:45:08.700014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:45:08.700014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:45:08.700014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:45:08.700014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:45:08.700014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:45:08.700014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:45:08.700014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:45:08.700014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:45:08.700014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:45:08.700014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:45:08.700014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:45:08.700014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:45:08.700014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:45:08.700391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:45:08.700391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:45:08.700391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:08.700391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:45:08.700391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:08.700391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:45:08.700391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:08.700391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:45:08.700391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:08.700391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:45:08.700391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:08.700391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:45:08.700391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:45:08.700391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:08.700391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:45:08.700391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:08.700391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:45:08.700391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:45:08.700391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:08.700824 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:45:08.700824 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:08.700824 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:45:08.804090 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:45:08.804734 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:45:08.804907 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:45:08.806105 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:45:08.806565 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:45:08.806933 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:08.808152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:45:08.808152 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:08.808152 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:45:08.808152 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:45:08.808152 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:08.808152 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:45:08.808152 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:08.808152 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:45:08.809671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:08.809671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:45:08.809671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:45:08.809671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:45:08.809671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:45:08.809671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:45:08.809671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:45:08.809671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:08.809671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:45:08.809671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:08.809671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:45:08.809671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:08.810324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:45:08.810324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:08.810324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:45:08.810324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:08.810324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:45:08.810324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:45:08.810324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:08.810324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:45:08.810324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:08.810324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:45:08.810324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:45:08.810324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:08.810324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:45:08.810959 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:08.810959 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:45:08.811094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:08.811094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:45:08.811094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:45:08.811094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:45:08.811094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:45:08.811094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:45:08.811094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:45:08.811094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:45:08.811094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:45:08.811094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:45:08.811094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:45:08.811094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:45:08.811094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:45:08.811094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:45:08.811094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:45:08.811094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:45:08.812245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:45:08.812245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:45:08.812245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:08.812245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:45:08.812245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:08.812245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:45:08.812245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:08.812245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:45:08.812245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:08.812245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:45:08.812245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:08.812245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:45:08.812245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:45:08.812245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:08.812245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:45:08.812245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:08.812245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:45:08.812245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:45:08.812245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:08.813376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:45:08.813376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:08.813376 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:45:08.813376 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:45:08.813616 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:45:08.813616 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:45:08.813801 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:45:08.814362 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:45:08.814668 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:08.815466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:45:08.815466 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:08.815466 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:45:08.815466 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:45:08.815466 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:08.815466 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:45:08.815466 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:08.815466 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:45:08.816235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:08.816235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:45:08.816235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:45:08.816235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:45:08.816235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:08.816235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:45:08.816235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:08.816235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:45:08.816235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:08.816235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:45:08.816235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:08.816235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:45:08.816235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:08.816235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:45:08.816235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:45:08.816639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:08.816639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:45:08.816639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:08.816639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:45:08.816639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:45:08.816639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:08.816639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:45:08.816639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:08.816639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:45:08.817062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:08.817062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:45:08.817062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:45:08.817062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:45:08.817062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:45:08.817062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:45:08.817062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:45:08.817062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:45:08.817062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:45:08.817062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:45:08.817062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:08.817062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:45:08.817385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:08.817385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:45:08.817385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:08.817385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:45:08.817385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:08.817385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:45:08.817385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:08.817385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:45:08.817385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:45:08.817385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:08.817385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:45:08.817385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:08.817385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:45:08.817385 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:45:08.817769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:08.817769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:45:08.817769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:08.817769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:45:08.818122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:08.818122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:45:08.818122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:45:08.818122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:45:08.818122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:45:08.818122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:45:08.818122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:45:08.818122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:45:08.818122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:45:08.818122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:45:08.818122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:45:08.818122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:45:08.818122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:45:08.818122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:45:08.818122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:45:08.818122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:45:08.818530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:45:08.818530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:45:08.818530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:45:08.818530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:45:08.818530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:45:08.818530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:08.818530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:45:08.818530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:08.818530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:45:08.818530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:08.818530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:45:08.818530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:08.818530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:45:08.818530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:08.818530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:45:08.818530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:45:08.818530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:08.818530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:45:08.819029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:08.819029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:45:08.819029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:45:08.819029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:08.819029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:45:08.819029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:08.819029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:45:08.821198 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:45:08.821508 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:45:08.821765 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:45:08.821765 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:45:08.822242 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:45:08.822522 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:08.823039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:45:08.823039 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:08.823039 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:45:08.823039 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:45:08.823039 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:08.823039 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:45:08.823039 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:08.823039 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:45:08.823758 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:08.823758 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:45:08.823758 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:45:08.823758 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:45:08.823758 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:08.823758 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:45:08.823758 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:08.823758 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:45:08.823758 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:08.823758 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:45:08.823758 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:08.823758 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:45:08.823758 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:08.823758 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:45:08.823758 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:45:08.824211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:08.824211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:45:08.824211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:08.824211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:45:08.824211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:45:08.824211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:08.824211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:45:08.824211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:08.824211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:45:08.824626 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:08.824626 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:45:08.824626 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:45:08.824626 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:45:08.824626 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:45:08.824626 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:45:08.824626 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:45:08.824626 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:45:08.824626 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:45:08.824626 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:08.824626 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:45:08.824994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:08.824994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:45:08.824994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:08.824994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:45:08.824994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:08.824994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:45:08.824994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:08.824994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:45:08.824994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:45:08.824994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:08.824994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:45:08.824994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:08.824994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:45:08.824994 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:45:08.825390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:08.825390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:45:08.825390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:08.825390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:45:08.825670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:08.825670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:45:08.825670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:45:08.825670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:45:08.825670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:45:08.825670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:45:08.825670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:45:08.825670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:45:08.825670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:45:08.825670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:45:08.825670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:45:08.825670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:45:08.825670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:45:08.825670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:45:08.825670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:45:08.825670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:45:08.825670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:45:08.826162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:45:08.826162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:45:08.826162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:45:08.826162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:08.826162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:45:08.826162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:08.826162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:45:08.826162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:08.826162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:45:08.826162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:08.826162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:45:08.826162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:08.826162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:45:08.826162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:45:08.826162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:08.826162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:45:08.826162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:08.826162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:45:08.826162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:45:08.826855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:08.826855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:45:08.826855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:08.826855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:45:08.827500 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:45:08.827825 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:45:08.827932 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:45:08.828080 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:45:08.828549 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:45:08.828983 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:45:08.829223 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:08.830050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:45:08.830050 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:08.830050 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:45:08.830050 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:45:08.830050 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:08.830050 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:45:08.830050 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:08.830050 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:45:08.831030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:08.831030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:45:08.831030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:45:08.831030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:45:08.831030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:45:08.831030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:45:08.831030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:45:08.831030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:08.831030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:45:08.831030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:08.831030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:45:08.831030 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:08.831515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:45:08.831515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:08.831515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:45:08.831515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:08.831515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:45:08.831515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:45:08.831515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:08.831515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:45:08.831515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:08.831515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:45:08.831515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:45:08.831515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:08.831515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:45:08.831944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:08.831944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:45:08.831944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:08.831944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:45:08.831944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:45:08.831944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:45:08.831944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:45:08.831944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:45:08.831944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:45:08.831944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:45:08.831944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:45:08.831944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:45:08.831944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:45:08.831944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:45:08.831944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:45:08.832511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:45:08.832511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:45:08.832511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:45:08.832511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:45:08.832511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:45:08.832511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:08.832511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:45:08.832511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:08.832511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:45:08.832511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:08.832511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:45:08.832511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:08.832511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:45:08.832511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:08.832511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:45:08.832511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:45:08.832511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:08.832511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:45:08.832511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:08.833059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:45:08.833059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:45:08.833059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:08.833059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:45:08.833059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:08.833059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:45:13.661859 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:45:13.662531 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:45:13.662531 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:45:13.663929 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:45:13.702665 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:45:13.703267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:13.704367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:45:13.704367 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:13.704367 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:45:13.704367 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:45:13.704367 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:13.704367 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:45:13.704367 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:13.704367 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:45:13.705321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:13.705321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:45:13.705321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:45:13.705321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:45:13.705321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:45:13.705321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:45:13.705321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:45:13.705321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:13.705321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:45:13.705321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:13.705321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:45:13.705321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:13.706217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:45:13.706217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:13.706217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:45:13.706217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:13.706217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:45:13.706217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:45:13.706217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:13.706217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:45:13.706217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:13.706217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:45:13.706217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:45:13.706217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:13.706217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:45:13.706917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:13.706917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:45:13.706917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:13.706917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:45:13.706917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:45:13.706917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:45:13.706917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:45:13.706917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:45:13.706917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:45:13.706917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:45:13.706917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:45:13.706917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:45:13.706917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:45:13.706917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:45:13.706917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:45:13.707736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:45:13.707736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:45:13.707736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:45:13.707736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:45:13.707736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:45:13.707736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:13.707736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:45:13.707736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:13.707736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:45:13.707736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:13.707736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:45:13.707736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:13.707736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:45:13.707736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:13.707736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:45:13.707736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:45:13.707736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:13.707736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:45:13.707736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:13.709236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:45:13.709236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:45:13.709236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:13.709236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:45:13.709236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:13.709236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:45:13.804837 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:45:13.805560 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:45:13.805824 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:45:13.806665 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:45:13.806977 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:45:13.807176 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:13.807963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:45:13.807963 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:13.807963 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:45:13.807963 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:45:13.807963 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:13.807963 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:45:13.807963 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:13.807963 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:45:13.809758 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:13.809758 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:45:13.809758 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:45:13.809758 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:45:13.809758 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:45:13.809758 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:45:13.809758 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:45:13.809758 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:13.809758 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:45:13.809758 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:13.809758 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:45:13.809758 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:13.810123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:45:13.810123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:13.810123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:45:13.810123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:13.810123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:45:13.810123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:45:13.810123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:13.810123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:45:13.810123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:13.810123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:45:13.810123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:45:13.810123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:13.810123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:45:13.810465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:13.810465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:45:13.811693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:13.811693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:45:13.811693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:45:13.811693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:45:13.811693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:45:13.811693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:45:13.811693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:45:13.811693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:45:13.811693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:45:13.811693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:45:13.811693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:45:13.811693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:45:13.811693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:45:13.811693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:45:13.811693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:45:13.811693 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:45:13.812563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:45:13.812563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:45:13.812563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:13.812563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:45:13.812563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:13.812563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:45:13.812563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:13.812563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:45:13.812563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:13.812563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:45:13.812563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:13.812563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:45:13.812563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:45:13.812563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:13.812563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:45:13.812563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:13.812563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:45:13.812563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:45:13.812563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:13.813708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:45:13.813708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:13.813708 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:45:13.813708 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:45:13.814106 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:45:13.814286 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:45:13.814413 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:45:13.814972 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:45:13.815386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:13.816047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:45:13.816047 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:13.816047 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:45:13.816047 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:45:13.816047 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:13.816047 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:45:13.816047 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:13.816047 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:45:13.816878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:13.816878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:45:13.816878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:45:13.816878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:45:13.816878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:13.816878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:45:13.816878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:13.816878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:45:13.816878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:13.816878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:45:13.816878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:13.816878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:45:13.816878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:13.816878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:45:13.816878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:45:13.817410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:13.817410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:45:13.817410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:13.817410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:45:13.817410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:45:13.817410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:13.817410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:45:13.817410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:13.817410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:45:13.817967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:13.817967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:45:13.817967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:45:13.817967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:45:13.817967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:45:13.817967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:45:13.817967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:45:13.817967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:45:13.817967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:45:13.817967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:45:13.817967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:13.817967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:45:13.818365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:13.818365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:45:13.818365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:13.818365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:45:13.818365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:13.818365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:45:13.818365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:13.818365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:45:13.818365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:45:13.818365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:13.818365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:45:13.818365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:13.818365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:45:13.818365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:45:13.818758 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:13.818758 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:45:13.818758 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:13.818758 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:45:13.819401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:13.819401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:45:13.819401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:45:13.819401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:45:13.819401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:45:13.819401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:45:13.819401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:45:13.819401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:45:13.819401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:45:13.819401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:45:13.819401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:45:13.819401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:45:13.819401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:45:13.819401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:45:13.819401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:45:13.819401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:45:13.820114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:45:13.820114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:45:13.820114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:45:13.820114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:45:13.820114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:45:13.820114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:13.820114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:45:13.820114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:13.820114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:45:13.820114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:13.820114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:45:13.820114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:13.820114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:45:13.820114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:13.820114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:45:13.820114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:45:13.820114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:13.820114 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:45:13.820882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:13.820882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:45:13.820882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:45:13.820882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:13.820882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:45:13.820882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:13.820882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:45:13.821639 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:45:13.821968 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:45:13.822088 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:45:13.822228 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:45:13.822538 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:45:13.822757 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:13.823289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:45:13.823289 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:13.823289 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:45:13.823289 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:45:13.823289 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:13.823289 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:45:13.823289 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:13.823289 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:45:13.824225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:13.824225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:45:13.824225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:45:13.824225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:45:13.824225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:13.824225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:45:13.824225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:13.824225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:45:13.824225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:13.824225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:45:13.824225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:13.824225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:45:13.824225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:13.824225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:45:13.824225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:45:13.824876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:13.824876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:45:13.824876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:13.824876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:45:13.824876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:45:13.824876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:13.824876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:45:13.824876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:13.824876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:45:13.825228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:13.825228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:45:13.825228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:45:13.825228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:45:13.825228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:45:13.825228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:45:13.825228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:45:13.825228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:45:13.825228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:45:13.825228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:13.825228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:45:13.825684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:13.825684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:45:13.825684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:13.825684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:45:13.825684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:13.825684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:45:13.825684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:13.825684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:45:13.825684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:45:13.825684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:13.825684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:45:13.825684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:13.825684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:45:13.825684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:45:13.826260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:13.826260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:45:13.826260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:13.826260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:45:13.826260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:13.826260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:45:13.826260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:45:13.826260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:45:13.826260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:45:13.826260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:45:13.826260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:45:13.826260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:45:13.826260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:45:13.826260 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:45:13.826804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:45:13.826804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:45:13.826804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:45:13.826804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:45:13.826804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:45:13.826804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:45:13.826804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:45:13.826804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:45:13.826804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:45:13.826804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:45:13.826804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:13.826804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:45:13.826804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:13.826804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:45:13.826804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:13.826804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:45:13.826804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:13.826804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:45:13.826804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:13.827551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:45:13.827551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:45:13.827551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:13.827551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:45:13.827551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:13.827551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:45:13.827551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:45:13.827551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:13.827551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:45:13.827551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:13.827551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:45:13.828112 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:45:13.828423 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:45:13.828590 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:45:13.828757 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:45:13.829272 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:45:13.829549 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:45:13.829771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:13.830299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:45:13.830299 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:13.830299 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:45:13.830299 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:45:13.830299 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:13.830299 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:45:13.830299 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:13.830299 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:45:13.831146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:13.831146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:45:13.831146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:45:13.831146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:45:13.831146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:45:13.831146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:45:13.831146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:45:13.831146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:13.831146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:45:13.831146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:13.831146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:45:13.831146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:13.831544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:45:13.831544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:13.831544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:45:13.831544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:13.831544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:45:13.831544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:45:13.831544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:13.831544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:45:13.831544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:13.831544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:45:13.831544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:45:13.831544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:13.831544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:45:13.831941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:13.831941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:45:13.832140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:13.832140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:45:13.832140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:45:13.832140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:45:13.832140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:45:13.832140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:45:13.832140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:45:13.832140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:45:13.832140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:45:13.832140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:45:13.832140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:45:13.832140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:45:13.832140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:45:13.832140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:45:13.832140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:45:13.832140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:45:13.832670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:45:13.832670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:45:13.832670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:13.832670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:45:13.832670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:13.832670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:45:13.832670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:13.832670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:45:13.832670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:13.832670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:45:13.832670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:13.832670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:45:13.832670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:45:13.832670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:13.832670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:45:13.832670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:13.832670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:45:13.832670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:45:13.832670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:13.833198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:45:13.833198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:13.833198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:45:18.662630 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:45:18.663360 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:45:18.663434 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:45:18.664913 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:45:18.705214 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:45:18.705480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:18.706341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:45:18.706341 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:18.706341 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:45:18.706341 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:45:18.706341 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:18.706341 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:45:18.706341 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:18.706341 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:45:18.707062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.043 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:18.707062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:45:18.707062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:45:18.707062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:45:18.707062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:45:18.707062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:45:18.707062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:45:18.707062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:18.707062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:45:18.707062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:18.707062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:45:18.707062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:18.708359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:45:18.708359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:18.708359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:45:18.708359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:18.708359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:45:18.708359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:45:18.708359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:18.708359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:45:18.708359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:18.708359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:45:18.708359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:45:18.708359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:18.708359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:45:18.708726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:18.708726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:45:18.708726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:18.708726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:45:18.708726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:45:18.708726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:45:18.708726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:45:18.708726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:45:18.708726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:45:18.708726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:45:18.708726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:45:18.708726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:45:18.708726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:45:18.708726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:45:18.708726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:45:18.709147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:45:18.709147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:45:18.709147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:45:18.709147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:45:18.709147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:45:18.709147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:18.709147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:45:18.709147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:18.709147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:45:18.709147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:18.709147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:45:18.709147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:18.709147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:45:18.709147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:18.709147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:45:18.709147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:45:18.709147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:18.709147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:45:18.709147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:18.709574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:45:18.709574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:45:18.709574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:18.709574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:45:18.709574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:18.709574 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:45:18.805518 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:45:18.805832 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:45:18.806288 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:45:18.807221 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:45:18.808977 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:45:18.808977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:18.808977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:45:18.808977 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:18.808977 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:45:18.808977 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:45:18.808977 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:18.808977 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:45:18.808977 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:18.808977 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:45:18.811000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:18.811000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:45:18.811000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:45:18.811000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:45:18.811000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:45:18.811000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:45:18.811000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:45:18.811000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:18.811000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:45:18.811000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:18.811000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:45:18.811000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:18.811501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:45:18.811501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:18.811501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:45:18.811501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:18.811501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:45:18.811501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:45:18.811501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:18.811501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:45:18.811501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:18.811501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:45:18.811501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:45:18.811501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:18.811501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:45:18.812361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:18.812361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:45:18.813052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:18.813052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:45:18.813052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:45:18.813052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:45:18.813052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:45:18.813052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:45:18.813052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:45:18.813052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:45:18.813052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:45:18.813052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:45:18.813052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:45:18.813052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:45:18.813052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:45:18.813052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:45:18.813052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:45:18.813052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:45:18.813649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:45:18.813649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:45:18.813649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:18.813649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:45:18.813649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:18.813649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:45:18.813649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:18.813649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:45:18.813649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:18.813649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:45:18.813649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:18.813649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:45:18.813649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:45:18.813649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:18.813649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:45:18.813649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:18.813649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:45:18.813649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:45:18.813649 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:18.814374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:45:18.814374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:18.814374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:45:18.814374 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:45:18.814374 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:45:18.814597 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:45:18.814597 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:45:18.815421 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:45:18.815858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:18.816558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:45:18.816558 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:18.816558 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:45:18.816558 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:45:18.816558 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:18.816558 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:45:18.816558 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:18.816558 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:45:18.817347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:18.817347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:45:18.817347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:45:18.817347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:45:18.817347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:18.817347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:45:18.817347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:18.817347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:45:18.817347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:18.817347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:45:18.817347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:18.817347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:45:18.817347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:18.817347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:45:18.817347 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:45:18.818048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:18.818048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:45:18.818048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:18.818048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:45:18.818048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:45:18.818048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:18.818048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:45:18.818048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:18.818048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:45:18.818435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:18.818435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:45:18.818435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:45:18.818435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:45:18.818435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:45:18.818435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:45:18.818435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:45:18.818435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:45:18.818435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:45:18.818435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:45:18.818435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:18.818435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:45:18.819022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:18.819022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:45:18.819022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:18.819022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:45:18.819022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:18.819022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:45:18.819022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:18.819022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:45:18.819022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:45:18.819022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:18.819022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:45:18.819022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:18.819022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:45:18.819022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:45:18.819526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:18.819526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:45:18.819526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:18.819526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:45:18.819526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:18.819526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:45:18.819526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:45:18.819526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:45:18.819526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:45:18.819526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:45:18.819526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:45:18.819526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:45:18.819526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:45:18.819526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:45:18.820067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:45:18.820067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:45:18.820067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:45:18.820067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:45:18.820067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:45:18.820067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:45:18.820067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:45:18.820067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:45:18.820067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:45:18.820067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:45:18.820067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:45:18.820067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:18.820067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:45:18.820067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:18.820067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:45:18.820067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:18.820067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:45:18.820067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:18.820067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:45:18.821112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:18.821112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:45:18.821112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:45:18.821112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:18.821112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:45:18.821112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:18.821112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:45:18.821112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:45:18.821112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:18.821112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:45:18.821112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:18.821112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:45:18.822008 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:45:18.822498 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:45:18.822498 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:45:18.822498 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:45:18.822944 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:45:18.823279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:18.824008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:45:18.824008 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:18.824008 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:45:18.824008 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:45:18.824008 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:18.824008 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:45:18.824008 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:18.824008 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:45:18.824959 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:18.824959 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:45:18.824959 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:45:18.824959 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:45:18.824959 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:18.824959 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:45:18.824959 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:18.824959 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:45:18.824959 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:18.824959 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:45:18.824959 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:18.824959 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:45:18.824959 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:18.824959 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:45:18.824959 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:45:18.825359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:18.825359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:45:18.825359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:18.825359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:45:18.825359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:45:18.825359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:18.825359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:45:18.825359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:18.825359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:45:18.825767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:18.825767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:45:18.825767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:45:18.825767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:45:18.825767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:45:18.825767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:45:18.825767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:45:18.825767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:45:18.825767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:45:18.825767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:18.825767 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:45:18.826225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:18.826225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:45:18.826225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:18.826225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:45:18.826225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:18.826225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:45:18.826225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:18.826225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:45:18.826225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:45:18.826225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:18.826225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:45:18.826225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:18.826225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:45:18.826225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:45:18.826588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:18.826588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:45:18.826588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:18.826588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:45:18.826765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:18.826765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:45:18.826765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:45:18.826765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:45:18.826765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:45:18.826765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:45:18.826765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:45:18.826765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:45:18.826765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:45:18.826765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:45:18.826765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:45:18.826765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:45:18.826765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:45:18.826765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:45:18.826765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:45:18.826765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:45:18.826765 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:45:18.827371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:45:18.827371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:45:18.827371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:45:18.827371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:18.827371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:45:18.827371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:18.827371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:45:18.827371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:18.827371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:45:18.827371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:18.827371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:45:18.827371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:18.827371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:45:18.827371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:45:18.827371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:18.827371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:45:18.827371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:18.827371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:45:18.827371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:45:18.828001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:18.828001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:45:18.828001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:18.828001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:45:18.828986 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:45:18.828986 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:45:18.828986 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:45:18.829156 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:45:18.829485 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:45:18.831526 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:45:18.831709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:18.832298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:45:18.832298 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:18.832298 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:45:18.832298 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:45:18.832298 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:18.832298 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:45:18.832298 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:18.832298 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:45:18.833138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:18.833138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:45:18.833138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:45:18.833138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:45:18.833138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:45:18.833138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:45:18.833138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:45:18.833138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:18.833138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:45:18.833138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:18.833138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:45:18.833138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:18.833623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:45:18.833623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:18.833623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:45:18.833623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:18.833623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:45:18.833623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:45:18.833623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:18.833623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:45:18.833623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:18.833623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:45:18.833623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:45:18.833623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:18.833623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:45:18.834115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:18.834115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:45:18.834115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:18.834115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:45:18.834115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:45:18.834115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:45:18.834115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:45:18.834115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:45:18.834115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:45:18.834115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:45:18.834115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:45:18.834115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:45:18.834115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:45:18.834115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:45:18.834115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:45:18.834662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:45:18.834662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:45:18.834662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:45:18.834662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:45:18.834662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:45:18.834662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:18.834662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:45:18.834662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:18.834662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:45:18.834662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:18.834662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:45:18.834662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:18.834662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:45:18.834662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:18.834662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:45:18.834662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:45:18.834662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:18.834662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:45:18.834662 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:18.835355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:45:18.835355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:45:18.835355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:18.835355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:45:18.835355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:18.835355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:45:23.663480 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:45:23.664326 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:45:23.664326 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:45:23.665190 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:45:23.703401 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:45:23.703624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:23.705292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:45:23.705292 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:23.705292 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:45:23.705292 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:45:23.705292 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:23.705292 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:45:23.705292 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:23.705292 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:45:23.705598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:23.705598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:45:23.705598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:45:23.705598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:45:23.705598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:45:23.705598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:45:23.705598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:45:23.705598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:23.705598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:45:23.705598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:23.705598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:45:23.705598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:23.706107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:45:23.706107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:23.706107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:45:23.706107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:23.706107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:45:23.706107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:45:23.706107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:23.706107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:45:23.706107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:23.706107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:45:23.706107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:45:23.706107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:23.706107 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:45:23.706499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:23.706499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:45:23.706499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:23.706499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:45:23.706499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:45:23.706499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:45:23.706499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:45:23.706499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:45:23.706499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:45:23.706499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:45:23.706499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:45:23.706499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:45:23.706499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:45:23.706499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:45:23.706499 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:45:23.706998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:45:23.706998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:45:23.706998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:45:23.706998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:45:23.706998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:45:23.706998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:23.706998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:45:23.706998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:23.706998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:45:23.706998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:23.706998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:45:23.706998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:23.706998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:45:23.706998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:23.706998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:45:23.706998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:45:23.706998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:23.706998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:45:23.706998 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:23.707566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:45:23.707566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:45:23.707566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:23.707566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:45:23.707566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:23.707566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:45:23.806141 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:45:23.806992 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:45:23.807185 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:45:23.808404 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:45:23.808757 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:45:23.809021 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:23.810094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:45:23.810094 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:23.810094 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:45:23.810094 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:45:23.810094 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:23.810094 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:45:23.810094 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:23.810094 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:45:23.811922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:23.811922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:45:23.811922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:45:23.811922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:45:23.811922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:45:23.811922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:45:23.811922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:45:23.811922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:23.811922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:45:23.811922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:23.811922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:45:23.811922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:23.812340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:45:23.812340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:23.812340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:45:23.812340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:23.812340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:45:23.812340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:45:23.812340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:23.812340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:45:23.812340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:23.812340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:45:23.812340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:45:23.812340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:23.812340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:45:23.812735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:23.812735 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:45:23.814716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:23.814716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:45:23.814716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:45:23.814716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:45:23.814716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:45:23.814716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:45:23.814716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:45:23.814716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:45:23.814716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:45:23.814716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:45:23.814716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:45:23.814716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:45:23.814716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:45:23.814716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:45:23.814716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:45:23.814716 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:45:23.815302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:45:23.815302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:45:23.815302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:23.815302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:45:23.815302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:23.815302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:45:23.815302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:23.815302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:45:23.815302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:23.815302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:45:23.815302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:23.815302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:45:23.815302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:45:23.815302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:23.815302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:45:23.815302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:23.815302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:45:23.815302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:45:23.815302 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:23.815817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:45:23.815817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:23.815817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:45:23.815817 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:45:23.815817 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:45:23.815817 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:45:23.815817 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:45:23.816196 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:45:23.816395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:23.816943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:45:23.816943 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:23.816943 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:45:23.816943 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:45:23.816943 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:23.816943 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:45:23.816943 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:23.816943 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:45:23.817835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:23.817835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:45:23.817835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:45:23.817835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:45:23.817835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:23.817835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:45:23.817835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:23.817835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:45:23.817835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:23.817835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:45:23.817835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:23.817835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:45:23.817835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:23.817835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:45:23.817835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:45:23.818289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:23.818289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:45:23.818289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:23.818289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:45:23.818289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:45:23.818289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:23.818289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:45:23.818289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:23.818289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:45:23.818726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:23.818726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:45:23.818726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:45:23.818726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:45:23.818726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:45:23.818726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:45:23.818726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:45:23.818726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:45:23.818726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:45:23.818726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:45:23.818726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:23.818726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:45:23.819163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:23.819163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:45:23.819163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:23.819163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:45:23.819163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:23.819163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:45:23.819163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:23.819163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:45:23.819163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:45:23.819163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:23.819163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:45:23.819163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:23.819163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:45:23.819163 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:45:23.819558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:23.819558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:45:23.819558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:23.819558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:45:23.819930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:23.819930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:45:23.819930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:45:23.819930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:45:23.819930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:45:23.819930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:45:23.819930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:45:23.819930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:45:23.819930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:45:23.819930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:45:23.819930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:45:23.819930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:45:23.819930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:45:23.819930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:45:23.819930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:45:23.819930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:45:23.820530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:45:23.820530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:45:23.820530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:45:23.820530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:45:23.820530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:45:23.820530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:23.820530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:45:23.820530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:23.820530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:45:23.820530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:23.820530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:45:23.820530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:23.820530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:45:23.820530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:23.820530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:45:23.820530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:45:23.820530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:23.820530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:45:23.821121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:23.821121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:45:23.821121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:45:23.821121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:23.821121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:45:23.821121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:23.821121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:45:23.822414 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:45:23.822691 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:45:23.822845 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:45:23.822982 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:45:23.823424 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:45:23.823794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:23.824554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:45:23.824554 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:23.824554 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:45:23.824554 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:45:23.824554 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:23.824554 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:45:23.824554 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:23.824554 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:45:23.825325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:23.825325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:45:23.825325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:45:23.825325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:45:23.825325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:23.825325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:45:23.825325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:23.825325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:45:23.825325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:23.825325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:45:23.825325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:23.825325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:45:23.825325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:23.825325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:45:23.825325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:45:23.825802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:23.825802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:45:23.825802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:23.825802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:45:23.825802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:45:23.825802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:23.825802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:45:23.825802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:23.825802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:45:23.826363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:23.826363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:45:23.826363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:45:23.826363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:45:23.826363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:45:23.826363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:45:23.826363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:45:23.826363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:45:23.826363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:45:23.826363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:23.826363 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:45:23.826723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:23.826723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:45:23.826723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:23.826723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:45:23.826723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:23.826723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:45:23.826723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:23.826723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:45:23.826723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:45:23.826723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:23.826723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:45:23.826723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:23.826723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:45:23.826723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:45:23.827118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:23.827118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:45:23.827118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:23.827118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:45:23.827447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:23.827447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:45:23.827447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:45:23.827447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:45:23.827447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:45:23.827447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:45:23.827447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:45:23.827447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:45:23.827447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:45:23.827447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:45:23.827447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:45:23.827447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:45:23.827447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:45:23.827447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:45:23.827447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:45:23.827447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:45:23.827447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:45:23.827967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:45:23.827967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:45:23.827967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:45:23.827967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:23.827967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:45:23.827967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:23.827967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:45:23.827967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:23.827967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:45:23.827967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:23.827967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:45:23.827967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:23.827967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:45:23.827967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:45:23.827967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:23.827967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:45:23.827967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:23.827967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:45:23.827967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:45:23.828568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:23.828568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:45:23.828568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:23.828568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:45:23.829137 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:45:23.829445 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:45:23.829563 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:45:23.829758 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:45:23.830199 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:45:23.830539 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:45:23.830763 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:23.831511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:45:23.831511 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:23.831511 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:45:23.831511 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:45:23.831511 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:23.831511 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:45:23.831511 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:23.831511 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:45:23.832395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:23.832395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:45:23.832395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:45:23.832395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:45:23.832395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:45:23.832395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:45:23.832395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:45:23.832395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:23.832395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:45:23.832395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:23.832395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:45:23.832395 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:23.832790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:45:23.832790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:23.832790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:45:23.832790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:23.832790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:45:23.832790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:45:23.832790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:23.832790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:45:23.832790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:23.832790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:45:23.832790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:45:23.832790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:23.832790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:45:23.833129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:23.833129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:45:23.833407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:23.833407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:45:23.833407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:45:23.833407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:45:23.833407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:45:23.833407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:45:23.833407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:45:23.833407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:45:23.833407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:45:23.833407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:45:23.833407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:45:23.833407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:45:23.833407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:45:23.833407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:45:23.833407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:45:23.833407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:45:23.833893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:45:23.833893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:45:23.833893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:23.833893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:45:23.833893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:23.833893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:45:23.833893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:23.833893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:45:23.833893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:23.833893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:45:23.833893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:23.833893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:45:23.833893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:45:23.833893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:23.833893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:45:23.833893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:23.833893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:45:23.833893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:45:23.833893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:23.834411 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:45:23.834411 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:23.834411 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:45:28.664079 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:45:28.664919 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:45:28.664919 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:45:28.666066 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:45:28.703734 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:45:28.703734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:28.703734 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:45:28.703734 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:28.703734 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:45:28.703734 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:45:28.703734 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:28.703734 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:45:28.703734 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:28.703734 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:45:28.704707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:28.704707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:45:28.704707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:45:28.704707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:45:28.704707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:45:28.704707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:45:28.704707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:45:28.704707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:28.704707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:45:28.704707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:28.704707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:45:28.704707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:28.705045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:45:28.705045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:28.705045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:45:28.705045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:28.705045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:45:28.705045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:45:28.705045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:28.705045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:45:28.705045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:28.705045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:45:28.705045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:45:28.705045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:28.705045 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:45:28.705316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:28.705316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:45:28.705802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:28.705802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:45:28.705802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:45:28.705802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:45:28.705802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:45:28.705802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:45:28.705802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:45:28.705802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:45:28.705802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:45:28.705802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:45:28.705802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:45:28.705802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:45:28.705802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:45:28.705802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:45:28.705802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:45:28.705802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:45:28.706157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:45:28.706157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:45:28.706157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:28.706157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:45:28.706157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:28.706157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:45:28.706157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:28.706157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:45:28.706157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:28.706157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:45:28.706157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:28.706157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:45:28.706157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:45:28.706157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:28.706157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:45:28.706157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:28.706157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:45:28.706157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:45:28.706157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:28.706521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:45:28.706521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:28.706521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:45:28.807002 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:45:28.807502 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:45:28.807685 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:45:28.808636 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:45:28.808892 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:45:28.809330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:28.810133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:45:28.810133 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:28.810133 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:45:28.810133 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:45:28.810133 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:28.810133 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:45:28.810133 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:28.810133 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:45:28.811259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:28.811259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:45:28.811259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:45:28.811259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:45:28.811259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:45:28.811259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:45:28.811259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:45:28.811259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:28.811259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:45:28.811259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:28.811259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:45:28.811259 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:28.811878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:45:28.811878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:28.811878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:45:28.811878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:28.811878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:45:28.811878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:45:28.811878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:28.811878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:45:28.811878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:28.811878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:45:28.811878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:45:28.811878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:28.811878 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:45:28.812673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:28.812673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:45:28.812673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:28.812673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:45:28.812673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:45:28.812673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:45:28.812673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:45:28.812673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:45:28.812673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:45:28.812673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:45:28.812673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:45:28.812673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:45:28.812673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:45:28.812673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:45:28.812673 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:45:28.813382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:45:28.813382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:45:28.813382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:45:28.813382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:45:28.813382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:45:28.813382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:28.813382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:45:28.813382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:28.813382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:45:28.813382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:28.813382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:45:28.813382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:28.813382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:45:28.813382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:28.813382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:45:28.813382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:45:28.813382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:28.813382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:45:28.813382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:28.814261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:45:28.814261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:45:28.814261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:28.814261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:45:28.814261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:28.814261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:45:28.815632 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:45:28.815915 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:45:28.816032 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:45:28.816157 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:45:28.816551 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:45:28.816982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:28.817812 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:45:28.817812 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:28.817812 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:45:28.817812 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:45:28.817812 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:28.817812 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:45:28.817812 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:28.817812 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:45:28.818686 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:28.818686 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:45:28.818686 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:45:28.818686 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:45:28.818686 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:28.818686 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:45:28.818686 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:28.818686 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:45:28.818686 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:28.818686 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:45:28.818686 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:28.818686 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:45:28.818686 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:28.818686 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:45:28.818686 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:45:28.819143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:28.819143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:45:28.819143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:28.819143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:45:28.819143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:45:28.819143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:28.819143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:45:28.819143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:28.819143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:45:28.819607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:28.819607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:45:28.819607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:45:28.819607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:45:28.819607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:45:28.819607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:45:28.819607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:45:28.819607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:45:28.819607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:45:28.819607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:45:28.819607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:28.819607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:45:28.820081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:28.820081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:45:28.820081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:28.820081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:45:28.820081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:28.820081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:45:28.820081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:28.820081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:45:28.820081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:45:28.820081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:28.820081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:45:28.820081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:28.820081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:45:28.820081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:45:28.820628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:28.820628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:45:28.820628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:28.820628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:45:28.820776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:28.820776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:45:28.820776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:45:28.820776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:45:28.820776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:45:28.820776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:45:28.820776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:45:28.820776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:45:28.820776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:45:28.820776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:45:28.820776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:45:28.820776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:45:28.820776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:45:28.820776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:45:28.820776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:45:28.820776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:45:28.821234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:45:28.821234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:45:28.821234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:45:28.821234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:45:28.821234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:45:28.821234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:28.821234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:45:28.821234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:28.821234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:45:28.821234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:28.821234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:45:28.821234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:28.821234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:45:28.821234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:28.821234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:45:28.821234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:45:28.821234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:28.821234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:45:28.821945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:28.821945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:45:28.821945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:45:28.821945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:28.821945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:45:28.821945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:28.821945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:45:28.822761 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:45:28.823000 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:45:28.823194 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:45:28.823419 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:45:28.823906 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:45:28.824166 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:28.824827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:45:28.824827 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:28.824827 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:45:28.824827 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:45:28.824827 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:28.824827 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:45:28.824827 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:28.824827 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:45:28.825822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:28.825822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:45:28.825822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:45:28.825822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:45:28.825822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:28.825822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:45:28.825822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:28.825822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:45:28.825822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:28.825822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:45:28.825822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:28.825822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:45:28.825822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:28.825822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:45:28.825822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:45:28.826233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:28.826233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:45:28.826233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:28.826233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:45:28.826233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:45:28.826233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:28.826233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:45:28.826233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:28.826233 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:45:28.826884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:28.826884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:45:28.826884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:45:28.826884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:45:28.826884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:45:28.826884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:45:28.826884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:45:28.826884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:45:28.826884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:45:28.826884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:28.826884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:45:28.827266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:28.827266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:45:28.827266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:28.827266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:45:28.827266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:28.827266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:45:28.827266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:28.827266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:45:28.827266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:45:28.827266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:28.827266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:45:28.827266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:28.827266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:45:28.827266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:45:28.827678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:28.827678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:45:28.827678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:28.827678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:45:28.828057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:28.828057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:45:28.828057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:45:28.828057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:45:28.828057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:45:28.828057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:45:28.828057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:45:28.828057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:45:28.828057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:45:28.828057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:45:28.828057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:45:28.828057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:45:28.828057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:45:28.828057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:45:28.828057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:45:28.828057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:45:28.828057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:45:28.828553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:45:28.828553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:45:28.828553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:45:28.828553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:28.828553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:45:28.828553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:28.828553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:45:28.828553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:28.828553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:45:28.828553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:28.828553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:45:28.828553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:28.828553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:45:28.828553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:45:28.828553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:28.828553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:45:28.828553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:28.828553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:45:28.828553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:45:28.829103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:28.829103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:45:28.829103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:28.829103 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:45:28.829771 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:45:28.830118 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:45:28.830237 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:45:28.830400 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:45:28.830867 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:45:28.831211 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:45:28.831416 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:28.832146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:45:28.832146 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:28.832146 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:45:28.832146 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:45:28.832146 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:28.832146 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:45:28.832146 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:28.832146 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:45:28.833060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:28.833060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:45:28.833060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:45:28.833060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:45:28.833060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:45:28.833060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:45:28.833060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:45:28.833060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:28.833060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:45:28.833060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:28.833060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:45:28.833060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:28.833512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:45:28.833512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:28.833512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:45:28.833512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:28.833512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:45:28.833512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:45:28.833512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:28.833512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:45:28.833512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:28.833512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:45:28.833512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:45:28.833512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:28.833512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:45:28.833950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:28.833950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:45:28.834263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:28.834263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:45:28.834263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:45:28.834263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:45:28.834263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:45:28.834263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:45:28.834263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:45:28.834263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:45:28.834263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:45:28.834263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:45:28.834263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:45:28.834263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:45:28.834263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:45:28.834263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:45:28.834263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:45:28.834263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:45:28.834768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:45:28.834768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:45:28.834768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:28.834768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:45:28.834768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:28.834768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:45:28.834768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:28.834768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:45:28.834768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:28.834768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:45:28.834768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:28.834768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:45:28.834768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:45:28.834768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:28.834768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:45:28.834768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:28.834768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:45:28.834768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:45:28.834768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:28.835384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:45:28.835384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:28.835384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:45:33.664903 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:45:33.666020 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:45:33.666020 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:45:33.666317 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:45:33.711397 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:45:33.711698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:33.712549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:45:33.712549 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:33.712549 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:45:33.712549 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:45:33.712549 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:33.712549 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:45:33.712549 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:33.712549 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:45:33.713684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.047 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:33.713684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:45:33.713684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:45:33.713684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:45:33.713684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:45:33.713684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:45:33.713684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:45:33.713684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:33.713684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:45:33.713684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:33.713684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:45:33.713684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:33.714432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:45:33.714432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:33.714432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:45:33.714432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:33.714432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:45:33.714432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:45:33.714432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:33.714432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:45:33.714432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:33.714432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:45:33.714432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:45:33.714432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:33.714432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:45:33.715135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:33.715135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:45:33.715135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:33.715135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:45:33.715135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:45:33.715135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:45:33.715135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:45:33.715135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:45:33.715135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:45:33.715135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:45:33.715135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:45:33.715135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:45:33.715135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:45:33.715135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:45:33.715135 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:45:33.715952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:45:33.715952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:45:33.715952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:45:33.715952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:45:33.715952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:45:33.715952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:33.715952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:45:33.715952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:33.715952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:45:33.715952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:33.715952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:45:33.715952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:33.715952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:45:33.715952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:33.715952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:45:33.715952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:45:33.715952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:33.715952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:45:33.715952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:33.716975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:45:33.716975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:45:33.716975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:33.716975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:45:33.716975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:33.716975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:45:33.808231 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:45:33.808867 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:45:33.808982 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:45:33.810017 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:45:33.810276 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:45:33.810450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:33.811258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:45:33.811258 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:33.811258 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:45:33.811258 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:45:33.811258 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:33.811258 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:45:33.811258 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:33.811258 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:45:33.813040 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:33.813040 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:45:33.813040 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:45:33.813040 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:45:33.813040 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:45:33.813040 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:45:33.813040 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:45:33.813040 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:33.813040 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:45:33.813040 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:33.813040 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:45:33.813040 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:33.813448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:45:33.813448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:33.813448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:45:33.813448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:33.813448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:45:33.813448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:45:33.813448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:33.813448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:45:33.813448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:33.813448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:45:33.813448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:45:33.813448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:33.813448 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:45:33.813855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:33.813855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:45:33.815776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:33.815776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:45:33.815776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:45:33.815776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:45:33.815776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:45:33.815776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:45:33.815776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:45:33.815776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:45:33.815776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:45:33.815776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:45:33.815776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:45:33.815776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:45:33.815776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:45:33.815776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:45:33.815776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:45:33.815776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:45:33.816290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:45:33.816290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:45:33.816290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:33.816290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:45:33.816290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:33.816290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:45:33.816290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:33.816290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:45:33.816290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:33.816290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:45:33.816290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:33.816290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:45:33.816290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:45:33.816290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:33.816290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:45:33.816290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:33.816290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:45:33.816290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:45:33.816290 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:33.816978 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:45:33.816978 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:33.816978 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:45:33.816978 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:45:33.816978 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:45:33.816978 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:45:33.816978 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:45:33.816978 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:45:33.817283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:33.817780 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:45:33.817780 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:33.817780 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:45:33.817780 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:45:33.817780 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:33.817780 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:45:33.817780 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:33.817780 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:45:33.818586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:33.818586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:45:33.818586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:45:33.818586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:45:33.818586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:33.818586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:45:33.818586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:33.818586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:45:33.818586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:33.818586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:45:33.818586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:33.818586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:45:33.818586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:33.818586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:45:33.818586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:45:33.818949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:33.818949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:45:33.818949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:33.818949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:45:33.818949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:45:33.818949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:33.818949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:45:33.818949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:33.818949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:45:33.819511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:33.819511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:45:33.819511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:45:33.819511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:45:33.819511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:45:33.819511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:45:33.819511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:45:33.819511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:45:33.819511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:45:33.819511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:45:33.819511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:33.819511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:45:33.819787 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:33.819787 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:45:33.819787 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:33.819787 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:45:33.819787 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:33.819787 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:45:33.819787 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:33.819787 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:45:33.819787 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:45:33.819787 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:33.819787 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:45:33.819787 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:33.819787 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:45:33.819787 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:45:33.820075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:33.820075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:45:33.820075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:33.820075 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:45:33.820679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:33.820679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:45:33.820679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:45:33.820679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:45:33.820679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:45:33.820679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:45:33.820679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:45:33.820679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:45:33.820679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:45:33.820679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:45:33.820679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:45:33.820679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:45:33.820679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:45:33.820679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:45:33.820679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:45:33.820679 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:45:33.821017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:45:33.821017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:45:33.821017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:45:33.821017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:45:33.821017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:45:33.821017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:33.821017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:45:33.821017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:33.821017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:45:33.821017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:33.821017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:45:33.821017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:33.821017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:45:33.821017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:33.821017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:45:33.821017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:45:33.821017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:33.821017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:45:33.821393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:33.821393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:45:33.821393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:45:33.821393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:33.821393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:45:33.821393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:33.821393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:45:33.823296 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:45:33.823592 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:45:33.823835 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:45:33.823975 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:45:33.824346 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:45:33.824586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:33.825154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:45:33.825154 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:33.825154 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:45:33.825154 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:45:33.825154 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:33.825154 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:45:33.825154 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:33.825154 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:45:33.826047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:33.826047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:45:33.826047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:45:33.826047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:45:33.826047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:33.826047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:45:33.826047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:33.826047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:45:33.826047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:33.826047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:45:33.826047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:33.826047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:45:33.826047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:33.826047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:45:33.826047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:45:33.826557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:33.826557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:45:33.826557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:33.826557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:45:33.826557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:45:33.826557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:33.826557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:45:33.826557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:33.826557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:45:33.827007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:33.827007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:45:33.827007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:45:33.827007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:45:33.827007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:45:33.827007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:45:33.827007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:45:33.827007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:45:33.827007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:45:33.827007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:33.827007 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:45:33.827399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:33.827399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:45:33.827399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:33.827399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:45:33.827399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:33.827399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:45:33.827399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:33.827399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:45:33.827399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:45:33.827399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:33.827399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:45:33.827399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:33.827399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:45:33.827399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:45:33.827863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:33.827863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:45:33.827863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:33.827863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:45:33.828128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:33.828128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:45:33.828128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:45:33.828128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:45:33.828128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:45:33.828128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:45:33.828128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:45:33.828128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:45:33.828128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:45:33.828128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:45:33.828128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:45:33.828128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:45:33.828128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:45:33.828128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:45:33.828128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:45:33.828128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:45:33.828128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:45:33.828695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:45:33.828695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:45:33.828695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:45:33.828695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:33.828695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:45:33.828695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:33.828695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:45:33.828695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:33.828695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:45:33.828695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:33.828695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:45:33.828695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:33.828695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:45:33.828695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:45:33.828695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:33.828695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:45:33.828695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:33.828695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:45:33.828695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:45:33.829313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:33.829313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:45:33.829313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:33.829313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:45:33.830349 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:45:33.830519 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:45:33.830674 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:45:33.830875 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:45:33.832070 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:45:33.832701 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:45:33.833015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:33.833893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:45:33.833893 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:33.833893 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:45:33.833893 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:45:33.833893 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:33.833893 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:45:33.833893 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:33.833893 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:45:33.834849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:33.834849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:45:33.834849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:45:33.834849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:45:33.834849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:45:33.834849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:45:33.834849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:45:33.834849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:33.834849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:45:33.834849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:33.834849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:45:33.834849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:33.835208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:45:33.835208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:33.835208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:45:33.835208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:33.835208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:45:33.835208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:45:33.835208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:33.835208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:45:33.835208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:33.835208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:45:33.835208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:45:33.835208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:33.835208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:45:33.835568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:33.835568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:45:33.835909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:33.835909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:45:33.835909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:45:33.835909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:45:33.835909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:45:33.835909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:45:33.835909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:45:33.835909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:45:33.835909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:45:33.835909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:45:33.835909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:45:33.835909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:45:33.835909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:45:33.835909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:45:33.835909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:45:33.835909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:45:33.836383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:45:33.836383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:45:33.836383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:33.836383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:45:33.836383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:33.836383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:45:33.836383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:33.836383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:45:33.836383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:33.836383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:45:33.836383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:33.836383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:45:33.836383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:45:33.836383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:33.836383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:45:33.836383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:33.836383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:45:33.836383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:45:33.836383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:33.836968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:45:33.836968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:33.836968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:45:38.665501 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:45:38.666493 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:45:38.666493 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:45:38.667654 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:45:38.750629 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:45:38.751059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:38.751807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:45:38.751807 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:38.751807 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:45:38.751807 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:45:38.751807 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:38.751807 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:45:38.751807 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:38.751807 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:45:38.752587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.086 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:38.752587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:45:38.752587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:45:38.752587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:45:38.752587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:45:38.752587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:45:38.752587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:45:38.752587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:38.752587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:45:38.752587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:38.752587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:45:38.752587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:38.753146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:45:38.753146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:38.753146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:45:38.753146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:38.753146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:45:38.753146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:45:38.753146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:38.753146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:45:38.753146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:38.753146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:45:38.753146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:45:38.753146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:38.753146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:45:38.753503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:38.753503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:45:38.753503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:38.753503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:45:38.753503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:45:38.753503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:45:38.753503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:45:38.753503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:45:38.753503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:45:38.753503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:45:38.753503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:45:38.753503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:45:38.753503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:45:38.753503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:45:38.753503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:45:38.753910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:45:38.753910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:45:38.753910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:45:38.753910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:45:38.753910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:45:38.753910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:38.753910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:45:38.753910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:38.753910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:45:38.753910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:38.753910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:45:38.753910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:38.753910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:45:38.753910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:38.753910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:45:38.753910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:45:38.753910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:38.753910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:45:38.753910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:38.754453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:45:38.754453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:45:38.754453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:38.754453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:45:38.754453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:38.754453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:45:38.808894 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:45:38.809478 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:45:38.809624 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:45:38.810955 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:45:38.811402 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:45:38.811780 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:38.812935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:45:38.812935 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:38.812935 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:45:38.812935 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:45:38.812935 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:38.812935 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:45:38.812935 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:38.812935 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:45:38.814506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:38.814506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:45:38.814506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:45:38.814506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:45:38.814506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:45:38.814506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:45:38.814506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:45:38.814506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:38.814506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:45:38.814506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:38.814506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:45:38.814506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:38.815341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:45:38.815341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:38.815341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:45:38.815341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:38.815341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:45:38.815341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:45:38.815341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:38.815341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:45:38.815341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:38.815341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:45:38.815341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:45:38.815341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:38.815341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:45:38.815981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:38.815981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:45:38.816186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:38.816186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:45:38.816186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:45:38.816186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:45:38.816186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:45:38.816186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:45:38.816186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:45:38.816186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:45:38.816186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:45:38.816186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:45:38.816186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:45:38.816186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:45:38.816186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:45:38.816186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:45:38.816186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:45:38.816186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:45:38.816599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:45:38.816599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:45:38.816599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:38.816599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:45:38.816599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:38.816599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:45:38.816599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:38.816599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:45:38.816599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:38.816599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:45:38.816599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:38.816599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:45:38.816599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:45:38.816599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:38.816599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:45:38.816599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:38.816599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:45:38.816599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:45:38.816599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:38.817093 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:45:38.817093 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:38.817093 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:45:38.817093 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:45:38.817203 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:45:38.817321 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:45:38.817564 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:45:38.818709 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:45:38.819020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:38.819836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:45:38.819836 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:38.819836 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:45:38.819836 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:45:38.819836 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:38.819836 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:45:38.819836 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:38.819836 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:45:38.821118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:38.821118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:45:38.821118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:45:38.821118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:45:38.821118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:38.821118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:45:38.821118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:38.821118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:45:38.821118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:38.821118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:45:38.821118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:38.821118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:45:38.821118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:38.821118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:45:38.821118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:45:38.821488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:38.821488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:45:38.821488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:38.821488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:45:38.821488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:45:38.821488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:38.821488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:45:38.821488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:38.821488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:45:38.822506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:38.822506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:45:38.822506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:45:38.822506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:45:38.822506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:45:38.822506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:45:38.822506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:45:38.822506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:45:38.822506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:45:38.822506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:45:38.822506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:38.822506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:45:38.822840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:38.822840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:45:38.822840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:38.822840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:45:38.822840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:38.822840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:45:38.822840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:38.822840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:45:38.822840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:45:38.822840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:38.822840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:45:38.822840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:38.822840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:45:38.822840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:45:38.823183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:38.823183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:45:38.823183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:38.823183 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:45:38.824014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:38.824014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:45:38.824014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:45:38.824014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:45:38.824014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:45:38.824014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:45:38.824014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:45:38.824014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:45:38.824014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:45:38.824014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:45:38.824014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:45:38.824014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:45:38.824014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:45:38.824014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:45:38.824014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:45:38.824014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:45:38.824412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:45:38.824412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:45:38.824412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:45:38.824412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:45:38.824412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:45:38.824412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:38.824412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:45:38.824412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:38.824412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:45:38.824412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:38.824412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:45:38.824412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:38.824412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:45:38.824412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:38.824412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:45:38.824412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:45:38.824412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:38.824412 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:45:38.824886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:38.824886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:45:38.824886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:45:38.824886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:38.824886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:45:38.824886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:38.824886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:45:38.824886 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:45:38.825122 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:45:38.825122 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:45:38.825238 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:45:38.825858 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:45:38.826270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:38.827133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:45:38.827133 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:38.827133 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:45:38.827133 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:45:38.827133 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:38.827133 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:45:38.827133 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:38.827133 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:45:38.827951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:38.827951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:45:38.827951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:45:38.827951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:45:38.827951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:38.827951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:45:38.827951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:38.827951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:45:38.827951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:38.827951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:45:38.827951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:38.827951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:45:38.827951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:38.827951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:45:38.827951 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:45:38.828324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:38.828324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:45:38.828324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:38.828324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:45:38.828324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:45:38.828324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:38.828324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:45:38.828324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:38.828324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:45:38.828890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:38.828890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:45:38.828890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:45:38.828890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:45:38.828890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:45:38.828890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:45:38.828890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:45:38.828890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:45:38.828890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:45:38.828890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:38.828890 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:45:38.829198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:38.829198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:45:38.829198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:38.829198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:45:38.829198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:38.829198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:45:38.829198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:38.829198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:45:38.829198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:45:38.829198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:38.829198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:45:38.829198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:38.829198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:45:38.829198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:45:38.829551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:38.829551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:45:38.829551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:38.829551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:45:38.830177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:38.830177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:45:38.830177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:45:38.830177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:45:38.830177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:45:38.830177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:45:38.830177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:45:38.830177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:45:38.830177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:45:38.830177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:45:38.830177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:45:38.830177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:45:38.830177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:45:38.830177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:45:38.830177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:45:38.830177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:45:38.830177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:45:38.830657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:45:38.830657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:45:38.830657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:45:38.830657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:38.830657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:45:38.830657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:38.830657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:45:38.830657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:38.830657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:45:38.830657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:38.830657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:45:38.830657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:38.830657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:45:38.830657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:45:38.830657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:38.830657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:45:38.830657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:38.830657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:45:38.830657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:45:38.831144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:38.831144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:45:38.831144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:38.831144 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:45:38.831144 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:45:38.831144 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:45:38.831144 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:45:38.831338 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:45:38.832510 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:45:38.832809 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:45:38.832982 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:38.833589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:45:38.833589 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:38.833589 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:45:38.833589 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:45:38.833589 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:38.833589 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:45:38.833589 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:38.833589 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:45:38.834537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:38.834537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:45:38.834537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:45:38.834537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:45:38.834537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:45:38.834537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:45:38.834537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:45:38.834537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:38.834537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:45:38.834537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:38.834537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:45:38.834537 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:38.834893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:45:38.834893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:38.834893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:45:38.834893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:38.834893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:45:38.834893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:45:38.834893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:38.834893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:45:38.834893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:38.834893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:45:38.834893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:45:38.834893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:38.834893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:45:38.835273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:38.835273 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:45:38.835579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:38.835579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:45:38.835579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:45:38.835579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:45:38.835579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:45:38.835579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:45:38.835579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:45:38.835579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:45:38.835579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:45:38.835579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:45:38.835579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:45:38.835579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:45:38.835579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:45:38.835579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:45:38.835579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:45:38.835579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:45:38.836019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:45:38.836019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:45:38.836019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:38.836019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:45:38.836019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:38.836019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:45:38.836019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:38.836019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:45:38.836019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:38.836019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:45:38.836019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:38.836019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:45:38.836019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:45:38.836019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:38.836019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:45:38.836019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:38.836019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:45:38.836019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:45:38.836019 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:38.836518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:45:38.836518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:38.836518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:45:43.666025 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:45:43.667083 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:45:43.667083 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:45:43.668014 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:45:43.702042 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:45:43.702357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:43.703096 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:45:43.703096 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:43.703096 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:45:43.703096 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:45:43.703096 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:43.703096 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:45:43.703096 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:43.703096 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:45:43.704204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:43.704204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:45:43.704204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:45:43.704204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:45:43.704204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:45:43.704204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:45:43.704204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:45:43.704204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:43.704204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:45:43.704204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:43.704204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:45:43.704204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:43.704715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:45:43.704715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:43.704715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:45:43.704715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:43.704715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:45:43.704715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:45:43.704715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:43.704715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:45:43.704715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:43.704715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:45:43.704715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:45:43.704715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:43.704715 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:45:43.705140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:43.705140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:45:43.705322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:43.705322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:45:43.705322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:45:43.705322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:45:43.705322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:45:43.705322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:45:43.705322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:45:43.705322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:45:43.705322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:45:43.705322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:45:43.705322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:45:43.705322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:45:43.705322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:45:43.705322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:45:43.705322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:45:43.705322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:45:43.706286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:45:43.706286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:45:43.706286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:43.706286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:45:43.706286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:43.706286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:45:43.706286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:43.706286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:45:43.706286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:43.706286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:45:43.706286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:43.706286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:45:43.706286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:45:43.706286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:43.706286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:45:43.706286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:43.706286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:45:43.706286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:45:43.706286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:43.707300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:45:43.707300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:43.707300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:45:43.809544 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:45:43.810295 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:45:43.810541 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:45:43.811902 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:45:43.812296 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:45:43.812546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:43.813382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:45:43.813382 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:43.813382 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:45:43.813382 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:45:43.813382 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:43.813382 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:45:43.813382 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:43.813382 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:45:43.814487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:43.814487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:45:43.814487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:45:43.814487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:45:43.814487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:45:43.814487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:45:43.814487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:45:43.814487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:43.814487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:45:43.814487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:43.814487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:45:43.814487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:43.815316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:45:43.815316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:43.815316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:45:43.815316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:43.815316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:45:43.815316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:45:43.815316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:43.815316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:45:43.815316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:43.815316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:45:43.815316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:45:43.815316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:43.815316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:45:43.816059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:43.816059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:45:43.816059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:43.816059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:45:43.816059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:45:43.816059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:45:43.816059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:45:43.816059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:45:43.816059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:45:43.816059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:45:43.816059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:45:43.816059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:45:43.816059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:45:43.816059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:45:43.816059 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:45:43.816814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:45:43.816814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:45:43.816814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:45:43.816814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:45:43.816814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:45:43.816814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:43.816814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:45:43.816814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:43.816814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:45:43.816814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:43.816814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:45:43.816814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:43.816814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:45:43.816814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:43.816814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:45:43.816814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:45:43.816814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:43.816814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:45:43.816814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:43.817632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:45:43.817632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:45:43.817632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:43.817632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:45:43.817632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:43.817632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:45:43.817987 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:45:43.818072 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:45:43.818213 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:45:43.818292 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:45:43.818894 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:45:43.819194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:43.819920 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:45:43.819920 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:43.819920 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:45:43.819920 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:45:43.819920 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:43.819920 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:45:43.819920 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:43.819920 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:45:43.820695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:43.820695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:45:43.820695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:45:43.820695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:45:43.820695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:43.820695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:45:43.820695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:43.820695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:45:43.820695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:43.820695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:45:43.820695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:43.820695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:45:43.820695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:43.820695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:45:43.820695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:45:43.821559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:43.821559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:45:43.821559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:43.821559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:45:43.821559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:45:43.821559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:43.821559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:45:43.821559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:43.821559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:45:43.821559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:43.821559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:45:43.821559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:45:43.821559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:45:43.821559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:45:43.822317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:45:43.822317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:45:43.822317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:45:43.822317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:45:43.822317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:45:43.822317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:43.822317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:45:43.822317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:43.822317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:45:43.822317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:43.822317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:45:43.822317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:43.822317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:45:43.823063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:43.823063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:45:43.823063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:45:43.823063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:43.823063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:45:43.823063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:43.823063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:45:43.823063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:45:43.823063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:43.823063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:45:43.823063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:43.823063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:45:43.823648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:43.823648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:45:43.823648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:45:43.823648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:45:43.823648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:45:43.823648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:45:43.823648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:45:43.823648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:45:43.823648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:45:43.823648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:45:43.823648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:45:43.823648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:45:43.823648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:45:43.823648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:45:43.823648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:45:43.823648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:45:43.824513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:45:43.824513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:45:43.824513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:45:43.824513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:45:43.824513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:45:43.824513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:43.824513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:45:43.824513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:43.824513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:45:43.824513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:43.824513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:45:43.824513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:43.824513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:45:43.824513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:43.824513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:45:43.824513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:45:43.824513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:43.824513 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:45:43.825203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:43.825203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:45:43.825203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:45:43.825203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:43.825203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:45:43.825203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:43.825203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:45:43.825203 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:45:43.825526 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:45:43.826010 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:45:43.826237 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:45:43.826861 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:45:43.827366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:43.829171 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:45:43.829171 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:43.829171 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:45:43.829171 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:45:43.829171 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:43.829171 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:45:43.829171 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:43.829171 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:45:43.830956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:43.830956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:45:43.830956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:45:43.830956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:45:43.830956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:43.830956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:45:43.830956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:43.830956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:45:43.830956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:43.830956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:45:43.830956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:43.830956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:45:43.830956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:43.830956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:45:43.830956 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:45:43.831414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:43.831414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:45:43.831414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:43.831414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:45:43.831414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:45:43.831414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:43.831414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:45:43.831414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:43.831414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:45:43.831414 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:45:43.833336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:43.833336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:45:43.833336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:45:43.833336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:45:43.833336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:45:43.833336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:45:43.833336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:45:43.833336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:45:43.833336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:45:43.833336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:43.833336 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:45:43.833804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:43.833804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:45:43.833804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:43.833804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:45:43.833804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:43.833804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:45:43.833804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:43.833804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:45:43.833804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:45:43.833804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:43.833804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:45:43.833804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:43.833804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:45:43.833804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:45:43.834188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:43.834188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:45:43.834188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:43.834188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:45:43.836108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:43.836108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:45:43.836108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:45:43.836108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:45:43.836108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:45:43.836108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:45:43.836108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:45:43.836108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:45:43.836108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:45:43.836108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:45:43.836108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:45:43.836108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:45:43.836108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:45:43.836108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:45:43.836108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:45:43.836108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:45:43.836108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:45:43.836565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:45:43.836565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:45:43.836565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:45:43.836565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:43.836565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:45:43.836565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:43.836565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:45:43.836565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:43.836565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:45:43.836565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:43.836565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:45:43.836565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:43.836565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:45:43.836565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:45:43.836565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:43.836565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:45:43.836565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:43.836565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:45:43.836565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:45:43.837077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:43.837077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:45:43.837077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:43.837077 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:45:43.837077 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:45:43.837077 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:45:43.837077 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:45:43.837077 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:45:43.837380 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:45:43.837602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:43.838270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:45:43.838270 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:43.838270 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:45:43.838270 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:45:43.838270 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:43.838270 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:45:43.838270 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:43.838270 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:45:43.839193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:43.839193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:45:43.839193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:45:43.839193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:45:43.839193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:45:43.839193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:45:43.839193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:45:43.839193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:43.839193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:45:43.839193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:43.839193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:45:43.839193 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:43.839586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:45:43.839586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:43.839586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:45:43.839586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:43.839586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:45:43.839586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:45:43.839586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:43.839586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:45:43.839586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:43.839586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:45:43.839586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:45:43.839586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:43.839586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:45:43.839961 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:43.839961 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:45:43.840398 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:43.840398 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:45:43.840398 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:45:43.840398 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:45:43.840398 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:45:43.840398 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:45:43.840398 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:45:43.840398 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:45:43.840398 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:45:43.840398 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:45:43.840398 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:45:43.840398 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:45:43.840398 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:45:43.840398 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:45:43.840398 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:45:43.840398 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:45:43.840822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:45:43.840822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:45:43.840822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:43.840822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:45:43.840822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:43.840822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:45:43.840822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:43.840822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:45:43.840822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:43.840822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:45:43.840822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:43.840822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:45:43.840822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:45:43.840822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:43.840822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:45:43.840822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:43.840822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:45:43.840822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:45:43.840822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:43.841414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:45:43.841414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:43.841414 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:45:48.667284 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:45:48.668179 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:45:48.668179 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:45:48.668647 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:45:48.704760 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:45:48.705068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:48.705864 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:45:48.705864 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:48.705864 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:45:48.705864 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:45:48.705864 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:48.705864 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:45:48.705864 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:48.705864 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:45:48.706641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:48.706641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:45:48.706641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:45:48.706641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:45:48.706641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:45:48.706641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:45:48.706641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:45:48.706641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:48.706641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:45:48.706641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:48.706641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:45:48.706641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:48.707277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:45:48.707277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:48.707277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:45:48.707277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:48.707277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:45:48.707277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:45:48.707277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:48.707277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:45:48.707277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:48.707277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:45:48.707277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:45:48.707277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:48.707277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:45:48.712224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:48.712224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:45:48.712224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:48.712224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:45:48.712224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:45:48.712224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:45:48.712224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:45:48.712224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:45:48.712224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:45:48.712224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:45:48.712224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:45:48.712224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:45:48.712224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:45:48.712224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:45:48.712224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:45:48.712786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:45:48.712786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:45:48.712786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:45:48.712786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:45:48.712786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:45:48.712786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:48.712786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:45:48.712786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:48.712786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:45:48.712786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:48.712786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:45:48.712786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:48.712786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:45:48.712786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:48.712786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:45:48.712786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:45:48.712786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:48.712786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:45:48.712786 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:48.713270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:45:48.713270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:45:48.713270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:48.713270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:45:48.713270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:48.713270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:45:48.810354 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:45:48.811130 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:45:48.811321 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:45:48.812593 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:45:48.813116 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:45:48.813345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:48.814506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:45:48.814506 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:48.814506 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:45:48.814506 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:45:48.814506 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:48.814506 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:45:48.814506 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:48.814506 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:45:48.816142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:48.816142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:45:48.816142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:45:48.816142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:45:48.816142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:45:48.816142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:45:48.816142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:45:48.816142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:48.816142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:45:48.816142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:48.816142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:45:48.816142 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:48.816475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:45:48.816475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:48.816475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:45:48.816475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:48.816475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:45:48.816475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:45:48.816475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:48.816475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:45:48.816475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:48.816475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:45:48.816475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:45:48.816475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:48.816475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:45:48.816863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:48.816863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:45:48.819323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:48.819323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:45:48.819323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:45:48.819323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:45:48.819323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:45:48.819323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:45:48.819323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:45:48.819323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:45:48.819323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:45:48.819323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:45:48.819323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:45:48.819323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:45:48.819323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:45:48.819323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:45:48.819323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:45:48.819323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:45:48.819773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:45:48.819773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:45:48.819773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:48.819773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:45:48.819773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:48.819773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:45:48.819773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:48.819773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:45:48.819773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:48.819773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:45:48.819773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:48.819773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:45:48.819773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:45:48.819773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:48.819773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:45:48.819773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:48.819773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:45:48.819773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:45:48.819773 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:48.820264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:45:48.820264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:48.820264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:45:48.820264 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:45:48.820264 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:45:48.820408 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:45:48.820629 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:45:48.821075 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:45:48.821287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:48.821904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:45:48.821904 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:48.821904 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:45:48.821904 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:45:48.821904 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:48.821904 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:45:48.821904 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:48.821904 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:45:48.822682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:48.822682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:45:48.822682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:45:48.822682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:45:48.822682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:48.822682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:45:48.822682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:48.822682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:45:48.822682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:48.822682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:45:48.822682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:48.822682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:45:48.822682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:48.822682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:45:48.822682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:45:48.823106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:48.823106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:45:48.823106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:48.823106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:45:48.823106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:45:48.823106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:48.823106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:45:48.823106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:48.823106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:45:48.823615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:48.823615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:45:48.823615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:45:48.823615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:45:48.823615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:45:48.823615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:45:48.823615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:45:48.823615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:45:48.823615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:45:48.823615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:45:48.823615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:48.823615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:45:48.823947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:48.823947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:45:48.823947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:48.823947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:45:48.823947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:48.823947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:45:48.823947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:48.823947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:45:48.823947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:45:48.823947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:48.823947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:45:48.823947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:48.823947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:45:48.823947 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:45:48.824314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:48.824314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:45:48.824314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:48.824314 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:45:48.824642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:48.824642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:45:48.824642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:45:48.824642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:45:48.824642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:45:48.824642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:45:48.824642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:45:48.824642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:45:48.824642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:45:48.824642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:45:48.824642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:45:48.824642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:45:48.824642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:45:48.824642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:45:48.824642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:45:48.824642 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:45:48.825057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:45:48.825057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:45:48.825057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:45:48.825057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:45:48.825057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:45:48.825057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:48.825057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:45:48.825057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:48.825057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:45:48.825057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:48.825057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:45:48.825057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:48.825057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:45:48.825057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:48.825057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:45:48.825057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:45:48.825057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:48.825057 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:45:48.825510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:48.825510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:45:48.825510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:45:48.825510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:48.825510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:45:48.825510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:48.825510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:45:48.825687 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:45:48.825972 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:45:48.826079 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:45:48.826202 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:45:48.826766 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:45:48.827204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:48.828002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:45:48.828002 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:48.828002 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:45:48.828002 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:45:48.828002 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:48.828002 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:45:48.828002 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:48.828002 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:45:48.828755 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:48.828755 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:45:48.828755 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:45:48.828755 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:45:48.828755 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:48.828755 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:45:48.828755 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:48.828755 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:45:48.828755 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:48.828755 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:45:48.828755 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:48.828755 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:45:48.828755 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:48.828755 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:45:48.828755 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:45:48.829141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:48.829141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:45:48.829141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:48.829141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:45:48.829141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:45:48.829141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:48.829141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:45:48.829141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:48.829141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:45:48.829618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:48.829618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:45:48.829618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:45:48.829618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:45:48.829618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:45:48.829618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:45:48.829618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:45:48.829618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:45:48.829618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:45:48.829618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:48.829618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:45:48.829986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:48.829986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:45:48.829986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:48.829986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:45:48.829986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:48.829986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:45:48.829986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:48.829986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:45:48.829986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:45:48.829986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:48.829986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:45:48.829986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:48.829986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:45:48.829986 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:45:48.830392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:48.830392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:45:48.830392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:48.830392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:45:48.830737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:48.830737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:45:48.830737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:45:48.830737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:45:48.830737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:45:48.830737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:45:48.830737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:45:48.830737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:45:48.830737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:45:48.830737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:45:48.830737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:45:48.830737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:45:48.830737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:45:48.830737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:45:48.830737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:45:48.830737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:45:48.830737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:45:48.831228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:45:48.831228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:45:48.831228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:45:48.831228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:48.831228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:45:48.831228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:48.831228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:45:48.831228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:48.831228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:45:48.831228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:48.831228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:45:48.831228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:48.831228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:45:48.831228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:45:48.831228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:48.831228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:45:48.831228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:48.831228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:45:48.831228 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:45:48.832095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:48.832095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:45:48.832095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:48.832095 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:45:48.836136 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:45:48.836427 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:45:48.836556 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:45:48.836733 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:45:48.837318 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:45:48.837892 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:45:48.838264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:48.839136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:45:48.839136 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:48.839136 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:45:48.839136 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:45:48.839136 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:48.839136 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:45:48.839136 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:48.839136 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:45:48.840066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:48.840066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:45:48.840066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:45:48.840066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:45:48.840066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:45:48.840066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:45:48.840066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:45:48.840066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:48.840066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:45:48.840066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:48.840066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:45:48.840066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:48.840410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:45:48.840410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:48.840410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:45:48.840410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:48.840410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:45:48.840410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:45:48.840410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:48.840410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:45:48.840410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:48.840410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:45:48.840410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:45:48.840410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:48.840410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:45:48.840794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:48.840794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:45:48.841062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:48.841062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:45:48.841062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:45:48.841062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:45:48.841062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:45:48.841062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:45:48.841062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:45:48.841062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:45:48.841062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:45:48.841062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:45:48.841062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:45:48.841062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:45:48.841062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:45:48.841062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:45:48.841062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:45:48.841062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:45:48.841515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:45:48.841515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:45:48.841515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:48.841515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:45:48.841515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:48.841515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:45:48.841515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:48.841515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:45:48.841515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:48.841515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:45:48.841515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:48.841515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:45:48.841515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:45:48.841515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:48.841515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:45:48.841515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:48.841515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:45:48.841515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:45:48.841515 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:48.842090 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:45:48.842090 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:48.842090 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:45:53.667909 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:45:53.669072 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:45:53.669072 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:45:53.669520 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:45:53.703303 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:45:53.703561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:53.704410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:45:53.704410 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:53.704410 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:45:53.704410 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:45:53.704410 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:53.704410 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:45:53.704410 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:53.704410 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:45:53.705891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:53.705891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:45:53.705891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:45:53.705891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:45:53.705891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:45:53.705891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:45:53.705891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:45:53.705891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:53.705891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:45:53.705891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:53.705891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:45:53.705891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:53.706876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:45:53.706876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:53.706876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:45:53.706876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:53.706876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:45:53.706876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:45:53.706876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:53.706876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:45:53.706876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:53.706876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:45:53.706876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:45:53.706876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:53.706876 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:45:53.707858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:53.707858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:45:53.707858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:53.707858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:45:53.707858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:45:53.707858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:45:53.707858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:45:53.707858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:45:53.707858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:45:53.707858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:45:53.707858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:45:53.707858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:45:53.707858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:45:53.707858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:45:53.707858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:45:53.708964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:45:53.708964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:45:53.708964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:45:53.708964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:45:53.708964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:45:53.708964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:53.708964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:45:53.708964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:53.708964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:45:53.708964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:53.708964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:45:53.708964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:53.708964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:45:53.708964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:53.708964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:45:53.708964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:45:53.708964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:53.708964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:45:53.708964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:53.710281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:45:53.710281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:45:53.710281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:53.710281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:45:53.710281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:53.710281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:45:53.811209 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:45:53.820205 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:45:53.826116 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:45:53.836590 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:45:53.855333 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:45:53.855608 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:45:53.857244 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:45:53.857588 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:45:53.858072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:53.859980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:45:53.859980 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:53.859980 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:45:53.859980 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:45:53.859980 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:53.859980 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:45:53.859980 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:53.859980 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:45:53.861674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:53.861674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:45:53.861674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:45:53.861674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:45:53.861674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:45:53.861674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:45:53.861674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:45:53.861674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:53.861674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:45:53.861674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:53.861674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:45:53.861674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:53.862552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:45:53.862552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:53.862552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:45:53.862552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:53.862552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:45:53.862552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:45:53.862552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:53.862552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:45:53.862552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:53.862552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:45:53.862552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:45:53.862552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:53.862552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:45:53.863186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:53.863186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:45:53.863730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:53.863730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:45:53.863730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:45:53.863730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:45:53.863730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:45:53.863730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:45:53.863730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:45:53.863730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:45:53.863730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:45:53.863730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:45:53.863730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:45:53.863730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:45:53.863730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:45:53.863730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:45:53.863730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:45:53.863730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:45:53.864771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:45:53.864771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:45:53.864771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:53.864771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:45:53.864771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:53.864771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:45:53.864771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:53.864771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:45:53.864771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:53.864771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:45:53.864771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:53.864771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:45:53.864771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:45:53.864771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:53.864771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:45:53.864771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:53.864771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:45:53.864771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:45:53.864771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:53.865684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:45:53.865684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:53.865684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:45:53.865684 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:45:53.865684 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:45:53.865684 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:45:53.865684 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:45:53.865684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:53.866168 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:45:53.866168 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:53.866168 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:45:53.866168 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:45:53.866168 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:53.866168 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:45:53.866168 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:53.866168 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:45:53.866707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:53.866707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:45:53.866707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:45:53.866707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:45:53.866707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:53.866707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:45:53.866707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:53.866707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:45:53.866707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:53.866707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:45:53.866707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:53.866707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:45:53.866707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:53.866707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:45:53.866707 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:45:53.867581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:53.867581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:45:53.867581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:53.867581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:45:53.867581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:45:53.867581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:53.867581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:45:53.867581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:53.867581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:45:53.867581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:53.867581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:45:53.867581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:45:53.867581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:45:53.867581 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:45:53.868198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:45:53.868198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:45:53.868198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:45:53.868198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:45:53.868198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:45:53.868198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:53.868198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:45:53.868198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:53.868198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:45:53.868198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:53.868198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:45:53.868198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:53.868198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:45:53.868573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:53.868573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:45:53.868573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:45:53.868573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:53.868573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:45:53.868573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:53.868573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:45:53.868573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:45:53.868573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:53.868573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:45:53.868573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:53.868573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:45:53.868933 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:53.868933 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:45:53.868933 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:45:53.868933 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:45:53.868933 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:45:53.868933 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:45:53.868933 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:45:53.868933 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:45:53.868933 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:45:53.868933 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:45:53.868933 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:45:53.868933 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:45:53.868933 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:45:53.868933 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:45:53.868933 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:45:53.868933 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:45:53.869348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:45:53.869348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:45:53.869348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:45:53.869348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:45:53.869348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:45:53.869348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:53.869348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:45:53.869348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:53.869348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:45:53.869348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:53.869348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:45:53.869348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:53.869348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:45:53.869348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:53.869348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:45:53.869348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:45:53.869348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:53.869348 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:45:53.869883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:53.869883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:45:53.869883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:45:53.869883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:53.869883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:45:53.869883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:53.869883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:45:53.869883 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:45:53.869883 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:45:53.869883 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:45:53.870190 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:45:53.870190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:53.870190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:45:53.870190 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:53.870190 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:45:53.870190 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:45:53.870190 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:53.870190 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:45:53.870190 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:53.870190 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:45:53.870465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:53.870465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:45:53.870465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:45:53.870465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:45:53.870465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:53.870465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:45:53.870465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:53.870465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:45:53.870465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:53.870465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:45:53.870465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:53.870465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:45:53.870465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:53.870465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:45:53.870465 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:45:53.870868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:53.870868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:45:53.870868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:53.870868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:45:53.870868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:45:53.870868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:53.870868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:45:53.870868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:53.870868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:45:53.871128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:53.871128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:45:53.871128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:45:53.871128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:45:53.871128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:45:53.871128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:45:53.871128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:45:53.871128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:45:53.871128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:45:53.871128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:53.871128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:45:53.871469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:53.871469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:45:53.871469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:53.871469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:45:53.871469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:53.871469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:45:53.871469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:53.871469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:45:53.871469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:45:53.871469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:53.871469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:45:53.871469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:53.871469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:45:53.871469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:45:53.871869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:53.871869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:45:53.871869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:53.871869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:45:53.871869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:53.871869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:45:53.871869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:45:53.871869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:45:53.871869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:45:53.871869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:45:53.871869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:45:53.871869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:45:53.871869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:45:53.871869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:45:53.872286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:45:53.872286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:45:53.872286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:45:53.872286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:45:53.872286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:45:53.872286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:45:53.872286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:45:53.872286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:45:53.872286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:45:53.872286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:45:53.872286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:53.872286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:45:53.872286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:53.872286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:45:53.872286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:53.872286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:45:53.872286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:53.872286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:45:53.872286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:53.872791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:45:53.872791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:45:53.872791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:53.872791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:45:53.872791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:53.872791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:45:53.872791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:45:53.872791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:53.872791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:45:53.872791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:53.872791 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:45:53.872791 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:45:53.872791 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:45:53.873160 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:45:53.873160 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:45:53.873160 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:45:53.873160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:53.873629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:45:53.873629 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:53.873629 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:45:53.873629 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:45:53.873629 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:53.873629 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:45:53.873629 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:53.873629 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:45:53.874598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:53.874598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:45:53.874598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:45:53.874598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:45:53.874598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:45:53.874598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:45:53.874598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:45:53.874598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:53.874598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:45:53.874598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:53.874598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:45:53.874598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:53.874976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:45:53.874976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:53.874976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:45:53.874976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:53.874976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:45:53.874976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:45:53.874976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:53.874976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:45:53.874976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:53.874976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:45:53.874976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:45:53.874976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:53.874976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:45:53.875331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:53.875331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:45:53.876000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:53.876000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:45:53.876000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:45:53.876000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:45:53.876000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:45:53.876000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:45:53.876000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:45:53.876000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:45:53.876000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:45:53.876000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:45:53.876000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:45:53.876000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:45:53.876000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:45:53.876000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:45:53.876000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:45:53.876000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:45:53.877282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:45:53.877282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:45:53.877282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:53.877282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:45:53.877282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:53.877282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:45:53.877282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:53.877282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:45:53.877282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:53.877282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:45:53.877282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:53.877282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:45:53.877282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:45:53.877282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:53.877282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:45:53.877282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:53.877282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:45:53.877282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:45:53.877282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:53.878601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:45:53.878601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:53.878601 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:45:58.668913 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:45:58.669817 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:45:58.669817 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:45:58.671129 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:45:58.673540 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG dbcounter [-] [106314] Writing DB stats neutron:SELECT=240 {{(pid=106314) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 09 09:45:58.709453 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:45:58.709891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:58.710845 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:45:58.710845 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:58.710845 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:45:58.710845 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:45:58.710845 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:58.710845 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:45:58.710845 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:58.710845 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:45:58.711911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:58.711911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:45:58.711911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:45:58.711911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:45:58.711911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:45:58.711911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:45:58.711911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:45:58.711911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:58.711911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:45:58.711911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:58.711911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:45:58.711911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:58.712349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:45:58.712349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:58.712349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:45:58.712349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:58.712349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:45:58.712349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:45:58.712349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:58.712349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:45:58.712349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:58.712349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:45:58.712349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:45:58.712349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:58.712349 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:45:58.713012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:58.713012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:45:58.713012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:58.713012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:45:58.713012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:45:58.713012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:45:58.713012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:45:58.713012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:45:58.713012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:45:58.713012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:45:58.713012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:45:58.713012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:45:58.713012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:45:58.713012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:45:58.713012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:45:58.713783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:45:58.713783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:45:58.713783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:45:58.713783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:45:58.713783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:45:58.713783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:58.713783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:45:58.713783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:58.713783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:45:58.713783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:58.713783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:45:58.713783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:58.713783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:45:58.713783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:58.713783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:45:58.713783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:45:58.713783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:58.713783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:45:58.713783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:58.714660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:45:58.714660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:45:58.714660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:58.714660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:45:58.714660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:58.714660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:45:58.853907 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:45:58.853907 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:45:58.854204 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:45:58.855196 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:45:58.855492 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:45:58.855720 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:58.856626 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:45:58.856626 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:58.856626 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:45:58.856626 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:45:58.856626 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:58.856626 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:45:58.856626 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:58.856626 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:45:58.858247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:58.858247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:45:58.858247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:45:58.858247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:45:58.858247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:45:58.858247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:45:58.858247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:45:58.858247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:58.858247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:45:58.858247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:58.858247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:45:58.858247 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:58.859023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:45:58.859023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:58.859023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:45:58.859023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:58.859023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:45:58.859023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:45:58.859023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:58.859023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:45:58.859023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:58.859023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:45:58.859023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:45:58.859023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:58.859023 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:45:58.859605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:58.859605 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:45:58.860079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:58.860079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:45:58.860079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:45:58.860079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:45:58.860079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:45:58.860079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:45:58.860079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:45:58.860079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:45:58.860079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:45:58.860079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:45:58.860079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:45:58.860079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:45:58.860079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:45:58.860079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:45:58.860079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:45:58.860079 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:45:58.861696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:45:58.861696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:45:58.861696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:58.861696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:45:58.861696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:58.861696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:45:58.861696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:58.861696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:45:58.861696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:58.861696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:45:58.861696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:58.861696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:45:58.861696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:45:58.861696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:58.861696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:45:58.861696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:58.861696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:45:58.861696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:45:58.861696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:58.862588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:45:58.862588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:58.862588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:45:58.864288 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:45:58.864726 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:45:58.864948 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:45:58.865152 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:45:58.865533 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:45:58.865762 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:58.866230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:45:58.866230 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:58.866230 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:45:58.866230 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:45:58.866230 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:58.866230 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:45:58.866230 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:58.866230 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:45:58.866976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:58.866976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:45:58.866976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:45:58.866976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:45:58.866976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:58.866976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:45:58.866976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:58.866976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:45:58.866976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:58.866976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:45:58.866976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:58.866976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:45:58.866976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:58.866976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:45:58.866976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:45:58.867437 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:58.867437 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:45:58.867437 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:58.867437 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:45:58.867437 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:45:58.867437 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:58.867437 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:45:58.867437 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:58.867437 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:45:58.867803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:58.867803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:45:58.867803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:45:58.867803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:45:58.867803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:45:58.867803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:45:58.867803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:45:58.867803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:45:58.867803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:45:58.867803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:45:58.867803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:58.867803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:45:58.868215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:58.868215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:45:58.868215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:58.868215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:45:58.868215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:58.868215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:45:58.868215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:58.868215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:45:58.868215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:45:58.868215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:58.868215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:45:58.868215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:58.868215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:45:58.868215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:45:58.868647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:58.868647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:45:58.868647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:58.868647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:45:58.868965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:58.868965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:45:58.868965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:45:58.868965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:45:58.868965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:45:58.868965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:45:58.868965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:45:58.868965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:45:58.868965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:45:58.868965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:45:58.868965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:45:58.868965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:45:58.868965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:45:58.868965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:45:58.868965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:45:58.868965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:45:58.869421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:45:58.869421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:45:58.869421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:45:58.869421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:45:58.869421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:45:58.869421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:58.869421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:45:58.869421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:58.869421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:45:58.869421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:58.869421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:45:58.869421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:58.869421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:45:58.869421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:58.869421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:45:58.869421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:45:58.869421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:58.869421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:45:58.870243 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:58.870243 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:45:58.870243 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:45:58.870243 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:58.870243 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:45:58.870243 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:58.870243 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:45:58.870243 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:45:58.870243 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:45:58.870243 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:45:58.870672 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:45:58.870672 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:45:58.870672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:58.870851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:45:58.870851 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:58.870851 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:45:58.870851 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:45:58.870851 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:58.870851 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:45:58.870851 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:58.870851 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:45:58.871563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:58.871563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:45:58.871563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:45:58.871563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:45:58.871563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:58.871563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:45:58.871563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:58.871563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:45:58.871563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:58.871563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:45:58.871563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:58.871563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:45:58.871563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:58.871563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:45:58.871563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:45:58.872206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:58.872206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:45:58.872206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:58.872206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:45:58.872206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:45:58.872206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:58.872206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:45:58.872206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:58.872206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:45:58.872577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:58.872577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:45:58.872577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:45:58.872577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:45:58.872577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:45:58.872577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:45:58.872577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:45:58.872577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:45:58.872577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:45:58.872577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:58.872577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:45:58.873015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:58.873015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:45:58.873015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:58.873015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:45:58.873015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:58.873015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:45:58.873015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:58.873015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:45:58.873015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:45:58.873015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:58.873015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:45:58.873015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:58.873015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:45:58.873015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:45:58.873609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:58.873609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:45:58.873609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:58.873609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:45:58.873798 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:58.873798 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:45:58.873798 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:45:58.873798 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:45:58.873798 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:45:58.873798 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:45:58.873798 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:45:58.873798 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:45:58.873798 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:45:58.873798 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:45:58.873798 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:45:58.873798 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:45:58.873798 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:45:58.873798 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:45:58.873798 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:45:58.873798 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:45:58.873798 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:45:58.874721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:45:58.874721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:45:58.874721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:45:58.874721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:58.874721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:45:58.874721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:58.874721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:45:58.874721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:58.874721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:45:58.874721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:58.874721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:45:58.874721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:58.874721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:45:58.874721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:45:58.874721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:58.874721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:45:58.874721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:58.874721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:45:58.874721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:45:58.875421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:58.875421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:45:58.875421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:58.875421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:45:58.875421 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:45:58.875421 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:45:58.875421 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:45:58.875421 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:45:58.875956 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:45:58.875956 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:45:58.876138 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:58.876911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:45:58.876911 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:58.876911 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:45:58.876911 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:45:58.876911 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:58.876911 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:45:58.876911 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:58.876911 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:45:58.877807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:58.877807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:45:58.877807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:45:58.877807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:45:58.877807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:45:58.877807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:45:58.877807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:45:58.877807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:58.877807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:45:58.877807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:58.877807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:45:58.877807 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:58.878256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:45:58.878256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:58.878256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:45:58.878256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:58.878256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:45:58.878256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:45:58.878256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:58.878256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:45:58.878256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:58.878256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:45:58.878256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:45:58.878256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:58.878256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:45:58.878615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:58.878615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:45:58.878968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:58.878968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:45:58.878968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:45:58.878968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:45:58.878968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:45:58.878968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:45:58.878968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:45:58.878968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:45:58.878968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:45:58.878968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:45:58.878968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:45:58.878968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:45:58.878968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:45:58.878968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:45:58.878968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:45:58.878968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:45:58.879900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:45:58.879900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:45:58.879900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:58.879900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:45:58.879900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:45:58.879900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:45:58.879900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:45:58.879900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:45:58.879900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:45:58.879900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:45:58.879900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:45:58.879900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:45:58.879900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:45:58.879900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:45:58.879900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:45:58.879900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:45:58.879900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:45:58.879900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:45:58.879900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:45:58.880851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:45:58.880851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:45:58.880851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:46:03.669506 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:46:03.670134 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:46:03.670356 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:46:03.671835 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:46:03.712777 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:46:03.712961 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:03.713965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:46:03.713965 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:03.713965 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:46:03.713965 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:46:03.713965 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:03.713965 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:46:03.713965 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:03.713965 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:46:03.715278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.044 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:03.715278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:46:03.715278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:46:03.715278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:46:03.715278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:46:03.715278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:46:03.715278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:46:03.715278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:03.715278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:46:03.715278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:03.715278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:46:03.715278 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:03.715726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:46:03.715726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:03.715726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:46:03.715726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:03.715726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:46:03.715726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:46:03.715726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:03.715726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:46:03.715726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:03.715726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:46:03.715726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:46:03.715726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:03.715726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:46:03.716133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:03.716133 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:46:03.716316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:03.716316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:46:03.716316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:46:03.716316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:46:03.716316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:46:03.716316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:46:03.716316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:46:03.716316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:46:03.716316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:46:03.716316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:46:03.716316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:46:03.716316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:46:03.716316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:46:03.716316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:46:03.716316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:46:03.716316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:46:03.716926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:46:03.716926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:46:03.716926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:03.716926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:46:03.716926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:03.716926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:46:03.716926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:03.716926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:46:03.716926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:03.716926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:46:03.716926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:03.716926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:46:03.716926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:46:03.716926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:03.716926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:46:03.716926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:03.716926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:46:03.716926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:46:03.716926 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:03.717469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:46:03.717469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:03.717469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:46:03.854149 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:46:03.854873 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:46:03.855132 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:46:03.856575 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:46:03.857010 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:46:03.857380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:03.858317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:46:03.858317 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:03.858317 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:46:03.858317 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:46:03.858317 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:03.858317 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:46:03.858317 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:03.858317 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:46:03.860070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:03.860070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:46:03.860070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:46:03.860070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:46:03.860070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:46:03.860070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:46:03.860070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:46:03.860070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:03.860070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:46:03.860070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:03.860070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:46:03.860070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:03.860796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:46:03.860796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:03.860796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:46:03.860796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:03.860796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:46:03.860796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:46:03.860796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:03.860796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:46:03.860796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:03.860796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:46:03.860796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:46:03.860796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:03.860796 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:46:03.861419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:03.861419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:46:03.861783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:03.861783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:46:03.861783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:46:03.861783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:46:03.861783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:46:03.861783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:46:03.861783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:46:03.861783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:46:03.861783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:46:03.861783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:46:03.861783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:46:03.861783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:46:03.861783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:46:03.861783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:46:03.861783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:46:03.861783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:46:03.862469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:46:03.862469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:46:03.862469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:03.862469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:46:03.862469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:03.862469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:46:03.862469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:03.862469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:46:03.862469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:03.862469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:46:03.862469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:03.862469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:46:03.862469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:46:03.862469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:03.862469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:46:03.862469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:03.862469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:46:03.862469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:46:03.862469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:03.863383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:46:03.863383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:03.863383 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:46:03.864935 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:46:03.865332 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:46:03.865421 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:46:03.865572 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:46:03.866020 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:46:03.866262 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:03.866979 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:46:03.866979 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:03.866979 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:46:03.866979 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:46:03.866979 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:03.866979 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:46:03.866979 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:03.866979 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:46:03.867830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:03.867830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:46:03.867830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:46:03.867830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:46:03.867830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:03.867830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:46:03.867830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:03.867830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:46:03.867830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:03.867830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:46:03.867830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:03.867830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:46:03.867830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:03.867830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:46:03.867830 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:46:03.868294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:03.868294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:46:03.868294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:03.868294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:46:03.868294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:46:03.868294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:03.868294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:46:03.868294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:03.868294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:46:03.868721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:03.868721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:46:03.868721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:46:03.868721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:46:03.868721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:46:03.868721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:46:03.868721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:46:03.868721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:46:03.868721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:46:03.868721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:46:03.868721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:03.868721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:46:03.869100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:03.869100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:46:03.869100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:03.869100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:46:03.869100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:03.869100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:46:03.869100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:03.869100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:46:03.869100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:46:03.869100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:03.869100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:46:03.869100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:03.869100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:46:03.869100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:46:03.869514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:03.869514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:46:03.869514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:03.869514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:46:03.869863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:03.869863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:46:03.869863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:46:03.869863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:46:03.869863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:46:03.869863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:46:03.869863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:46:03.869863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:46:03.869863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:46:03.869863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:46:03.869863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:46:03.869863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:46:03.869863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:46:03.869863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:46:03.869863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:46:03.869863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:46:03.870307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:46:03.870307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:46:03.870307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:46:03.870307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:46:03.870307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:46:03.870307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:03.870307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:46:03.870307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:03.870307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:46:03.870307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:03.870307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:46:03.870307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:03.870307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:46:03.870307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:03.870307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:46:03.870307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:46:03.870307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:03.870307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:46:03.870805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:03.870805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:46:03.870805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:46:03.870805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:03.870805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:46:03.870805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:03.870805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:46:03.870805 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:46:03.870805 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:46:03.870805 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:46:03.871126 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:46:03.871126 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:46:03.871338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:03.871914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:46:03.871914 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:03.871914 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:46:03.871914 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:46:03.871914 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:03.871914 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:46:03.871914 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:03.871914 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:46:03.872719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:03.872719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:46:03.872719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:46:03.872719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:46:03.872719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:03.872719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:46:03.872719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:03.872719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:46:03.872719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:03.872719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:46:03.872719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:03.872719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:46:03.872719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:03.872719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:46:03.872719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:46:03.873154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:03.873154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:46:03.873154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:03.873154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:46:03.873154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:46:03.873154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:03.873154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:46:03.873154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:03.873154 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:46:03.873653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:03.873653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:46:03.873653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:46:03.873653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:46:03.873653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:46:03.873653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:46:03.873653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:46:03.873653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:46:03.873653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:46:03.873653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:03.873653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:46:03.874143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:03.874143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:46:03.874143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:03.874143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:46:03.874143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:03.874143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:46:03.874143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:03.874143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:46:03.874143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:46:03.874143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:03.874143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:46:03.874143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:03.874143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:46:03.874143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:46:03.874608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:03.874608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:46:03.874608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:03.874608 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:46:03.875015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:03.875015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:46:03.875015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:46:03.875015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:46:03.875015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:46:03.875015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:46:03.875015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:46:03.875015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:46:03.875015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:46:03.875015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:46:03.875015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:46:03.875015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:46:03.875015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:46:03.875015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:46:03.875015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:46:03.875015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:46:03.875015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:46:03.875512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:46:03.875512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:46:03.875512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:46:03.875512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:03.875512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:46:03.875512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:03.875512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:46:03.875512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:03.875512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:46:03.875512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:03.875512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:46:03.875512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:03.875512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:46:03.875512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:46:03.875512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:03.875512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:46:03.875512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:03.875512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:46:03.875512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:46:03.876446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:03.876446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:46:03.876446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:03.876446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:46:03.876446 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:46:03.876446 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:46:03.876446 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:46:03.876446 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:46:03.876927 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:46:03.876927 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:46:03.877049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:03.877857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:46:03.877857 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:03.877857 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:46:03.877857 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:46:03.877857 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:03.877857 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:46:03.877857 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:03.877857 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:46:03.878820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:03.878820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:46:03.878820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:46:03.878820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:46:03.878820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:46:03.878820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:46:03.878820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:46:03.878820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:03.878820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:46:03.878820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:03.878820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:46:03.878820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:03.879224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:46:03.879224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:03.879224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:46:03.879224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:03.879224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:46:03.879224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:46:03.879224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:03.879224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:46:03.879224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:03.879224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:46:03.879224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:46:03.879224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:03.879224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:46:03.879595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:03.879595 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:46:03.879842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:03.879842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:46:03.879842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:46:03.879842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:46:03.879842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:46:03.879842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:46:03.879842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:46:03.879842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:46:03.879842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:46:03.879842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:46:03.879842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:46:03.879842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:46:03.879842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:46:03.879842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:46:03.879842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:46:03.879842 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:46:03.880280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:46:03.880280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:46:03.880280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:03.880280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:46:03.880280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:03.880280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:46:03.880280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:03.880280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:46:03.880280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:03.880280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:46:03.880280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:03.880280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:46:03.880280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:46:03.880280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:03.880280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:46:03.880280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:03.880280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:46:03.880280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:46:03.880280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:03.880841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:46:03.880841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:03.880841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:46:08.670539 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:46:08.671892 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:46:08.671892 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:46:08.673315 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:46:08.707449 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:46:08.707728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:08.708264 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:46:08.708264 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:08.708264 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:46:08.708264 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:46:08.708264 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:08.708264 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:46:08.708264 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:08.708264 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:46:08.709067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:08.709067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:46:08.709067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:46:08.709067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:46:08.709067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:46:08.709067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:46:08.709067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:46:08.709067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:08.709067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:46:08.709067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:08.709067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:46:08.709067 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:08.709508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:46:08.709508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:08.709508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:46:08.709508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:08.709508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:46:08.709508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:46:08.709508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:08.709508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:46:08.709508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:08.709508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:46:08.709508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:46:08.709508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:08.709508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:46:08.710119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:08.710119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:46:08.710119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:08.710119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:46:08.710119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:46:08.710119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:46:08.710119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:46:08.710119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:46:08.710119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:46:08.710119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:46:08.710119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:46:08.710119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:46:08.710119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:46:08.710119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:46:08.710119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:46:08.711468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:46:08.711468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:46:08.711468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:46:08.711468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:46:08.711468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:46:08.711468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:08.711468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:46:08.711468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:08.711468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:46:08.711468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:08.711468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:46:08.711468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:08.711468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:46:08.711468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:08.711468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:46:08.711468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:46:08.711468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:08.711468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:46:08.711468 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:08.712061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:46:08.712061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:46:08.712061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:08.712061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:46:08.712061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:08.712061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:46:08.854887 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:46:08.855239 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:46:08.855441 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:46:08.856390 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:46:08.856574 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:46:08.856814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:08.857514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:46:08.857514 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:08.857514 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:46:08.857514 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:46:08.857514 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:08.857514 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:46:08.857514 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:08.857514 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:46:08.859108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:08.859108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:46:08.859108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:46:08.859108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:46:08.859108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:46:08.859108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:46:08.859108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:46:08.859108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:08.859108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:46:08.859108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:08.859108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:46:08.859108 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:08.859717 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:46:08.859717 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:08.859717 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:46:08.859717 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:08.859717 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:46:08.859717 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:46:08.859717 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:08.859717 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:46:08.859717 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:08.859717 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:46:08.859717 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:46:08.859717 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:08.859717 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:46:08.860362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:08.860362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:46:08.861002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:08.861002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:46:08.861002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:46:08.861002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:46:08.861002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:46:08.861002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:46:08.861002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:46:08.861002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:46:08.861002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:46:08.861002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:46:08.861002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:46:08.861002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:46:08.861002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:46:08.861002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:46:08.861002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:46:08.861002 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:46:08.861827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:46:08.861827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:46:08.861827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:08.861827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:46:08.861827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:08.861827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:46:08.861827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:08.861827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:46:08.861827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:08.861827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:46:08.861827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:08.861827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:46:08.861827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:46:08.861827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:08.861827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:46:08.861827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:08.861827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:46:08.861827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:46:08.861827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:08.862517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:46:08.862517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:08.862517 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:46:08.865437 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:46:08.865691 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:46:08.865899 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:46:08.866117 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:46:08.866668 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:46:08.867040 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:08.867820 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:46:08.867820 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:08.867820 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:46:08.867820 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:46:08.867820 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:08.867820 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:46:08.867820 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:08.867820 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:46:08.868927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:08.868927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:46:08.868927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:46:08.868927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:46:08.868927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:08.868927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:46:08.868927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:08.868927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:46:08.868927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:08.868927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:46:08.868927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:08.868927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:46:08.868927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:08.868927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:46:08.868927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:46:08.869778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:08.869778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:46:08.869778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:08.869778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:46:08.869778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:46:08.869778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:08.869778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:46:08.869778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:08.869778 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:46:08.870351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:08.870351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:46:08.870351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:46:08.870351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:46:08.870351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:46:08.870351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:46:08.870351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:46:08.870351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:46:08.870351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:46:08.870351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:46:08.870351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:08.870351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:46:08.871028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:08.871028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:46:08.871028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:08.871028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:46:08.871028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:08.871028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:46:08.871028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:08.871028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:46:08.871028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:46:08.871028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:08.871028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:46:08.871028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:08.871028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:46:08.871028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:46:08.871638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:08.871638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:46:08.871638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:08.871638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:46:08.871862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:08.871862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:46:08.871862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:46:08.871862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:46:08.871862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:46:08.871862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:46:08.871862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:46:08.871862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:46:08.871862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:46:08.871862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:46:08.871862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:46:08.871862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:46:08.871862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:46:08.871862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:46:08.871862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:46:08.871862 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:46:08.872566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:46:08.872566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:46:08.872566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:46:08.872566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:46:08.872566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:46:08.872566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:08.872566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:46:08.872566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:08.872566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:46:08.872566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:08.872566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:46:08.872566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:08.872566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:46:08.872566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:08.872566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:46:08.872566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:46:08.872566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:08.872566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:46:08.873458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:08.873458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:46:08.873458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:46:08.873458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:08.873458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:46:08.873458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:08.873458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:46:08.873458 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:46:08.873458 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:46:08.873458 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:46:08.874008 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:46:08.874008 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:46:08.874008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:08.874368 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:46:08.874368 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:08.874368 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:46:08.874368 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:46:08.874368 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:08.874368 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:46:08.874368 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:08.874368 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:46:08.875080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:08.875080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:46:08.875080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:46:08.875080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:46:08.875080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:08.875080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:46:08.875080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:08.875080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:46:08.875080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:08.875080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:46:08.875080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:08.875080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:46:08.875080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:08.875080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:46:08.875080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:46:08.875508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:08.875508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:46:08.875508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:08.875508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:46:08.875508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:46:08.875508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:08.875508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:46:08.875508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:08.875508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:46:08.876230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:08.876230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:46:08.876230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:46:08.876230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:46:08.876230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:46:08.876230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:46:08.876230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:46:08.876230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:46:08.876230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:46:08.876230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:08.876230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:46:08.876568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:08.876568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:46:08.876568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:08.876568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:46:08.876568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:08.876568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:46:08.876568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:08.876568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:46:08.876568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:46:08.876568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:08.876568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:46:08.876568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:08.876568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:46:08.876568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:46:08.877013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:08.877013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:46:08.877013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:08.877013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:46:08.877013 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:46:08.877910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:08.877910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:46:08.877910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:46:08.877910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:46:08.877910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:46:08.877910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:46:08.877910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:46:08.877910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:46:08.877910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:46:08.877910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:46:08.877910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:46:08.877910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:46:08.877910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:46:08.877910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:46:08.877910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:46:08.877910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:46:08.877910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:46:08.878506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:46:08.878506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:46:08.878506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:46:08.878506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:08.878506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:46:08.878506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:08.878506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:46:08.878506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:08.878506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:46:08.878506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:08.878506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:46:08.878506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:08.878506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:46:08.878506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:46:08.878506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:08.878506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:46:08.878506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:08.878506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:46:08.878506 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:46:08.879043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:08.879043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:46:08.879043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:08.879043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:46:08.879043 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:46:08.879043 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:46:08.879043 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:46:08.879043 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:46:08.879317 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:46:08.879317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:08.879762 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:46:08.879762 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:08.879762 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:46:08.879762 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:46:08.879762 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:08.879762 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:46:08.879762 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:08.879762 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:46:08.880927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:08.880927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:46:08.880927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:46:08.880927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:46:08.880927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:46:08.880927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:46:08.880927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:46:08.880927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:08.880927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:46:08.880927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:08.880927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:46:08.880927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:08.881330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:46:08.881330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:08.881330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:46:08.881330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:08.881330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:46:08.881330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:46:08.881330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:08.881330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:46:08.881330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:08.881330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:46:08.881330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:46:08.881330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:08.881330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:46:08.881697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:08.881697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:46:08.882120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:08.882120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:46:08.882120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:46:08.882120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:46:08.882120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:46:08.882120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:46:08.882120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:46:08.882120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:46:08.882120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:46:08.882120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:46:08.882120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:46:08.882120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:46:08.882120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:46:08.882120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:46:08.882120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:46:08.882120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:46:08.882573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:46:08.882573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:46:08.882573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:08.882573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:46:08.882573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:08.882573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:46:08.882573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:08.882573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:46:08.882573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:08.882573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:46:08.882573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:08.882573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:46:08.882573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:46:08.882573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:08.882573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:46:08.882573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:08.882573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:46:08.882573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:46:08.882573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:08.883145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:46:08.883145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:08.883145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:46:13.672604 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:46:13.674344 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:46:13.674344 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:46:13.674535 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:46:13.712622 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:46:13.712866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:13.713520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:46:13.713520 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:13.713520 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:46:13.713520 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:46:13.713520 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:13.713520 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:46:13.713520 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:13.713520 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:46:13.714316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:13.714316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:46:13.714316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:46:13.714316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:46:13.714316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:46:13.714316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:46:13.714316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:46:13.714316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:13.714316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:46:13.714316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:13.714316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:46:13.714316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:13.714911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:46:13.714911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:13.714911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:46:13.714911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:13.714911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:46:13.714911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:46:13.714911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:13.714911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:46:13.714911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:13.714911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:46:13.714911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:46:13.714911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:13.714911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:46:13.715457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:13.715457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:46:13.715457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:13.715457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:46:13.715457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:46:13.715457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:46:13.715457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:46:13.715457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:46:13.715457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:46:13.715457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:46:13.715457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:46:13.715457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:46:13.715457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:46:13.715457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:46:13.715457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:46:13.716270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:46:13.716270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:46:13.716270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:46:13.716270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:46:13.716270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:46:13.716270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:13.716270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:46:13.716270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:13.716270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:46:13.716270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:13.716270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:46:13.716270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:13.716270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:46:13.716270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:13.716270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:46:13.716270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:46:13.716270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:13.716270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:46:13.716270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:13.717081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:46:13.717081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:46:13.717081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:13.717081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:46:13.717081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:13.717081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:46:13.855662 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:46:13.856392 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:46:13.856590 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:46:13.858504 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:46:13.858721 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:46:13.858906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:13.859648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:46:13.859648 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:13.859648 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:46:13.859648 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:46:13.859648 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:13.859648 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:46:13.859648 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:13.859648 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:46:13.861666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:13.861666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:46:13.861666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:46:13.861666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:46:13.861666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:46:13.861666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:46:13.861666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:46:13.861666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:13.861666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:46:13.861666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:13.861666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:46:13.861666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:13.862303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:46:13.862303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:13.862303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:46:13.862303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:13.862303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:46:13.862303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:46:13.862303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:13.862303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:46:13.862303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:13.862303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:46:13.862303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:46:13.862303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:13.862303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:46:13.862927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:13.862927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:46:13.863525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:13.863525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:46:13.863525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:46:13.863525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:46:13.863525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:46:13.863525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:46:13.863525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:46:13.863525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:46:13.863525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:46:13.863525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:46:13.863525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:46:13.863525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:46:13.863525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:46:13.863525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:46:13.863525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:46:13.863525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:46:13.864500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:46:13.864500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:46:13.864500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:13.864500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:46:13.864500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:13.864500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:46:13.864500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:13.864500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:46:13.864500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:13.864500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:46:13.864500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:13.864500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:46:13.864500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:46:13.864500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:13.864500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:46:13.864500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:13.864500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:46:13.864500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:46:13.864500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:13.865603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:46:13.865603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:13.865603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:46:13.865979 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:46:13.866299 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:46:13.866404 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:46:13.866531 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:46:13.867956 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:46:13.868116 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:13.868808 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:46:13.868808 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:13.868808 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:46:13.868808 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:46:13.868808 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:13.868808 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:46:13.868808 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:13.868808 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:46:13.869568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:13.869568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:46:13.869568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:46:13.869568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:46:13.869568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:13.869568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:46:13.869568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:13.869568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:46:13.869568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:13.869568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:46:13.869568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:13.869568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:46:13.869568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:13.869568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:46:13.869568 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:46:13.870047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:13.870047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:46:13.870047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:13.870047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:46:13.870047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:46:13.870047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:13.870047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:46:13.870047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:13.870047 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:46:13.870456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:13.870456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:46:13.870456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:46:13.870456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:46:13.870456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:46:13.870456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:46:13.870456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:46:13.870456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:46:13.870456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:46:13.870456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:46:13.870456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:13.870456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:46:13.870861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:13.870861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:46:13.870861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:13.870861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:46:13.870861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:13.870861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:46:13.870861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:13.870861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:46:13.870861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:46:13.870861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:13.870861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:46:13.870861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:13.870861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:46:13.870861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:46:13.871263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:13.871263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:46:13.871263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:13.871263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:46:13.871576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:13.871576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:46:13.871576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:46:13.871576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:46:13.871576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:46:13.871576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:46:13.871576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:46:13.871576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:46:13.871576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:46:13.871576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:46:13.871576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:46:13.871576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:46:13.871576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:46:13.871576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:46:13.871576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:46:13.871576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:46:13.872058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:46:13.872058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:46:13.872058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:46:13.872058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:46:13.872058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:46:13.872058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:13.872058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:46:13.872058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:13.872058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:46:13.872058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:13.872058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:46:13.872058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:13.872058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:46:13.872058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:13.872058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:46:13.872058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:46:13.872058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:13.872058 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:46:13.872633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:13.872633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:46:13.872633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:46:13.872633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:13.872633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:46:13.872633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:13.872633 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:46:13.872633 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:46:13.872983 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:46:13.872983 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:46:13.873058 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:46:13.873428 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:46:13.873667 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:13.874237 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:46:13.874237 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:13.874237 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:46:13.874237 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:46:13.874237 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:13.874237 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:46:13.874237 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:13.874237 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:46:13.875051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:13.875051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:46:13.875051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:46:13.875051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:46:13.875051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:13.875051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:46:13.875051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:13.875051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:46:13.875051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:13.875051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:46:13.875051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:13.875051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:46:13.875051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:13.875051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:46:13.875051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:46:13.875596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:13.875596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:46:13.875596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:13.875596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:46:13.875596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:46:13.875596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:13.875596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:46:13.875596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:13.875596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:46:13.875995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:13.875995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:46:13.875995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:46:13.875995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:46:13.875995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:46:13.875995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:46:13.875995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:46:13.875995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:46:13.875995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:46:13.875995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:13.875995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:46:13.876481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:13.876481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:46:13.876481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:13.876481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:46:13.876481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:13.876481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:46:13.876481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:13.876481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:46:13.876481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:46:13.876481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:13.876481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:46:13.876481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:13.876481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:46:13.876481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:46:13.876918 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:13.876918 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:46:13.876918 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:13.876918 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:46:13.877297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:13.877297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:46:13.877297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:46:13.877297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:46:13.877297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:46:13.877297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:46:13.877297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:46:13.877297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:46:13.877297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:46:13.877297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:46:13.877297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:46:13.877297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:46:13.877297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:46:13.877297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:46:13.877297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:46:13.877297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:46:13.877297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:46:13.877781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:46:13.877781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:46:13.877781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:46:13.877781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:13.877781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:46:13.877781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:13.877781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:46:13.877781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:13.877781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:46:13.877781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:13.877781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:46:13.877781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:13.877781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:46:13.877781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:46:13.877781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:13.877781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:46:13.877781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:13.877781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:46:13.877781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:46:13.878244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:13.878244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:46:13.878244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:13.878244 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:46:13.878417 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:46:13.878759 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:46:13.878885 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:46:13.879064 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:46:13.879377 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:46:13.879599 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:46:13.879753 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:13.880316 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:46:13.880316 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:13.880316 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:46:13.880316 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:46:13.880316 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:13.880316 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:46:13.880316 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:13.880316 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:46:13.881351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:13.881351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:46:13.881351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:46:13.881351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:46:13.881351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:46:13.881351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:46:13.881351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:46:13.881351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:13.881351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:46:13.881351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:13.881351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:46:13.881351 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:13.881696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:46:13.881696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:13.881696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:46:13.881696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:13.881696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:46:13.881696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:46:13.881696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:13.881696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:46:13.881696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:13.881696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:46:13.881696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:46:13.881696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:13.881696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:46:13.882112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:13.882112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:46:13.882425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:13.882425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:46:13.882425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:46:13.882425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:46:13.882425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:46:13.882425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:46:13.882425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:46:13.882425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:46:13.882425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:46:13.882425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:46:13.882425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:46:13.882425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:46:13.882425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:46:13.882425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:46:13.882425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:46:13.882425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:46:13.882869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:46:13.882869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:46:13.882869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:13.882869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:46:13.882869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:13.882869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:46:13.882869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:13.882869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:46:13.882869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:13.882869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:46:13.882869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:13.882869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:46:13.882869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:46:13.882869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:13.882869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:46:13.882869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:13.882869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:46:13.882869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:46:13.882869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:13.884117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:46:13.884117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:13.884117 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:46:18.676099 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:46:18.676099 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:46:18.676099 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:46:18.676099 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:46:18.714762 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:46:18.715145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:18.715873 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:46:18.715873 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:18.715873 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:46:18.715873 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:46:18.715873 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:18.715873 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:46:18.715873 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:18.715873 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:46:18.716691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:18.716691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:46:18.716691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:46:18.716691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:46:18.716691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:46:18.716691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:46:18.716691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:46:18.716691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:18.716691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:46:18.716691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:18.716691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:46:18.716691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:18.717683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:46:18.717683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:18.717683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:46:18.717683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:18.717683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:46:18.717683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:46:18.717683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:18.717683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:46:18.717683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:18.717683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:46:18.717683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:46:18.717683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:18.717683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:46:18.718132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:18.718132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:46:18.718132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:18.718132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:46:18.718132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:46:18.718132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:46:18.718132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:46:18.718132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:46:18.718132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:46:18.718132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:46:18.718132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:46:18.718132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:46:18.718132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:46:18.718132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:46:18.718132 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:46:18.718615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:46:18.718615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:46:18.718615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:46:18.718615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:46:18.718615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:46:18.718615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:18.718615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:46:18.718615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:18.718615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:46:18.718615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:18.718615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:46:18.718615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:18.718615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:46:18.718615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:18.718615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:46:18.718615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:46:18.718615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:18.718615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:46:18.718615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:18.719208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:46:18.719208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:46:18.719208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:18.719208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:46:18.719208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:18.719208 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:46:18.857300 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:46:18.857495 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:46:18.857606 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:46:18.858784 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:46:18.859030 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:46:18.859205 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:18.860399 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:46:18.860399 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:18.860399 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:46:18.860399 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:46:18.860399 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:18.860399 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:46:18.860399 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:18.860399 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:46:18.861187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:18.861187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:46:18.861187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:46:18.861187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:46:18.861187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:46:18.861187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:46:18.861187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:46:18.861187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:18.861187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:46:18.861187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:18.861187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:46:18.861187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:18.861632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:46:18.861632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:18.861632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:46:18.861632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:18.861632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:46:18.861632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:46:18.861632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:18.861632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:46:18.861632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:18.861632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:46:18.861632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:46:18.861632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:18.861632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:46:18.862004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:18.862004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:46:18.862256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:18.862256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:46:18.862256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:46:18.862256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:46:18.862256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:46:18.862256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:46:18.862256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:46:18.862256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:46:18.862256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:46:18.862256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:46:18.862256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:46:18.862256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:46:18.862256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:46:18.862256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:46:18.862256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:46:18.862256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:46:18.862725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:46:18.862725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:46:18.862725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:18.862725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:46:18.862725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:18.862725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:46:18.862725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:18.862725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:46:18.862725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:18.862725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:46:18.862725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:18.862725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:46:18.862725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:46:18.862725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:18.862725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:46:18.862725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:18.862725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:46:18.862725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:46:18.862725 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:18.863312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:46:18.863312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:18.863312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:46:18.866515 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:46:18.866996 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:46:18.867187 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:46:18.867385 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:46:18.867812 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:46:18.867981 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:18.868541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:46:18.868541 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:18.868541 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:46:18.868541 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:46:18.868541 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:18.868541 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:46:18.868541 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:18.868541 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:46:18.869643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:18.869643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:46:18.869643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:46:18.869643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:46:18.869643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:18.869643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:46:18.869643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:18.869643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:46:18.869643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:18.869643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:46:18.869643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:18.869643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:46:18.869643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:18.869643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:46:18.869643 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:46:18.870215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:18.870215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:46:18.870215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:18.870215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:46:18.870215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:46:18.870215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:18.870215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:46:18.870215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:18.870215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:46:18.870526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:18.870526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:46:18.870526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:46:18.870526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:46:18.870526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:46:18.870526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:46:18.870526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:46:18.870526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:46:18.870526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:46:18.870526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:46:18.870526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:18.870526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:46:18.870973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:18.870973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:46:18.870973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:18.870973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:46:18.870973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:18.870973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:46:18.870973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:18.870973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:46:18.870973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:46:18.870973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:18.870973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:46:18.870973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:18.870973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:46:18.870973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:46:18.871378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:18.871378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:46:18.871378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:18.871378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:46:18.871691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:18.871691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:46:18.871691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:46:18.871691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:46:18.871691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:46:18.871691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:46:18.871691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:46:18.871691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:46:18.871691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:46:18.871691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:46:18.871691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:46:18.871691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:46:18.871691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:46:18.871691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:46:18.871691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:46:18.871691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:46:18.872215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:46:18.872215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:46:18.872215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:46:18.872215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:46:18.872215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:46:18.872215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:18.872215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:46:18.872215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:18.872215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:46:18.872215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:18.872215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:46:18.872215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:18.872215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:46:18.872215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:18.872215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:46:18.872215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:46:18.872215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:18.872215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:46:18.872818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:18.872818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:46:18.872818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:46:18.872818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:18.872818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:46:18.872818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:18.872818 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:46:18.873034 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:46:18.873201 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:46:18.873311 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:46:18.873469 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:46:18.873906 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:46:18.874099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:18.874596 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:46:18.874596 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:18.874596 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:46:18.874596 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:46:18.874596 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:18.874596 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:46:18.874596 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:18.874596 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:46:18.875400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:18.875400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:46:18.875400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:46:18.875400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:46:18.875400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:18.875400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:46:18.875400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:18.875400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:46:18.875400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:18.875400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:46:18.875400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:18.875400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:46:18.875400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:18.875400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:46:18.875400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:46:18.875897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:18.875897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:46:18.875897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:18.875897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:46:18.875897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:46:18.875897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:18.875897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:46:18.875897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:18.875897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:46:18.876310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:18.876310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:46:18.876310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:46:18.876310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:46:18.876310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:46:18.876310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:46:18.876310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:46:18.876310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:46:18.876310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:46:18.876310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:18.876310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:46:18.876685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:18.876685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:46:18.876685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:18.876685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:46:18.876685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:18.876685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:46:18.876685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:18.876685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:46:18.876685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:46:18.876685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:18.876685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:46:18.876685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:18.876685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:46:18.876685 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:46:18.877120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:18.877120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:46:18.877120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:18.877120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:46:18.877461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:18.877461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:46:18.877461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:46:18.877461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:46:18.877461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:46:18.877461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:46:18.877461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:46:18.877461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:46:18.877461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:46:18.877461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:46:18.877461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:46:18.877461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:46:18.877461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:46:18.877461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:46:18.877461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:46:18.877461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:46:18.877461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:46:18.878027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:46:18.878027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:46:18.878027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:46:18.878027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:18.878027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:46:18.878027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:18.878027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:46:18.878027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:18.878027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:46:18.878027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:18.878027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:46:18.878027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:18.878027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:46:18.878027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:46:18.878027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:18.878027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:46:18.878027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:18.878027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:46:18.878027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:46:18.878570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:18.878570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:46:18.878570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:18.878570 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:46:18.878889 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:46:18.879227 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:46:18.879339 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:46:18.879547 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:46:18.880177 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:46:18.880533 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:46:18.880803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:18.881523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:46:18.881523 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:18.881523 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:46:18.881523 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:46:18.881523 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:18.881523 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:46:18.881523 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:18.881523 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:46:18.882510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:18.882510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:46:18.882510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:46:18.882510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:46:18.882510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:46:18.882510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:46:18.882510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:46:18.882510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:18.882510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:46:18.882510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:18.882510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:46:18.882510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:18.882780 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:46:18.882780 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:18.882780 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:46:18.882780 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:18.882780 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:46:18.882780 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:46:18.882780 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:18.882780 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:46:18.882780 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:18.882780 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:46:18.882780 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:46:18.882780 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:18.882780 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:46:18.883042 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:18.883042 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:46:18.883534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:18.883534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:46:18.883534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:46:18.883534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:46:18.883534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:46:18.883534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:46:18.883534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:46:18.883534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:46:18.883534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:46:18.883534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:46:18.883534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:46:18.883534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:46:18.883534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:46:18.883534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:46:18.883534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:46:18.883534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:46:18.883866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:46:18.883866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:46:18.883866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:18.883866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:46:18.883866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:18.883866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:46:18.883866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:18.883866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:46:18.883866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:18.883866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:46:18.883866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:18.883866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:46:18.883866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:46:18.883866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:18.883866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:46:18.883866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:18.883866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:46:18.883866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:46:18.883866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:18.884227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:46:18.884227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:18.884227 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:46:23.674505 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:46:23.675190 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:46:23.675374 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:46:23.676780 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:46:23.712659 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:46:23.713024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:23.713566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:46:23.713566 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:23.713566 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:46:23.713566 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:46:23.713566 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:23.713566 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:46:23.713566 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:23.713566 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:46:23.714519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:23.714519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:46:23.714519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:46:23.714519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:46:23.714519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:46:23.714519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:46:23.714519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:46:23.714519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:23.714519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:46:23.714519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:23.714519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:46:23.714519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:23.715480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:46:23.715480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:23.715480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:46:23.715480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:23.715480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:46:23.715480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:46:23.715480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:23.715480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:46:23.715480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:23.715480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:46:23.715480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:46:23.715480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:23.715480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:46:23.716390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:23.716390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:46:23.716390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:23.716390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:46:23.716390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:46:23.716390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:46:23.716390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:46:23.716390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:46:23.716390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:46:23.716390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:46:23.716390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:46:23.716390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:46:23.716390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:46:23.716390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:46:23.716390 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:46:23.717692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:46:23.717692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:46:23.717692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:46:23.717692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:46:23.717692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:46:23.717692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:23.717692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:46:23.717692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:23.717692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:46:23.717692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:23.717692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:46:23.717692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:23.717692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:46:23.717692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:23.717692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:46:23.717692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:46:23.717692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:23.717692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:46:23.717692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:23.719606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:46:23.719606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:46:23.719606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:23.719606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:46:23.719606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:23.719606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:46:23.857501 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:46:23.858189 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:46:23.858384 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:46:23.859510 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:46:23.859766 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:46:23.859939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:23.860637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:46:23.860637 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:23.860637 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:46:23.860637 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:46:23.860637 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:23.860637 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:46:23.860637 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:23.860637 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:46:23.862299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:23.862299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:46:23.862299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:46:23.862299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:46:23.862299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:46:23.862299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:46:23.862299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:46:23.862299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:23.862299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:46:23.862299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:23.862299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:46:23.862299 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:23.863378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:46:23.863378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:23.863378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:46:23.863378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:23.863378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:46:23.863378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:46:23.863378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:23.863378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:46:23.863378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:23.863378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:46:23.863378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:46:23.863378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:23.863378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:46:23.863957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:23.863957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:46:23.864214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:23.864214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:46:23.864214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:46:23.864214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:46:23.864214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:46:23.864214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:46:23.864214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:46:23.864214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:46:23.864214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:46:23.864214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:46:23.864214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:46:23.864214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:46:23.864214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:46:23.864214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:46:23.864214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:46:23.864214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:46:23.865534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:46:23.865534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:46:23.865534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:23.865534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:46:23.865534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:23.865534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:46:23.865534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:23.865534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:46:23.865534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:23.865534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:46:23.865534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:23.865534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:46:23.865534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:46:23.865534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:23.865534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:46:23.865534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:23.865534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:46:23.865534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:46:23.865534 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:23.866922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:46:23.866922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:23.866922 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:46:23.866922 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:46:23.867494 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:46:23.867592 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:46:23.867714 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:46:23.868171 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:46:23.868472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:23.869407 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:46:23.869407 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:23.869407 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:46:23.869407 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:46:23.869407 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:23.869407 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:46:23.869407 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:23.869407 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:46:23.870361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:23.870361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:46:23.870361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:46:23.870361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:46:23.870361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:23.870361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:46:23.870361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:23.870361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:46:23.870361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:23.870361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:46:23.870361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:23.870361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:46:23.870361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:23.870361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:46:23.870361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:46:23.871463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:23.871463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:46:23.871463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:23.871463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:46:23.871463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:46:23.871463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:23.871463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:46:23.871463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:23.871463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:46:23.871463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:23.871463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:46:23.871463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:46:23.871463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:46:23.871463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:46:23.871917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:46:23.871917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:46:23.871917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:46:23.871917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:46:23.871917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:46:23.871917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:23.871917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:46:23.871917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:23.871917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:46:23.871917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:23.871917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:46:23.871917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:23.871917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:46:23.872263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:23.872263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:46:23.872263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:46:23.872263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:23.872263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:46:23.872263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:23.872263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:46:23.872263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:46:23.872263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:23.872263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:46:23.872263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:23.872263 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:46:23.872558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:23.872558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:46:23.872558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:46:23.872558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:46:23.872558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:46:23.872558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:46:23.872558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:46:23.872558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:46:23.872558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:46:23.872558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:46:23.872558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:46:23.872558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:46:23.872558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:46:23.872558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:46:23.872558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:46:23.872558 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:46:23.872995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:46:23.872995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:46:23.872995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:46:23.872995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:46:23.872995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:46:23.872995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:23.872995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:46:23.872995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:23.872995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:46:23.872995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:23.872995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:46:23.872995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:23.872995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:46:23.872995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:23.872995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:46:23.872995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:46:23.872995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:23.872995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:46:23.873475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:23.873475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:46:23.873475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:46:23.873475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:23.873475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:46:23.873475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:23.873475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:46:23.873475 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:46:23.877076 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:46:23.877276 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:46:23.877509 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:46:23.878125 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:46:23.878493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:23.879397 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:46:23.879840 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:46:23.879840 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:23.879840 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:46:23.879840 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:46:23.879840 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:23.879840 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:46:23.879840 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:23.879840 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:46:23.881190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:23.881190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:46:23.881190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:46:23.881190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:46:23.881190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:23.881190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:46:23.881190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:23.881190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:46:23.881190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:23.881190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:46:23.881190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:23.881190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:46:23.881190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:23.881190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:46:23.881190 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:46:23.881580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:23.881580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:46:23.881580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:23.881580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:46:23.881580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:46:23.881580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:23.881580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:46:23.881580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:23.881580 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:46:23.882634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:23.882634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:46:23.882634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:46:23.882634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:46:23.882634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:46:23.882634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:46:23.882634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:46:23.882634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:46:23.882634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:46:23.882634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:23.882634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:46:23.883375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:23.883375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:46:23.883375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:23.883375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:46:23.883375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:23.883375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:46:23.883375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:23.883375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:46:23.883375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:46:23.883375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:23.883375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:46:23.883375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:23.883375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:46:23.883375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:46:23.884338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:23.884338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:46:23.884338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:23.884338 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:46:23.885577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:23.885577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:46:23.885577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:46:23.885577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:46:23.885577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:46:23.885577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:46:23.885577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:46:23.885577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:46:23.885577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:46:23.885577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:46:23.885577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:46:23.885577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:46:23.885577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:46:23.885577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:46:23.885577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:46:23.885577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:46:23.885577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:46:23.886458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:46:23.886458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:46:23.886458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:46:23.886458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:23.886458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:46:23.886458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:23.886458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:46:23.886458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:23.886458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:46:23.886458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:23.886458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:46:23.886458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:23.886458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:46:23.886458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:46:23.886458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:23.886458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:46:23.886458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:23.886458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:46:23.886458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:46:23.887632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:23.887632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:46:23.887632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:23.887632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:46:23.887632 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:46:23.887632 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:46:23.887632 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:46:23.887632 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:46:23.888157 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:46:23.888157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:23.888157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:46:23.888157 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:23.888157 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:46:23.888157 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:46:23.888157 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:23.888157 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:46:23.888157 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:23.888157 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:46:23.888157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:23.888157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:46:23.888157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:46:23.888157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:46:23.888975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:46:23.888975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:46:23.888975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:46:23.888975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:23.888975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:46:23.888975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:23.888975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:46:23.888975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:23.888975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:46:23.888975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:23.888975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:46:23.888975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:23.888975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:46:23.888975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:46:23.889775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:23.889775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:46:23.889775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:23.889775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:46:23.889775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:46:23.889775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:23.889775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:46:23.889775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:23.889775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:46:23.889775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:23.889775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:46:23.889775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:46:23.890122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:46:23.890122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:46:23.890122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:46:23.890122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:46:23.890122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:46:23.890122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:46:23.890122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:46:23.890122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:46:23.890122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:46:23.890122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:46:23.890122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:46:23.890122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:46:23.890122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:46:23.890122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:46:23.890122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:46:23.890122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:23.890122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:46:23.890122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:23.890122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:46:23.890122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:23.890615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:46:23.890615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:23.890615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:46:23.890615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:23.890615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:46:23.890615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:46:23.890615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:23.890615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:46:23.890615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:23.890615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:46:23.890615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:46:23.890615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:23.890615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:46:23.890615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:23.890615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:46:28.675090 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:46:28.675740 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:46:28.676096 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:46:28.677253 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:46:28.714962 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:46:28.715230 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:28.715985 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:46:28.715985 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:28.715985 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:46:28.715985 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:46:28.715985 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:28.715985 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:46:28.715985 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:28.715985 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:46:28.716737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:28.716737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:46:28.716737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:46:28.716737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:46:28.716737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:46:28.716737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:46:28.716737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:46:28.716737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:28.716737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:46:28.716737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:28.716737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:46:28.716737 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:28.717241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:46:28.717241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:28.717241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:46:28.717241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:28.717241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:46:28.717241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:46:28.717241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:28.717241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:46:28.717241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:28.717241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:46:28.717241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:46:28.717241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:28.717241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:46:28.717895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:28.717895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:46:28.717895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:28.717895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:46:28.717895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:46:28.717895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:46:28.717895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:46:28.717895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:46:28.717895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:46:28.717895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:46:28.717895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:46:28.717895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:46:28.717895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:46:28.717895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:46:28.717895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:46:28.718423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:46:28.718423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:46:28.718423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:46:28.718423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:46:28.718423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:46:28.718423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:28.718423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:46:28.718423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:28.718423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:46:28.718423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:28.718423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:46:28.718423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:28.718423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:46:28.718423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:28.718423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:46:28.718423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:46:28.718423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:28.718423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:46:28.718423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:28.719271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:46:28.719271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:46:28.719271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:28.719271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:46:28.719271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:28.719271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:46:28.858478 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:46:28.858974 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:46:28.859087 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:46:28.860239 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:46:28.860584 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:46:28.860993 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:28.861770 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:46:28.861770 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:28.861770 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:46:28.861770 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:46:28.861770 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:28.861770 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:46:28.861770 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:28.861770 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:46:28.862895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:28.862895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:46:28.862895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:46:28.862895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:46:28.862895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:46:28.862895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:46:28.862895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:46:28.862895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:28.862895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:46:28.862895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:28.862895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:46:28.862895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:28.863481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:46:28.863481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:28.863481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:46:28.863481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:28.863481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:46:28.863481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:46:28.863481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:28.863481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:46:28.863481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:28.863481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:46:28.863481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:46:28.863481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:28.863481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:46:28.864176 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:28.864176 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:46:28.864176 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:28.864176 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:46:28.864176 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:46:28.864176 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:46:28.864176 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:46:28.864176 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:46:28.864176 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:46:28.864176 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:46:28.864176 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:46:28.864176 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:46:28.864176 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:46:28.864176 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:46:28.864176 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:46:28.864942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:46:28.864942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:46:28.864942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:46:28.864942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:46:28.864942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:46:28.864942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:28.864942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:46:28.864942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:28.864942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:46:28.864942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:28.864942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:46:28.864942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:28.864942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:46:28.864942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:28.864942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:46:28.864942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:46:28.864942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:28.864942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:46:28.864942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:28.866493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:46:28.866493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:46:28.866493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:28.866493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:46:28.866493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:28.866493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:46:28.867530 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:46:28.867908 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:46:28.868043 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:46:28.868179 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:46:28.868758 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:46:28.869122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:28.869882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:46:28.869882 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:28.869882 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:46:28.869882 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:46:28.869882 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:28.869882 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:46:28.869882 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:28.869882 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:46:28.870821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:28.870821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:46:28.870821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:46:28.870821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:46:28.870821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:28.870821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:46:28.870821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:28.870821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:46:28.870821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:28.870821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:46:28.870821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:28.870821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:46:28.870821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:28.870821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:46:28.870821 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:46:28.871343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:28.871343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:46:28.871343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:28.871343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:46:28.871343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:46:28.871343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:28.871343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:46:28.871343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:28.871343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:46:28.872320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:28.872320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:46:28.872320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:46:28.872320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:46:28.872320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:46:28.872320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:46:28.872320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:46:28.872320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:46:28.872320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:46:28.872320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:46:28.872320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:28.872320 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:46:28.872855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:28.872855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:46:28.872855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:28.872855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:46:28.872855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:28.872855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:46:28.872855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:28.872855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:46:28.872855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:46:28.872855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:28.872855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:46:28.872855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:28.872855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:46:28.872855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:46:28.873242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:28.873242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:46:28.873242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:28.873242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:46:28.874025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:28.874025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:46:28.874025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:46:28.874025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:46:28.874025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:46:28.874025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:46:28.874025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:46:28.874025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:46:28.874025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:46:28.874025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:46:28.874025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:46:28.874025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:46:28.874025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:46:28.874025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:46:28.874025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:46:28.874025 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:46:28.874562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:46:28.874562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:46:28.874562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:46:28.874562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:46:28.874562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:46:28.874562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:28.874562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:46:28.874562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:28.874562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:46:28.874562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:28.874562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:46:28.874562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:28.874562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:46:28.874562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:28.874562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:46:28.874562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:46:28.874562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:28.874562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:46:28.875433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:28.875433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:46:28.875433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:46:28.875433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:28.875433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:46:28.875433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:28.875433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:46:28.875433 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:46:28.875433 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:46:28.876902 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:46:28.876902 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:46:28.876902 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:46:28.876902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:28.876902 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:46:28.876902 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:28.876902 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:46:28.876902 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:46:28.876902 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:28.876902 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:46:28.876902 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:28.876902 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:46:28.877562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:28.877562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:46:28.877562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:46:28.877562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:46:28.877562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:28.877562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:46:28.877562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:28.877562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:46:28.877562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:28.877562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:46:28.877562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:28.877562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:46:28.877562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:28.877562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:46:28.877562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:46:28.879016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:28.879016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:46:28.879016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:28.879016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:46:28.879016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:46:28.879016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:28.879016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:46:28.879016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:28.879016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:46:28.879016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:28.879016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:46:28.879016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:46:28.879016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:46:28.879016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:46:28.879527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:46:28.879527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:46:28.879527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:46:28.879527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:46:28.879527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:28.879527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:46:28.879527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:28.879527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:46:28.879527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:28.879527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:46:28.879527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:28.879527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:46:28.879527 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:28.879963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:46:28.879963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:46:28.879963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:28.879963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:46:28.879963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:28.879963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:46:28.879963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:46:28.879963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:28.879963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:46:28.879963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:28.879963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:46:28.879963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:28.880317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:46:28.880317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:46:28.880317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:46:28.880317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:46:28.880317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:46:28.880317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:46:28.880317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:46:28.880317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:46:28.880317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:46:28.880317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:46:28.880317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:46:28.880317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:46:28.880317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:46:28.880317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:46:28.880317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:46:28.880317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:46:28.880317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:46:28.880317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:46:28.880317 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:46:28.880879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:28.880879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:46:28.880879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:28.880879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:46:28.880879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:28.880879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:46:28.880879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:28.880879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:46:28.880879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:28.880879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:46:28.880879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:46:28.880879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:28.880879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:46:28.880879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:28.880879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:46:28.880879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:46:28.880879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:28.880879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:46:28.880879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:28.880879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:46:28.885885 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:46:28.886154 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:46:28.886266 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:46:28.886451 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:46:28.887094 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:46:28.887401 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:46:28.887592 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:28.888210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:46:28.888210 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:28.888210 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:46:28.888210 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:46:28.888210 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:28.888210 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:46:28.888210 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:28.888210 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:46:28.889068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:28.889068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:46:28.889068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:46:28.889068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:46:28.889068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:46:28.889068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:46:28.889068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:46:28.889068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:28.889068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:46:28.889068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:28.889068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:46:28.889068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:28.889901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:46:28.889901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:28.889901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:46:28.889901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:28.889901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:46:28.889901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:46:28.889901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:28.889901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:46:28.889901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:28.889901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:46:28.889901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:46:28.889901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:28.889901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:46:28.890589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:28.890589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:46:28.890589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:28.890589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:46:28.890589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:46:28.890589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:46:28.890589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:46:28.890589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:46:28.890589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:46:28.890589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:46:28.890589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:46:28.890589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:46:28.890589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:46:28.890589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:46:28.890589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:46:28.891225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:46:28.891225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:46:28.891225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:46:28.891225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:46:28.891225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:46:28.891225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:28.891225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:46:28.891225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:28.891225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:46:28.891225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:28.891225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:46:28.891225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:28.891225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:46:28.891225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:28.891225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:46:28.891225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:46:28.891225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:28.891225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:46:28.891225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:28.891782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:46:28.891782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:46:28.891782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:28.891782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:46:28.891782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:28.891782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:46:33.675906 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:46:33.676496 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:46:33.676601 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:46:33.677982 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:46:33.710722 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:46:33.711027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:33.711682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:46:33.711682 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:33.711682 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:46:33.711682 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:46:33.711682 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:33.711682 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:46:33.711682 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:33.711682 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:46:33.712519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:33.712519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:46:33.712519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:46:33.712519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:46:33.712519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:46:33.712519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:46:33.712519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:46:33.712519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:33.712519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:46:33.712519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:33.712519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:46:33.712519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:33.713039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:46:33.713039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:33.713039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:46:33.713039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:33.713039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:46:33.713039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:46:33.713039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:33.713039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:46:33.713039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:33.713039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:46:33.713039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:46:33.713039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:33.713039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:46:33.713480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:33.713480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:46:33.713480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:33.713480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:46:33.713480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:46:33.713480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:46:33.713480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:46:33.713480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:46:33.713480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:46:33.713480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:46:33.713480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:46:33.713480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:46:33.713480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:46:33.713480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:46:33.713480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:46:33.714020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:46:33.714020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:46:33.714020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:46:33.714020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:46:33.714020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:46:33.714020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:33.714020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:46:33.714020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:33.714020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:46:33.714020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:33.714020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:46:33.714020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:33.714020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:46:33.714020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:33.714020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:46:33.714020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:46:33.714020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:33.714020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:46:33.714020 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:33.714714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:46:33.714714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:46:33.714714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:33.714714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:46:33.714714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:33.714714 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:46:33.859112 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:46:33.859856 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:46:33.860058 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:46:33.861401 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:46:33.862412 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:46:33.862764 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:33.863709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:46:33.863709 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:33.863709 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:46:33.863709 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:46:33.863709 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:33.863709 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:46:33.863709 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:33.863709 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:46:33.864882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:33.864882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:46:33.864882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:46:33.864882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:46:33.864882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:46:33.864882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:46:33.864882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:46:33.864882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:33.864882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:46:33.864882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:33.864882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:46:33.864882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:33.865692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:46:33.865692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:33.865692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:46:33.865692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:33.865692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:46:33.865692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:46:33.865692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:33.865692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:46:33.865692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:33.865692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:46:33.865692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:46:33.865692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:33.865692 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:46:33.866289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:33.866289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:46:33.866289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:33.866289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:46:33.866289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:46:33.866289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:46:33.866289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:46:33.866289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:46:33.866289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:46:33.866289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:46:33.866289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:46:33.866289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:46:33.866289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:46:33.866289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:46:33.866289 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:46:33.866766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:46:33.866766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:46:33.866766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:46:33.866766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:46:33.866766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:46:33.866766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:33.866766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:46:33.866766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:33.866766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:46:33.866766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:33.866766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:46:33.866766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:33.866766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:46:33.866766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:33.866766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:46:33.866766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:46:33.866766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:33.866766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:46:33.866766 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:33.867303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:46:33.867303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:46:33.867303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:33.867303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:46:33.867303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:33.867303 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:46:33.868109 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:46:33.868558 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:46:33.868671 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:46:33.868812 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:46:33.869327 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:46:33.869590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:33.870277 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:46:33.870277 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:33.870277 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:46:33.870277 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:46:33.870277 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:33.870277 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:46:33.870277 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:33.870277 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:46:33.871136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:33.871136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:46:33.871136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:46:33.871136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:46:33.871136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:33.871136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:46:33.871136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:33.871136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:46:33.871136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:33.871136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:46:33.871136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:33.871136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:46:33.871136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:33.871136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:46:33.871136 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:46:33.871709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:33.871709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:46:33.871709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:33.871709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:46:33.871709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:46:33.871709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:33.871709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:46:33.871709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:33.871709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:46:33.871977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:33.871977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:46:33.871977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:46:33.871977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:46:33.871977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:46:33.871977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:46:33.871977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:46:33.871977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:46:33.871977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:46:33.871977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:46:33.871977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:33.871977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:46:33.872327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:33.872327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:46:33.872327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:33.872327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:46:33.872327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:33.872327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:46:33.872327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:33.872327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:46:33.872327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:46:33.872327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:33.872327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:46:33.872327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:33.872327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:46:33.872327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:46:33.872691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:33.872691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:46:33.872691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:33.872691 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:46:33.872946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:33.872946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:46:33.872946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:46:33.872946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:46:33.872946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:46:33.872946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:46:33.872946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:46:33.872946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:46:33.872946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:46:33.872946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:46:33.872946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:46:33.872946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:46:33.872946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:46:33.872946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:46:33.872946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:46:33.872946 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:46:33.873440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:46:33.873440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:46:33.873440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:46:33.873440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:46:33.873440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:46:33.873440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:33.873440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:46:33.873440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:33.873440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:46:33.873440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:33.873440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:46:33.873440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:33.873440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:46:33.873440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:33.873440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:46:33.873440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:46:33.873440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:33.873440 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:46:33.873941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:33.873941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:46:33.873941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:46:33.873941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:33.873941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:46:33.873941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:33.873941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:46:33.875682 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:46:33.875979 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:46:33.876082 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:46:33.876241 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:46:33.876648 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:46:33.876927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:33.877963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:46:33.877963 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:33.877963 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:46:33.877963 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:46:33.877963 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:33.877963 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:46:33.877963 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:33.877963 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:46:33.878697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:33.878697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:46:33.878697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:46:33.878697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:46:33.878697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:33.878697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:46:33.878697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:33.878697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:46:33.878697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:33.878697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:46:33.878697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:33.878697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:46:33.878697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:33.878697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:46:33.878697 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:46:33.879220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:33.879220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:46:33.879220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:33.879220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:46:33.879220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:46:33.879220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:33.879220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:46:33.879220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:33.879220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:46:33.879829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:33.879829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:46:33.879829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:46:33.879829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:46:33.879829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:46:33.879829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:46:33.879829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:46:33.879829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:46:33.879829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:46:33.879829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:33.879829 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:46:33.880258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:33.880258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:46:33.880258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:33.880258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:46:33.880258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:33.880258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:46:33.880258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:33.880258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:46:33.880258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:46:33.880258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:33.880258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:46:33.880258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:33.880258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:46:33.880258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:46:33.880723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:33.880723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:46:33.880723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:33.880723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:46:33.881552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:33.881552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:46:33.881552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:46:33.881552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:46:33.881552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:46:33.881552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:46:33.881552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:46:33.881552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:46:33.881552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:46:33.881552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:46:33.881552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:46:33.881552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:46:33.881552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:46:33.881552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:46:33.881552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:46:33.881552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:46:33.881552 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:46:33.882081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:46:33.882081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:46:33.882081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:46:33.882081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:33.882081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:46:33.882081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:33.882081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:46:33.882081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:33.882081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:46:33.882081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:33.882081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:46:33.882081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:33.882081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:46:33.882081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:46:33.882081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:33.882081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:46:33.882081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:33.882081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:46:33.882081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:46:33.882680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:33.882680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:46:33.882680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:33.882680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:46:33.886369 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:46:33.886714 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:46:33.886852 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:46:33.887019 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:46:33.887425 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:46:33.887657 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:46:33.887828 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:33.888324 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:46:33.888324 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:33.888324 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:46:33.888324 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:46:33.888324 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:33.888324 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:46:33.888324 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:33.888324 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:46:33.889298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:33.889298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:46:33.889298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:46:33.889298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:46:33.889298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:46:33.889298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:46:33.889298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:46:33.889298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:33.889298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:46:33.889298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:33.889298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:46:33.889298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:33.889783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:46:33.889783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:33.889783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:46:33.889783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:33.889783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:46:33.889783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:46:33.889783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:33.889783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:46:33.889783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:33.889783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:46:33.889783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:46:33.889783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:33.889783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:46:33.890179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:33.890179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:46:33.890357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:33.890357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:46:33.890357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:46:33.890357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:46:33.890357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:46:33.890357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:46:33.890357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:46:33.890357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:46:33.890357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:46:33.890357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:46:33.890357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:46:33.890357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:46:33.890357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:46:33.890357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:46:33.890357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:46:33.890357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:46:33.890957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:46:33.890957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:46:33.890957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:33.890957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:46:33.890957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:33.890957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:46:33.890957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:33.890957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:46:33.890957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:33.890957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:46:33.890957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:33.890957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:46:33.890957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:46:33.890957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:33.890957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:46:33.890957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:33.890957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:46:33.890957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:46:33.890957 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:33.891511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:46:33.891511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:33.891511 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:46:38.676551 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:46:38.677219 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:46:38.677219 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:46:38.678340 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:46:38.710576 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:46:38.710963 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:38.711844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:46:38.711844 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:38.711844 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:46:38.711844 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:46:38.711844 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:38.711844 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:46:38.711844 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:38.711844 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:46:38.712627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:38.712627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:46:38.712627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:46:38.712627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:46:38.712627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:46:38.712627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:46:38.712627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:46:38.712627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:38.712627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:46:38.712627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:38.712627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:46:38.712627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:38.713099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:46:38.713099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:38.713099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:46:38.713099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:38.713099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:46:38.713099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:46:38.713099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:38.713099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:46:38.713099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:38.713099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:46:38.713099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:46:38.713099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:38.713099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:46:38.713505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:38.713505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:46:38.713505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:38.713505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:46:38.713505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:46:38.713505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:46:38.713505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:46:38.713505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:46:38.713505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:46:38.713505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:46:38.713505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:46:38.713505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:46:38.713505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:46:38.713505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:46:38.713505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:46:38.714096 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:46:38.714096 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:46:38.714096 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:46:38.714096 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:46:38.714096 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:46:38.714096 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:38.714096 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:46:38.714096 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:38.714096 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:46:38.714096 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:38.714096 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:46:38.714096 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:38.714096 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:46:38.714096 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:38.714096 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:46:38.714096 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:46:38.714096 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:38.714096 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:46:38.714096 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:38.714674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:46:38.714674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:46:38.714674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:38.714674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:46:38.714674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:38.714674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:46:38.859777 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:46:38.860328 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:46:38.860466 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:46:38.861559 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:46:38.861993 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:46:38.862252 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:38.863310 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:46:38.863310 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:38.863310 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:46:38.863310 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:46:38.863310 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:38.863310 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:46:38.863310 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:38.863310 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:46:38.864889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:38.864889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:46:38.864889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:46:38.864889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:46:38.864889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:46:38.864889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:46:38.864889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:46:38.864889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:38.864889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:46:38.864889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:38.864889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:46:38.864889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:38.865238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:46:38.865238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:38.865238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:46:38.865238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:38.865238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:46:38.865238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:46:38.865238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:38.865238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:46:38.865238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:38.865238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:46:38.865238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:46:38.865238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:38.865238 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:46:38.865562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:38.865562 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:46:38.866623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:38.866623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:46:38.866623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:46:38.866623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:46:38.866623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:46:38.866623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:46:38.866623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:46:38.866623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:46:38.866623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:46:38.866623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:46:38.866623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:46:38.866623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:46:38.866623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:46:38.866623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:46:38.866623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:46:38.866623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:46:38.867388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:46:38.867388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:46:38.867388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:38.867388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:46:38.867388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:38.867388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:46:38.867388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:38.867388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:46:38.867388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:38.867388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:46:38.867388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:38.867388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:46:38.867388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:46:38.867388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:38.867388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:46:38.867388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:38.867388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:46:38.867388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:46:38.867388 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:38.868062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:46:38.868062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:38.868062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:46:38.868626 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:46:38.869081 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:46:38.869283 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:46:38.869470 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:46:38.870075 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:46:38.870441 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:38.871198 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:46:38.871198 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:38.871198 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:46:38.871198 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:46:38.871198 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:38.871198 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:46:38.871198 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:38.871198 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:46:38.872034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:38.872034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:46:38.872034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:46:38.872034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:46:38.872034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:38.872034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:46:38.872034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:38.872034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:46:38.872034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:38.872034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:46:38.872034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:38.872034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:46:38.872034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:38.872034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:46:38.872034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:46:38.872514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:38.872514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:46:38.872514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:38.872514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:46:38.872514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:46:38.872514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:38.872514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:46:38.872514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:38.872514 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:46:38.872945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:38.872945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:46:38.872945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:46:38.872945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:46:38.872945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:46:38.872945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:46:38.872945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:46:38.872945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:46:38.872945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:46:38.872945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:46:38.872945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:38.872945 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:46:38.873323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:38.873323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:46:38.873323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:38.873323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:46:38.873323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:38.873323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:46:38.873323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:38.873323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:46:38.873323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:46:38.873323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:38.873323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:46:38.873323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:38.873323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:46:38.873323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:46:38.873717 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:38.873717 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:46:38.873717 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:38.873717 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:46:38.874050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:38.874050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:46:38.874050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:46:38.874050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:46:38.874050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:46:38.874050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:46:38.874050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:46:38.874050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:46:38.874050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:46:38.874050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:46:38.874050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:46:38.874050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:46:38.874050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:46:38.874050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:46:38.874050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:46:38.874050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:46:38.874483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:46:38.874483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:46:38.874483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:46:38.874483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:46:38.874483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:46:38.874483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:38.874483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:46:38.874483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:38.874483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:46:38.874483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:38.874483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:46:38.874483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:38.874483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:46:38.874483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:38.874483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:46:38.874483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:46:38.874483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:38.874483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:46:38.875041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:38.875041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:46:38.875041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:46:38.875041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:38.875041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:46:38.875041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:38.875041 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:46:38.876129 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:46:38.876437 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:46:38.876560 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:46:38.876697 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:46:38.877184 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:46:38.877559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:38.878549 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:46:38.878549 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:38.878549 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:46:38.878549 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:46:38.878549 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:38.878549 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:46:38.878549 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:38.878549 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:46:38.880447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:38.880447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:46:38.880447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:46:38.880447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:46:38.880447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:38.880447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:46:38.880447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:38.880447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:46:38.880447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:38.880447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:46:38.880447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:38.880447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:46:38.880447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:38.880447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:46:38.880447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:46:38.880886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:38.880886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:46:38.880886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:38.880886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:46:38.880886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:46:38.880886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:38.880886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:46:38.880886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:38.880886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:46:38.882607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:38.882607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:46:38.882607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:46:38.882607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:46:38.882607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:46:38.882607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:46:38.882607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:46:38.882607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:46:38.882607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:46:38.882607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:38.882607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:46:38.883046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:38.883046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:46:38.883046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:38.883046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:46:38.883046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:38.883046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:46:38.883046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:38.883046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:46:38.883046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:46:38.883046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:38.883046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:46:38.883046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:38.883046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:46:38.883046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:46:38.883453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:38.883453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:46:38.883453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:38.883453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:46:38.884719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:38.884719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:46:38.884719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:46:38.884719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:46:38.884719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:46:38.884719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:46:38.884719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:46:38.884719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:46:38.884719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:46:38.884719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:46:38.884719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:46:38.884719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:46:38.884719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:46:38.884719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:46:38.884719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:46:38.884719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:46:38.884719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:46:38.885266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:46:38.885266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:46:38.885266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:46:38.885266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:38.885266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:46:38.885266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:38.885266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:46:38.885266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:38.885266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:46:38.885266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:38.885266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:46:38.885266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:38.885266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:46:38.885266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:46:38.885266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:38.885266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:46:38.885266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:38.885266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:46:38.885266 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:46:38.885889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:38.885889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:46:38.885889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:38.885889 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:46:38.886894 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:46:38.887351 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:46:38.887509 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:46:38.887734 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:46:38.888444 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:46:38.888951 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:46:38.889301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:38.890357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:46:38.890357 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:38.890357 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:46:38.890357 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:46:38.890357 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:38.890357 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:46:38.890357 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:38.890357 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:46:38.891322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:38.891322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:46:38.891322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:46:38.891322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:46:38.891322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:46:38.891322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:46:38.891322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:46:38.891322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:38.891322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:46:38.891322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:38.891322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:46:38.891322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:38.891699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:46:38.891699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:38.891699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:46:38.891699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:38.891699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:46:38.891699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:46:38.891699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:38.891699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:46:38.891699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:38.891699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:46:38.891699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:46:38.891699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:38.891699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:46:38.892083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:38.892083 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:46:38.892325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:38.892325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:46:38.892325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:46:38.892325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:46:38.892325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:46:38.892325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:46:38.892325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:46:38.892325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:46:38.892325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:46:38.892325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:46:38.892325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:46:38.892325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:46:38.892325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:46:38.892325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:46:38.892325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:46:38.892325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:46:38.892783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:46:38.892783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:46:38.892783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:38.892783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:46:38.892783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:38.892783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:46:38.892783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:38.892783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:46:38.892783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:38.892783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:46:38.892783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:38.892783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:46:38.892783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:46:38.892783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:38.892783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:46:38.892783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:38.892783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:46:38.892783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:46:38.892783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:38.893300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:46:38.893300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:38.893300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:46:43.677165 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:46:43.678000 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:46:43.678000 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:46:43.679269 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:46:43.714589 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:46:43.715056 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:43.715911 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:46:43.715911 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:43.715911 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:46:43.715911 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:46:43.715911 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:43.715911 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:46:43.715911 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:43.715911 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:46:43.717269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:43.717269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:46:43.717269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:46:43.717269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:46:43.717269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:46:43.717269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:46:43.717269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:46:43.717269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:43.717269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:46:43.717269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:43.717269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:46:43.717269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:43.717917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:46:43.717917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:43.717917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:46:43.717917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:43.717917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:46:43.717917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:46:43.717917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:43.717917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:46:43.717917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:43.717917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:46:43.717917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:46:43.717917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:43.717917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:46:43.718493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:43.718493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:46:43.718493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:43.718493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:46:43.718493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:46:43.718493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:46:43.718493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:46:43.718493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:46:43.718493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:46:43.718493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:46:43.718493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:46:43.718493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:46:43.718493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:46:43.718493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:46:43.718493 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:46:43.719165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:46:43.719165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:46:43.719165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:46:43.719165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:46:43.719165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:46:43.719165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:43.719165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:46:43.719165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:43.719165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:46:43.719165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:43.719165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:46:43.719165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:43.719165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:46:43.719165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:43.719165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:46:43.719165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:46:43.719165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:43.719165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:46:43.719165 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:43.720029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:46:43.720029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:46:43.720029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:43.720029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:46:43.720029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:43.720029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:46:43.860582 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:46:43.861122 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:46:43.861335 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:46:43.862636 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:46:43.862969 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:46:43.863203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:43.864111 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:46:43.864111 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:43.864111 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:46:43.864111 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:46:43.864111 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:43.864111 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:46:43.864111 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:43.864111 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:46:43.865761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:43.865761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:46:43.865761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:46:43.865761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:46:43.865761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:46:43.865761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:46:43.865761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:46:43.865761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:43.865761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:46:43.865761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:43.865761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:46:43.865761 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:43.866323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:46:43.866323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:43.866323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:46:43.866323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:43.866323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:46:43.866323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:46:43.866323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:43.866323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:46:43.866323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:43.866323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:46:43.866323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:46:43.866323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:43.866323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:46:43.866965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:43.866965 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:46:43.867538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:43.867538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:46:43.867538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:46:43.867538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:46:43.867538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:46:43.867538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:46:43.867538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:46:43.867538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:46:43.867538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:46:43.867538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:46:43.867538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:46:43.867538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:46:43.867538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:46:43.867538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:46:43.867538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:46:43.867538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:46:43.868401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:46:43.868401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:46:43.868401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:43.868401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:46:43.868401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:43.868401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:46:43.868401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:43.868401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:46:43.868401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:43.868401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:46:43.868401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:43.868401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:46:43.868401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:46:43.868401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:43.868401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:46:43.868401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:43.868401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:46:43.868401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:46:43.868401 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:43.869516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:46:43.869516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:43.869516 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:46:43.869516 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:46:43.869516 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:46:43.869852 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:46:43.869915 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:46:43.870283 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:46:43.870475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:43.871037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:46:43.871037 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:43.871037 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:46:43.871037 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:46:43.871037 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:43.871037 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:46:43.871037 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:43.871037 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:46:43.871790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:43.871790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:46:43.871790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:46:43.871790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:46:43.871790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:43.871790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:46:43.871790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:43.871790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:46:43.871790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:43.871790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:46:43.871790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:43.871790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:46:43.871790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:43.871790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:46:43.871790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:46:43.872567 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:43.872567 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:46:43.872567 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:43.872567 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:46:43.872567 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:46:43.872567 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:43.872567 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:46:43.872567 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:43.872567 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:46:43.873035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:43.873035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:46:43.873035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:46:43.873035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:46:43.873035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:46:43.873035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:46:43.873035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:46:43.873035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:46:43.873035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:46:43.873035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:46:43.873035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:43.873035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:46:43.873805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:43.873805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:46:43.873805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:43.873805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:46:43.873805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:43.873805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:46:43.873805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:43.873805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:46:43.873805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:46:43.873805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:43.873805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:46:43.873805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:43.873805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:46:43.873805 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:46:43.874168 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:43.874168 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:46:43.874168 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:43.874168 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:46:43.874168 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:43.874168 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:46:43.874168 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:46:43.874168 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:46:43.874168 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:46:43.874168 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:46:43.874168 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:46:43.874168 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:46:43.874168 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:46:43.874168 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:46:43.874569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:46:43.874569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:46:43.874569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:46:43.874569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:46:43.874569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:46:43.874569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:46:43.874569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:46:43.874569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:46:43.874569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:46:43.874569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:46:43.874569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:46:43.874569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:43.874569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:46:43.874569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:43.874569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:46:43.874569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:43.874569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:46:43.874569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:43.874569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:46:43.875696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:43.875696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:46:43.875696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:46:43.875696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:43.875696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:46:43.875696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:43.875696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:46:43.875696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:46:43.875696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:43.875696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:46:43.875696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:43.875696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:46:43.876564 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:46:43.876850 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:46:43.876972 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:46:43.877104 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:46:43.877557 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:46:43.877867 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:43.878590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:46:43.878590 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:43.878590 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:46:43.878590 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:46:43.878590 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:43.878590 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:46:43.878590 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:43.878590 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:46:43.879380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:43.879380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:46:43.879380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:46:43.879380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:46:43.879380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:43.879380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:46:43.879380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:43.879380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:46:43.879380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:43.879380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:46:43.879380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:43.879380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:46:43.879380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:43.879380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:46:43.879380 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:46:43.879891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:43.879891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:46:43.879891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:43.879891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:46:43.879891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:46:43.879891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:43.879891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:46:43.879891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:43.879891 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:46:43.880250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:43.880250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:46:43.880250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:46:43.880250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:46:43.880250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:46:43.880250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:46:43.880250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:46:43.880250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:46:43.880250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:46:43.880250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:43.880250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:46:43.880599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:43.880599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:46:43.880599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:43.880599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:46:43.880599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:43.880599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:46:43.880599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:43.880599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:46:43.880599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:46:43.880599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:43.880599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:46:43.880599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:43.880599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:46:43.880599 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:46:43.881001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:43.881001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:46:43.881001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:43.881001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:46:43.881470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:43.881470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:46:43.881470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:46:43.881470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:46:43.881470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:46:43.881470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:46:43.881470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:46:43.881470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:46:43.881470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:46:43.881470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:46:43.881470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:46:43.881470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:46:43.881470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:46:43.881470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:46:43.881470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:46:43.881470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:46:43.881470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:46:43.881975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:46:43.881975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:46:43.881975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:46:43.881975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:43.881975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:46:43.881975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:43.881975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:46:43.881975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:43.881975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:46:43.881975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:43.881975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:46:43.881975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:43.881975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:46:43.881975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:46:43.881975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:43.881975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:46:43.881975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:43.881975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:46:43.881975 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:46:43.882344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:43.882344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:46:43.882344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:43.882344 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:46:43.887419 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:46:43.887694 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:46:43.887898 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:46:43.888057 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:46:43.888539 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:46:43.888955 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:46:43.889196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:43.890066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:46:43.890066 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:43.890066 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:46:43.890066 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:46:43.890066 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:43.890066 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:46:43.890066 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:43.890066 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:46:43.890988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:43.890988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:46:43.890988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:46:43.890988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:46:43.890988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:46:43.890988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:46:43.890988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:46:43.890988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:43.890988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:46:43.890988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:43.890988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:46:43.890988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:43.891727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:46:43.891727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:43.891727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:46:43.891727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:43.891727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:46:43.891727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:46:43.891727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:43.891727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:46:43.891727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:43.891727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:46:43.891727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:46:43.891727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:43.891727 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:46:43.892330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:43.892330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:46:43.892330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:43.892330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:46:43.892330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:46:43.892330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:46:43.892330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:46:43.892330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:46:43.892330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:46:43.892330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:46:43.892330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:46:43.892330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:46:43.892330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:46:43.892330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:46:43.892330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:46:43.893281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:46:43.893281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:46:43.893281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:46:43.893281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:46:43.893281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:46:43.893281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:43.893281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:46:43.893281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:43.893281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:46:43.893281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:43.893281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:46:43.893281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:43.893281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:46:43.893281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:43.893281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:46:43.893281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:46:43.893281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:43.893281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:46:43.893281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:43.894647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:46:43.894647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:46:43.894647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:43.894647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:46:43.894647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:43.894647 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:46:48.677826 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:46:48.679067 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:46:48.679067 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:46:48.679420 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:46:48.707045 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:46:48.707536 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:48.707835 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:46:48.707835 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:48.707835 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:46:48.707835 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:46:48.707835 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:48.707835 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:46:48.707835 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:48.707835 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:46:48.708879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.029 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:48.708879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:46:48.708879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:46:48.708879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:46:48.708879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:46:48.708879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:46:48.708879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:46:48.708879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:48.708879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:46:48.708879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:48.708879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:46:48.708879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:48.710508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:46:48.710508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:48.710508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:46:48.710508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:48.710508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:46:48.710508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:46:48.710508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:48.710508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:46:48.710508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:48.710508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:46:48.710508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:46:48.710508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:48.710508 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:46:48.711155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:48.711155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:46:48.711155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:48.711155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:46:48.711155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:46:48.711155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:46:48.711155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:46:48.711155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:46:48.711155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:46:48.711155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:46:48.711155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:46:48.711155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:46:48.711155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:46:48.711155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:46:48.711155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:46:48.712893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:46:48.712893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:46:48.712893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:46:48.712893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:46:48.712893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:46:48.712893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:48.712893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:46:48.712893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:48.712893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:46:48.712893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:48.712893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:46:48.712893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:48.712893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:46:48.712893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:48.712893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:46:48.712893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:46:48.712893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:48.712893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:46:48.712893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:48.715042 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:46:48.715042 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:46:48.715042 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:48.715042 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:46:48.715042 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:48.715042 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:46:48.861268 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:46:48.861763 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:46:48.862050 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:46:48.863348 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:46:48.863798 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:46:48.864031 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:48.864904 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:46:48.864904 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:48.864904 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:46:48.864904 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:46:48.864904 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:48.864904 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:46:48.864904 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:48.864904 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:46:48.866102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:48.866102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:46:48.866102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:46:48.866102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:46:48.866102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:46:48.866102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:46:48.866102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:46:48.866102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:48.866102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:46:48.866102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:48.866102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:46:48.866102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:48.866658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:46:48.866658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:48.866658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:46:48.866658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:48.866658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:46:48.866658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:46:48.866658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:48.866658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:46:48.866658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:48.866658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:46:48.866658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:46:48.866658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:48.866658 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:46:48.867221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:48.867221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:46:48.867318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:48.867318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:46:48.867318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:46:48.867318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:46:48.867318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:46:48.867318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:46:48.867318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:46:48.867318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:46:48.867318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:46:48.867318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:46:48.867318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:46:48.867318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:46:48.867318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:46:48.867318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:46:48.867318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:46:48.867318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:46:48.867976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:46:48.867976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:46:48.867976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:48.867976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:46:48.867976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:48.867976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:46:48.867976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:48.867976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:46:48.867976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:48.867976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:46:48.867976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:48.867976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:46:48.867976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:46:48.867976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:48.867976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:46:48.867976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:48.867976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:46:48.867976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:46:48.867976 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:48.868752 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:46:48.868752 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:48.868752 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:46:48.869908 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:46:48.870184 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:46:48.870296 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:46:48.870461 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:46:48.871027 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:46:48.871358 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:48.872194 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:46:48.872194 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:48.872194 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:46:48.872194 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:46:48.872194 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:48.872194 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:46:48.872194 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:48.872194 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:46:48.873564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:48.873564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:46:48.873564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:46:48.873564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:46:48.873564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:48.873564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:46:48.873564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:48.873564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:46:48.873564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:48.873564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:46:48.873564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:48.873564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:46:48.873564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:48.873564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:46:48.873564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:46:48.874538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:48.874538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:46:48.874538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:48.874538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:46:48.874538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:46:48.874538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:48.874538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:46:48.874538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:48.874538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:46:48.875220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:48.875220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:46:48.875220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:46:48.875220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:46:48.875220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:46:48.875220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:46:48.875220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:46:48.875220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:46:48.875220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:46:48.875220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:46:48.875220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:48.875220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:46:48.875914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:48.875914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:46:48.875914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:48.875914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:46:48.875914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:48.875914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:46:48.875914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:48.875914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:46:48.875914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:46:48.875914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:48.875914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:46:48.875914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:48.875914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:46:48.875914 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:46:48.876661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:48.876661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:46:48.876661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:48.876661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:46:48.876661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:48.876661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:46:48.876661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:46:48.876661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:46:48.876661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:46:48.876661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:46:48.876661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:46:48.876661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:46:48.876661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:46:48.876661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:46:48.877456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:46:48.877456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:46:48.877456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:46:48.877456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:46:48.877456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:46:48.877456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:46:48.877456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:46:48.877456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:46:48.877456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:46:48.877456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:46:48.877456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:46:48.877456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:48.877456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:46:48.877456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:48.877456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:46:48.877456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:48.877456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:46:48.877456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:48.877456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:46:48.878051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:48.878051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:46:48.878051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:46:48.878051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:48.878051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:46:48.878051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:48.878051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:46:48.878051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:46:48.878051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:48.878051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:46:48.878051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:48.878051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:46:48.878051 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:46:48.878051 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:46:48.878420 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:46:48.878420 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:46:48.878420 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:46:48.878420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:48.879291 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:46:48.879291 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:48.879291 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:46:48.879291 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:46:48.879291 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:48.879291 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:46:48.879291 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:48.879291 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:46:48.880297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:48.880297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:46:48.880297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:46:48.880297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:46:48.880297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:48.880297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:46:48.880297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:48.880297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:46:48.880297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:48.880297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:46:48.880297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:48.880297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:46:48.880297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:48.880297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:46:48.880297 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:46:48.880681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:48.880681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:46:48.880681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:48.880681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:46:48.880681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:46:48.880681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:48.880681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:46:48.880681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:48.880681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:46:48.881282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:48.881282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:46:48.881282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:46:48.881282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:46:48.881282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:46:48.881282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:46:48.881282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:46:48.881282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:46:48.881282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:46:48.881282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:48.881282 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:46:48.881592 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:48.881592 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:46:48.881592 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:48.881592 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:46:48.881592 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:48.881592 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:46:48.881592 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:48.881592 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:46:48.881592 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:46:48.881592 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:48.881592 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:46:48.881592 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:48.881592 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:46:48.881592 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:46:48.882533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:48.882533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:46:48.882533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:48.882533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:46:48.882678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:48.882678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:46:48.882678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:46:48.882678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:46:48.882678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:46:48.882678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:46:48.882678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:46:48.882678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:46:48.882678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:46:48.882678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:46:48.882678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:46:48.882678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:46:48.882678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:46:48.882678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:46:48.882678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:46:48.882678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:46:48.882678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:46:48.883134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:46:48.883134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:46:48.883134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:46:48.883134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:48.883134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:46:48.883134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:48.883134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:46:48.883134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:48.883134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:46:48.883134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:48.883134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:46:48.883134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:48.883134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:46:48.883134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:46:48.883134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:48.883134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:46:48.883134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:48.883134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:46:48.883134 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:46:48.883594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:48.883594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:46:48.883594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:48.883594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:46:48.887844 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:46:48.888127 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:46:48.888252 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:46:48.888464 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:46:48.888937 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:46:48.889287 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:46:48.889509 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:48.890281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:46:48.890281 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:48.890281 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:46:48.890281 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:46:48.890281 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:48.890281 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:46:48.890281 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:48.890281 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:46:48.891378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:48.891378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:46:48.891378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:46:48.891378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:46:48.891378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:46:48.891378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:46:48.891378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:46:48.891378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:48.891378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:46:48.891378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:48.891378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:46:48.891378 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:48.891804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:46:48.891804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:48.891804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:46:48.891804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:48.891804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:46:48.891804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:46:48.891804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:48.891804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:46:48.891804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:48.891804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:46:48.891804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:46:48.891804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:48.891804 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:46:48.892217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:48.892217 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:46:48.893123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:48.893123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:46:48.893123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:46:48.893123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:46:48.893123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:46:48.893123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:46:48.893123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:46:48.893123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:46:48.893123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:46:48.893123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:46:48.893123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:46:48.893123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:46:48.893123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:46:48.893123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:46:48.893123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:46:48.893123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:46:48.893606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:46:48.893606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:46:48.893606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:48.893606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:46:48.893606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:48.893606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:46:48.893606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:48.893606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:46:48.893606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:48.893606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:46:48.893606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:48.893606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:46:48.893606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:46:48.893606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:48.893606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:46:48.893606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:48.893606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:46:48.893606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:46:48.893606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:48.894119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:46:48.894119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:48.894119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:46:53.678551 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:46:53.679312 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:46:53.679762 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:46:53.680791 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:46:53.717810 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:46:53.717810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:53.718475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:46:53.718475 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:53.718475 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:46:53.718475 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:46:53.718475 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:53.718475 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:46:53.718475 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:53.718475 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:46:53.719312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:53.719312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:46:53.719312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:46:53.719312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:46:53.719312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:46:53.719312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:46:53.719312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:46:53.719312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:53.719312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:46:53.719312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:53.719312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:46:53.719312 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:53.720090 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:46:53.720090 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:53.720090 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:46:53.720090 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:53.720090 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:46:53.720090 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:46:53.720090 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:53.720090 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:46:53.720090 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:53.720090 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:46:53.720090 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:46:53.720090 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:53.720090 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:46:53.720822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:53.720822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:46:53.720822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:53.720822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:46:53.720822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:46:53.720822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:46:53.720822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:46:53.720822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:46:53.720822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:46:53.720822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:46:53.720822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:46:53.720822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:46:53.720822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:46:53.720822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:46:53.720822 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:46:53.721612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:46:53.721612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:46:53.721612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:46:53.721612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:46:53.721612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:46:53.721612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:53.721612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:46:53.721612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:53.721612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:46:53.721612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:53.721612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:46:53.721612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:53.721612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:46:53.721612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:53.721612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:46:53.721612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:46:53.721612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:53.721612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:46:53.721612 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:53.722597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:46:53.722597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:46:53.722597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:53.722597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:46:53.722597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:53.722597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:46:53.862011 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:46:53.862858 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:46:53.863130 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:46:53.864501 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:46:53.864772 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:46:53.864927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:53.865698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:46:53.865698 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:53.865698 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:46:53.865698 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:46:53.865698 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:53.865698 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:46:53.865698 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:53.865698 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:46:53.867446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:53.867446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:46:53.867446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:46:53.867446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:46:53.867446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:46:53.867446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:46:53.867446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:46:53.867446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:53.867446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:46:53.867446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:53.867446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:46:53.867446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:53.868046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:46:53.868046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:53.868046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:46:53.868046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:53.868046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:46:53.868046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:46:53.868046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:53.868046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:46:53.868046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:53.868046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:46:53.868046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:46:53.868046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:53.868046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:46:53.868659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:53.868659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:46:53.869285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:53.869285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:46:53.869285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:46:53.869285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:46:53.869285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:46:53.869285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:46:53.869285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:46:53.869285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:46:53.869285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:46:53.869285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:46:53.869285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:46:53.869285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:46:53.869285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:46:53.869285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:46:53.869285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:46:53.869285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:46:53.870311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:46:53.870311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:46:53.870311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:53.870311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:46:53.870311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:53.870311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:46:53.870311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:53.870311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:46:53.870311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:53.870311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:46:53.870311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:53.870311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:46:53.870311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:46:53.870311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:53.870311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:46:53.870311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:53.870311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:46:53.870311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:46:53.870311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:53.871204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:46:53.871204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:53.871204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:46:53.871204 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:46:53.871204 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:46:53.871204 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:46:53.871204 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:46:53.871204 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:46:53.871624 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:53.871943 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:46:53.871943 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:53.871943 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:46:53.871943 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:46:53.871943 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:53.871943 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:46:53.871943 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:53.871943 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:46:53.872671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:53.872671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:46:53.872671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:46:53.872671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:46:53.872671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:53.872671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:46:53.872671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:53.872671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:46:53.872671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:53.872671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:46:53.872671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:53.872671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:46:53.872671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:53.872671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:46:53.872671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:46:53.873458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:53.873458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:46:53.873458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:53.873458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:46:53.873458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:46:53.873458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:53.873458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:46:53.873458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:53.873458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:46:53.874203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:53.874203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:46:53.874203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:46:53.874203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:46:53.874203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:46:53.874203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:46:53.874203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:46:53.874203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:46:53.874203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:46:53.874203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:46:53.874203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:53.874203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:46:53.874606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:53.874606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:46:53.874606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:53.874606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:46:53.874606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:53.874606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:46:53.874606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:53.874606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:46:53.874606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:46:53.874606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:53.874606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:46:53.874606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:53.874606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:46:53.874606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:46:53.875037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:53.875037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:46:53.875037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:53.875037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:46:53.875037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:53.875037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:46:53.875037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:46:53.875037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:46:53.875037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:46:53.875037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:46:53.875037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:46:53.875037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:46:53.875037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:46:53.875037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:46:53.875563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:46:53.875563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:46:53.875563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:46:53.875563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:46:53.875563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:46:53.875563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:46:53.875563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:46:53.875563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:46:53.875563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:46:53.875563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:46:53.875563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:46:53.875563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:53.875563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:46:53.875563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:53.875563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:46:53.875563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:53.875563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:46:53.875563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:53.875563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:46:53.876199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:53.876199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:46:53.876199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:46:53.876199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:53.876199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:46:53.876199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:53.876199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:46:53.876199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:46:53.876199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:53.876199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:46:53.876199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:53.876199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:46:53.877452 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:46:53.877852 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:46:53.877957 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:46:53.878090 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:46:53.878548 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:46:53.878988 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:53.879730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:46:53.879730 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:53.879730 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:46:53.879730 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:46:53.879730 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:53.879730 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:46:53.879730 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:53.879730 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:46:53.880489 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:53.880489 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:46:53.880489 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:46:53.880489 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:46:53.880489 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:53.880489 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:46:53.880489 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:53.880489 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:46:53.880489 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:53.880489 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:46:53.880489 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:53.880489 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:46:53.880489 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:53.880489 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:46:53.880489 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:46:53.881005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:53.881005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:46:53.881005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:53.881005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:46:53.881005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:46:53.881005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:53.881005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:46:53.881005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:53.881005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:46:53.881367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:53.881367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:46:53.881367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:46:53.881367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:46:53.881367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:46:53.881367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:46:53.881367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:46:53.881367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:46:53.881367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:46:53.881367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:53.881367 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:46:53.881810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:53.881810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:46:53.881810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:53.881810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:46:53.881810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:53.881810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:46:53.881810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:53.881810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:46:53.881810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:46:53.881810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:53.881810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:46:53.881810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:53.881810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:46:53.881810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:46:53.882203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:53.882203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:46:53.882203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:53.882203 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:46:53.882454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:53.882454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:46:53.882454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:46:53.882454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:46:53.882454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:46:53.882454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:46:53.882454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:46:53.882454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:46:53.882454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:46:53.882454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:46:53.882454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:46:53.882454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:46:53.882454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:46:53.882454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:46:53.882454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:46:53.882454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:46:53.882454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:46:53.883005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:46:53.883005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:46:53.883005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:46:53.883005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:53.883005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:46:53.883005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:53.883005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:46:53.883005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:53.883005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:46:53.883005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:53.883005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:46:53.883005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:53.883005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:46:53.883005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:46:53.883005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:53.883005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:46:53.883005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:53.883005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:46:53.883005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:46:53.883699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:53.883699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:46:53.883699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:53.883699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:46:53.888323 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:46:53.888648 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:46:53.888802 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:46:53.888940 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:46:53.889372 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:46:53.889678 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:46:53.889883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:53.890479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:46:53.890479 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:53.890479 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:46:53.890479 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:46:53.890479 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:53.890479 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:46:53.890479 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:53.890479 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:46:53.891437 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:53.891437 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:46:53.891437 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:46:53.891437 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:46:53.891437 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:46:53.891437 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:46:53.891437 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:46:53.891437 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:53.891437 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:46:53.891437 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:53.891437 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:46:53.891437 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:53.891839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:46:53.891839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:53.891839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:46:53.891839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:53.891839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:46:53.891839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:46:53.891839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:53.891839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:46:53.891839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:53.891839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:46:53.891839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:46:53.891839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:53.891839 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:46:53.892218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:53.892218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:46:53.892451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:53.892451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:46:53.892451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:46:53.892451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:46:53.892451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:46:53.892451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:46:53.892451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:46:53.892451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:46:53.892451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:46:53.892451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:46:53.892451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:46:53.892451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:46:53.892451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:46:53.892451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:46:53.892451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:46:53.892451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:46:53.892960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:46:53.892960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:46:53.892960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:53.892960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:46:53.892960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:53.892960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:46:53.892960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:53.892960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:46:53.892960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:53.892960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:46:53.892960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:53.892960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:46:53.892960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:46:53.892960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:53.892960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:46:53.892960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:53.892960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:46:53.892960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:46:53.892960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:53.893544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:46:53.893544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:53.893544 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:46:58.679387 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:46:58.679988 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:46:58.679988 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:46:58.680571 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:46:58.683977 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG dbcounter [-] [106314] Writing DB stats neutron:SELECT=240 {{(pid=106314) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 09 09:46:58.716479 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:46:58.716935 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:58.718050 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:46:58.718050 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:58.718050 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:46:58.718050 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:46:58.718050 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:58.718050 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:46:58.718050 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:58.718050 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:46:58.719275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:58.719275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:46:58.719275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:46:58.719275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:46:58.719275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:46:58.719275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:46:58.719275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:46:58.719275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:58.719275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:46:58.719275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:58.719275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:46:58.719275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:58.719619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:46:58.719619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:58.719619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:46:58.719619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:58.719619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:46:58.719619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:46:58.719619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:58.719619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:46:58.719619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:58.719619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:46:58.719619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:46:58.719619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:58.719619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:46:58.720003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:58.720003 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:46:58.720428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:58.720428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:46:58.720428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:46:58.720428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:46:58.720428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:46:58.720428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:46:58.720428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:46:58.720428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:46:58.720428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:46:58.720428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:46:58.720428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:46:58.720428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:46:58.720428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:46:58.720428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:46:58.720428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:46:58.720428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:46:58.720894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:46:58.720894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:46:58.720894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:58.720894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:46:58.720894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:58.720894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:46:58.720894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:58.720894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:46:58.720894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:58.720894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:46:58.720894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:58.720894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:46:58.720894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:46:58.720894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:58.720894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:46:58.720894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:58.720894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:46:58.720894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:46:58.720894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:58.721420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:46:58.721420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:58.721420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:46:58.862903 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:46:58.863775 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:46:58.863963 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:46:58.865049 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:46:58.865336 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:46:58.865502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:58.866275 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:46:58.866275 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:58.866275 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:46:58.866275 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:46:58.866275 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:58.866275 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:46:58.866275 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:58.866275 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:46:58.868053 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:58.868053 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:46:58.868053 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:46:58.868053 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:46:58.868053 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:46:58.868053 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:46:58.868053 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:46:58.868053 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:58.868053 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:46:58.868053 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:58.868053 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:46:58.868053 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:58.868598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:46:58.868598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:58.868598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:46:58.868598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:58.868598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:46:58.868598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:46:58.868598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:58.868598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:46:58.868598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:58.868598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:46:58.868598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:46:58.868598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:58.868598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:46:58.869155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:58.869155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:46:58.869634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:58.869634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:46:58.869634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:46:58.869634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:46:58.869634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:46:58.869634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:46:58.869634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:46:58.869634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:46:58.869634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:46:58.869634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:46:58.869634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:46:58.869634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:46:58.869634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:46:58.869634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:46:58.869634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:46:58.869634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:46:58.870354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:46:58.870354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:46:58.870354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:58.870354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:46:58.870354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:58.870354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:46:58.870354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:58.870354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:46:58.870354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:58.870354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:46:58.870354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:58.870354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:46:58.870354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:46:58.870354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:58.870354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:46:58.870354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:58.870354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:46:58.870354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:46:58.870354 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:58.871204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:46:58.871204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:58.871204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:46:58.871204 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:46:58.871204 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:46:58.871431 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:46:58.871487 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:46:58.872111 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:46:58.872436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:58.873219 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:46:58.873219 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:58.873219 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:46:58.873219 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:46:58.873219 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:58.873219 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:46:58.873219 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:58.873219 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:46:58.874073 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:58.874073 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:46:58.874073 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:46:58.874073 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:46:58.874073 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:58.874073 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:46:58.874073 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:58.874073 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:46:58.874073 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:58.874073 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:46:58.874073 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:58.874073 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:46:58.874073 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:58.874073 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:46:58.874073 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:46:58.874704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:58.874704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:46:58.874704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:58.874704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:46:58.874704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:46:58.874704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:58.874704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:46:58.874704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:58.874704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:46:58.875279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:58.875279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:46:58.875279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:46:58.875279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:46:58.875279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:46:58.875279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:46:58.875279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:46:58.875279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:46:58.875279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:46:58.875279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:46:58.875279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:58.875279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:46:58.876306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:58.876306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:46:58.876306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:58.876306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:46:58.876306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:58.876306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:46:58.876306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:58.876306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:46:58.876306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:46:58.876306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:58.876306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:46:58.876306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:58.876306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:46:58.876306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:46:58.876845 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:58.876845 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:46:58.876845 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:58.876845 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:46:58.876845 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:58.876845 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:46:58.876845 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:46:58.876845 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:46:58.876845 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:46:58.876845 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:46:58.876845 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:46:58.876845 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:46:58.876845 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:46:58.876845 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:46:58.877548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:46:58.877548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:46:58.877548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:46:58.877548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:46:58.877548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:46:58.877548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:46:58.877548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:46:58.877548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:46:58.877548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:46:58.877548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:46:58.877548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:46:58.877548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:58.877548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:46:58.877548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:58.877548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:46:58.877548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:58.877548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:46:58.877548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:58.877548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:46:58.878176 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:58.878176 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:46:58.878176 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:46:58.878176 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:58.878176 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:46:58.878176 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:58.878176 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:46:58.878176 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:46:58.878176 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:58.878176 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:46:58.878176 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:58.878176 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:46:58.878176 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:46:58.878627 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:46:58.878627 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:46:58.878627 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:46:58.879111 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:46:58.879472 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:58.880622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:46:58.880622 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:58.880622 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:46:58.880622 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:46:58.880622 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:58.880622 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:46:58.880622 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:58.880622 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:46:58.881460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:58.881460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:46:58.881460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:46:58.881460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:46:58.881460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:58.881460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:46:58.881460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:58.881460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:46:58.881460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:58.881460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:46:58.881460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:58.881460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:46:58.881460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:58.881460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:46:58.881460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:46:58.882008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:58.882008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:46:58.882008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:58.882008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:46:58.882008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:46:58.882008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:58.882008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:46:58.882008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:58.882008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:46:58.882420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:58.882420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:46:58.882420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:46:58.882420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:46:58.882420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:46:58.882420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:46:58.882420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:46:58.882420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:46:58.882420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:46:58.882420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:58.882420 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:46:58.882704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:58.882704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:46:58.882704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:58.882704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:46:58.882704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:58.882704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:46:58.882704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:58.882704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:46:58.882704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:46:58.882704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:58.882704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:46:58.882704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:58.882704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:46:58.882704 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:46:58.883090 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:58.883090 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:46:58.883090 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:58.883090 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:46:58.883479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:58.883479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:46:58.883479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:46:58.883479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:46:58.883479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:46:58.883479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:46:58.883479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:46:58.883479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:46:58.883479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:46:58.883479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:46:58.883479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:46:58.883479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:46:58.883479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:46:58.883479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:46:58.883479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:46:58.883479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:46:58.883479 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:46:58.884043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:46:58.884043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:46:58.884043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:46:58.884043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:58.884043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:46:58.884043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:58.884043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:46:58.884043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:58.884043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:46:58.884043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:58.884043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:46:58.884043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:58.884043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:46:58.884043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:46:58.884043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:58.884043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:46:58.884043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:58.884043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:46:58.884043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:46:58.884533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:58.884533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:46:58.884533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:58.884533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:46:58.888902 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:46:58.889254 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:46:58.889372 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:46:58.889539 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:46:58.890094 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:46:58.890449 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:46:58.890681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:58.891475 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:46:58.891475 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:58.891475 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:46:58.891475 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:46:58.891475 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:58.891475 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:46:58.891475 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:58.891475 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:46:58.892450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:58.892450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:46:58.892450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:46:58.892450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:46:58.892450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:46:58.892450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:46:58.892450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:46:58.892450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:58.892450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:46:58.892450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:58.892450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:46:58.892450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:58.892955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:46:58.892955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:58.892955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:46:58.892955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:58.892955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:46:58.892955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:46:58.892955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:58.892955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:46:58.892955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:58.892955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:46:58.892955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:46:58.892955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:58.892955 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:46:58.893505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:58.893505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:46:58.893505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:58.893505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:46:58.893505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:46:58.893505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:46:58.893505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:46:58.893505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:46:58.893505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:46:58.893505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:46:58.893505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:46:58.893505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:46:58.893505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:46:58.893505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:46:58.893505 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:46:58.894128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:46:58.894128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:46:58.894128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:46:58.894128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:46:58.894128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:46:58.894128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:58.894128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:46:58.894128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:46:58.894128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:46:58.894128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:46:58.894128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:46:58.894128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:46:58.894128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:46:58.894128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:46:58.894128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:46:58.894128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:46:58.894128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:46:58.894128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:46:58.894128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:46:58.894968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:46:58.894968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:46:58.894968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:46:58.894968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:46:58.894968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:46:58.894968 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:47:03.680260 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:47:03.681014 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:47:03.681014 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:47:03.681496 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:47:03.713457 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:47:03.713899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:03.714869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:47:03.714869 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:03.714869 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:47:03.714869 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:47:03.714869 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:03.714869 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:47:03.714869 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:03.714869 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:47:03.715781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:03.715781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:47:03.715781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:47:03.715781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:47:03.715781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:47:03.715781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:47:03.715781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:47:03.715781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:03.715781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:47:03.715781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:03.715781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:47:03.715781 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:03.716429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:47:03.716429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:03.716429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:47:03.716429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:03.716429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:47:03.716429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:47:03.716429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:03.716429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:47:03.716429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:03.716429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:47:03.716429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:47:03.716429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:03.716429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:47:03.717622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:03.717622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:47:03.717622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:03.717622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:47:03.717622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:47:03.717622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:47:03.717622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:47:03.717622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:47:03.717622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:47:03.717622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:47:03.717622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:47:03.717622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:47:03.717622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:47:03.717622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:47:03.717622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:47:03.719090 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:47:03.719090 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:47:03.719090 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:47:03.719090 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:47:03.719090 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:47:03.719090 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:03.719090 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:47:03.719090 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:03.719090 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:47:03.719090 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:03.719090 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:47:03.719090 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:03.719090 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:47:03.719090 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:03.719090 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:47:03.719090 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:47:03.719090 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:03.719090 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:47:03.719090 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:03.720699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:47:03.720699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:47:03.720699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:03.720699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:47:03.720699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:03.720699 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:47:03.863855 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:47:03.864267 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:47:03.864502 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:47:03.865679 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:47:03.866006 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:47:03.866232 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:03.867073 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:47:03.867073 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:03.867073 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:47:03.867073 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:47:03.867073 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:03.867073 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:47:03.867073 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:03.867073 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:47:03.868483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:03.868483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:47:03.868483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:47:03.868483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:47:03.868483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:47:03.868483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:47:03.868483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:47:03.868483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:03.868483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:47:03.868483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:03.868483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:47:03.868483 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:03.869519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:47:03.869519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:03.869519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:47:03.869519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:03.869519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:47:03.869519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:47:03.869519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:03.869519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:47:03.869519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:03.869519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:47:03.869519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:47:03.869519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:03.869519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:47:03.870147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:03.870147 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:47:03.870255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:03.870255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:47:03.870255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:47:03.870255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:47:03.870255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:47:03.870255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:47:03.870255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:47:03.870255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:47:03.870255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:47:03.870255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:47:03.870255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:47:03.870255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:47:03.870255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:47:03.870255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:47:03.870255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:47:03.870255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:47:03.871014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:47:03.871014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:47:03.871014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:03.871014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:47:03.871014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:03.871014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:47:03.871014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:03.871014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:47:03.871014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:03.871014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:47:03.871014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:03.871014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:47:03.871014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:47:03.871014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:03.871014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:47:03.871014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:03.871014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:47:03.871014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:47:03.871014 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:03.872210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:47:03.872210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:03.872210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:47:03.872210 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:47:03.872210 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:47:03.872210 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:47:03.872210 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:47:03.872210 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:47:03.873037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:03.873037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:47:03.873037 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:03.873037 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:47:03.873037 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:47:03.873037 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:03.873037 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:47:03.873037 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:03.873037 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:47:03.873436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:03.873436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:47:03.873436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:47:03.873436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:47:03.873436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:03.873436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:47:03.873436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:03.873436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:47:03.873436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:03.873436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:47:03.873436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:03.873436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:47:03.873436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:03.873436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:47:03.873436 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:47:03.874283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:03.874283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:47:03.874283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:03.874283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:47:03.874283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:47:03.874283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:03.874283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:47:03.874283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:03.874283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:47:03.874283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:03.874283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:47:03.874283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:47:03.874283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:47:03.874283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:47:03.874995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:47:03.874995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:47:03.874995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:47:03.874995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:47:03.874995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:47:03.874995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:03.874995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:47:03.874995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:03.874995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:47:03.874995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:03.874995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:47:03.874995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:03.874995 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:47:03.875721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:03.875721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:47:03.875721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:47:03.875721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:03.875721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:47:03.875721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:03.875721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:47:03.875721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:47:03.875721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:03.875721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:47:03.875721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:03.875721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:47:03.876453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:03.876453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:47:03.876453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:47:03.876453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:47:03.876453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:47:03.876453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:47:03.876453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:47:03.876453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:47:03.876453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:47:03.876453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:47:03.876453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:47:03.876453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:47:03.876453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:47:03.876453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:47:03.876453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:47:03.876453 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:47:03.877257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:47:03.877257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:47:03.877257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:47:03.877257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:47:03.877257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:47:03.877257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:03.877257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:47:03.877257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:03.877257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:47:03.877257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:03.877257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:47:03.877257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:03.877257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:47:03.877257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:03.877257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:47:03.877257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:47:03.877257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:03.877257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:47:03.878099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:03.878099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:47:03.878099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:47:03.878099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:03.878099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:47:03.878099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:03.878099 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:47:03.878514 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:47:03.878828 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:47:03.878922 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:47:03.879058 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:47:03.879464 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:47:03.879894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:03.880564 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:47:03.880564 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:03.880564 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:47:03.880564 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:47:03.880564 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:03.880564 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:47:03.880564 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:03.880564 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:47:03.881253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:03.881253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:47:03.881253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:47:03.881253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:47:03.881253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:03.881253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:47:03.881253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:03.881253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:47:03.881253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:03.881253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:47:03.881253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:03.881253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:47:03.881253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:03.881253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:47:03.881253 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:47:03.881621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:03.881621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:47:03.881621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:03.881621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:47:03.881621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:47:03.881621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:03.881621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:47:03.881621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:03.881621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:47:03.882113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:03.882113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:47:03.882113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:47:03.882113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:47:03.882113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:47:03.882113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:47:03.882113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:47:03.882113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:47:03.882113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:47:03.882113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:03.882113 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:47:03.882382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:03.882382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:47:03.882382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:03.882382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:47:03.882382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:03.882382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:47:03.882382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:03.882382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:47:03.882382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:47:03.882382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:03.882382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:47:03.882382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:03.882382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:47:03.882382 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:47:03.882669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:03.882669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:47:03.882669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:03.882669 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:47:03.883218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:03.883218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:47:03.883218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:47:03.883218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:47:03.883218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:47:03.883218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:47:03.883218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:47:03.883218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:47:03.883218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:47:03.883218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:47:03.883218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:47:03.883218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:47:03.883218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:47:03.883218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:47:03.883218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:47:03.883218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:47:03.883218 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:47:03.883591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:47:03.883591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:47:03.883591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:47:03.883591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:03.883591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:47:03.883591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:03.883591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:47:03.883591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:03.883591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:47:03.883591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:03.883591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:47:03.883591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:03.883591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:47:03.883591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:47:03.883591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:03.883591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:47:03.883591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:03.883591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:47:03.883591 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:47:03.883978 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:03.883978 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:47:03.883978 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:03.883978 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:47:03.889380 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:47:03.889681 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:47:03.889825 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:47:03.890077 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:47:03.890501 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:47:03.890922 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:47:03.891157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:03.891949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:47:03.891949 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:03.891949 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:47:03.891949 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:47:03.891949 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:03.891949 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:47:03.891949 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:03.891949 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:47:03.893512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:03.893512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:47:03.893512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:47:03.893512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:47:03.893512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:47:03.893512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:47:03.893512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:47:03.893512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:03.893512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:47:03.893512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:03.893512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:47:03.893512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:03.893905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:47:03.893905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:03.893905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:47:03.893905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:03.893905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:47:03.893905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:47:03.893905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:03.893905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:47:03.893905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:03.893905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:47:03.893905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:47:03.893905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:03.893905 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:47:03.894225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:03.894225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:47:03.895425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:03.895425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:47:03.895425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:47:03.895425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:47:03.895425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:47:03.895425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:47:03.895425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:47:03.895425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:47:03.895425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:47:03.895425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:47:03.895425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:47:03.895425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:47:03.895425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:47:03.895425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:47:03.895425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:47:03.895425 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:47:03.895893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:47:03.895893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:47:03.895893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:03.895893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:47:03.895893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:03.895893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:47:03.895893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:03.895893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:47:03.895893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:03.895893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:47:03.895893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:03.895893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:47:03.895893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:47:03.895893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:03.895893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:47:03.895893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:03.895893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:47:03.895893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:47:03.895893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:03.896456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:47:03.896456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:03.896456 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:47:08.680847 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:47:08.681830 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:47:08.681830 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:47:08.682635 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:47:08.717703 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:47:08.717999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:08.718597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:47:08.718597 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:08.718597 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:47:08.718597 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:47:08.718597 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:08.718597 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:47:08.718597 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:08.718597 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:47:08.719613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:08.719613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:47:08.719613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:47:08.719613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:47:08.719613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:47:08.719613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:47:08.719613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:47:08.719613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:08.719613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:47:08.719613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:08.719613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:47:08.719613 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:08.720222 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:47:08.720222 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:08.720222 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:47:08.720222 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:08.720222 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:47:08.720222 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:47:08.720222 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:08.720222 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:47:08.720222 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:08.720222 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:47:08.720222 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:47:08.720222 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:08.720222 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:47:08.720597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:08.720597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:47:08.720771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:08.720771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:47:08.720771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:47:08.720771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:47:08.720771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:47:08.720771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:47:08.720771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:47:08.720771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:47:08.720771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:47:08.720771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:47:08.720771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:47:08.720771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:47:08.720771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:47:08.720771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:47:08.720771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:47:08.720771 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:47:08.721375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:47:08.721375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:47:08.721375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:08.721375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:47:08.721375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:08.721375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:47:08.721375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:08.721375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:47:08.721375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:08.721375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:47:08.721375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:08.721375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:47:08.721375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:47:08.721375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:08.721375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:47:08.721375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:08.721375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:47:08.721375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:47:08.721375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:08.722267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:47:08.722267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:08.722267 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:47:08.864941 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:47:08.865769 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:47:08.865973 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:47:08.867269 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:47:08.867522 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:47:08.867906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:08.868339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:47:08.868339 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:08.868339 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:47:08.868339 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:47:08.868339 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:08.868339 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:47:08.868339 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:08.868339 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:47:08.870076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:08.870076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:47:08.870076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:47:08.870076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:47:08.870076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:47:08.870076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:47:08.870076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:47:08.870076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:08.870076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:47:08.870076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:08.870076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:47:08.870076 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:08.870598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:47:08.870598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:08.870598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:47:08.870598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:08.870598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:47:08.870598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:47:08.870598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:08.870598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:47:08.870598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:08.870598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:47:08.870598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:47:08.870598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:08.870598 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:47:08.871127 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:08.871127 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:47:08.871896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:08.871896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:47:08.871896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:47:08.871896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:47:08.871896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:47:08.871896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:47:08.871896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:47:08.871896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:47:08.871896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:47:08.871896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:47:08.871896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:47:08.871896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:47:08.871896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:47:08.871896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:47:08.871896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:47:08.871896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:47:08.872469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:47:08.872469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:47:08.872469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:08.872469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:47:08.872469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:08.872469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:47:08.872469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:08.872469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:47:08.872469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:08.872469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:47:08.872469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:08.872469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:47:08.872469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:47:08.872469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:08.872469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:47:08.872469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:08.872469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:47:08.872469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:47:08.872469 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:08.873097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:47:08.873097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:08.873097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:47:08.873097 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:47:08.873097 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:47:08.873097 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:47:08.873333 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:47:08.873904 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:47:08.874148 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:08.874892 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:47:08.874892 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:08.874892 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:47:08.874892 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:47:08.874892 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:08.874892 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:47:08.874892 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:08.874892 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:47:08.875824 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:08.875824 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:47:08.875824 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:47:08.875824 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:47:08.875824 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:08.875824 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:47:08.875824 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:08.875824 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:47:08.875824 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:08.875824 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:47:08.875824 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:08.875824 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:47:08.875824 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:08.875824 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:47:08.875824 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:47:08.876421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:08.876421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:47:08.876421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:08.876421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:47:08.876421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:47:08.876421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:08.876421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:47:08.876421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:08.876421 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:47:08.876732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:08.876732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:47:08.876732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:47:08.876732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:47:08.876732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:47:08.876732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:47:08.876732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:47:08.876732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:47:08.876732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:47:08.876732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:47:08.876732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:08.876732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:47:08.877157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:08.877157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:47:08.877157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:08.877157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:47:08.877157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:08.877157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:47:08.877157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:08.877157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:47:08.877157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:47:08.877157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:08.877157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:47:08.877157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:08.877157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:47:08.877157 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:47:08.877576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:08.877576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:47:08.877576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:08.877576 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:47:08.877724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:08.877724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:47:08.877724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:47:08.877724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:47:08.877724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:47:08.877724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:47:08.877724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:47:08.877724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:47:08.877724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:47:08.877724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:47:08.877724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:47:08.877724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:47:08.877724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:47:08.877724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:47:08.877724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:47:08.877724 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:47:08.878225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:47:08.878225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:47:08.878225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:47:08.878225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:47:08.878225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:47:08.878225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:08.878225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:47:08.878225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:08.878225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:47:08.878225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:08.878225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:47:08.878225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:08.878225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:47:08.878225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:08.878225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:47:08.878225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:47:08.878225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:08.878225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:47:08.879609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:08.879609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:47:08.879609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:47:08.879609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:08.879609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:47:08.879609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:08.879609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:47:08.879609 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:47:08.879609 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:47:08.879609 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:47:08.880330 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:47:08.880330 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:47:08.880330 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:08.880939 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:47:08.880939 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:08.880939 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:47:08.880939 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:47:08.880939 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:08.880939 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:47:08.880939 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:08.880939 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:47:08.881676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:08.881676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:47:08.881676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:47:08.881676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:47:08.881676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:08.881676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:47:08.881676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:08.881676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:47:08.881676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:08.881676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:47:08.881676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:08.881676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:47:08.881676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:08.881676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:47:08.881676 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:47:08.882168 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:08.882168 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:47:08.882168 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:08.882168 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:47:08.882168 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:47:08.882168 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:08.882168 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:47:08.882168 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:08.882168 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:47:08.882563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:08.882563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:47:08.882563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:47:08.882563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:47:08.882563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:47:08.882563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:47:08.882563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:47:08.882563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:47:08.882563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:47:08.882563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:08.882563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:47:08.883551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:08.883551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:47:08.883551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:08.883551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:47:08.883551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:08.883551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:47:08.883551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:08.883551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:47:08.883551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:47:08.883551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:08.883551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:47:08.883551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:08.883551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:47:08.883551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:47:08.884688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:08.884688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:47:08.884688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:08.884688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:47:08.884688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:08.884688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:47:08.884688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:47:08.884688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:47:08.884688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:47:08.884688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:47:08.884688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:47:08.884688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:47:08.884688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:47:08.884688 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:47:08.885603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:47:08.885603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:47:08.885603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:47:08.885603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:47:08.885603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:47:08.885603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:47:08.885603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:47:08.885603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:47:08.885603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:47:08.885603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:47:08.885603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:08.885603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:47:08.885603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:08.885603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:47:08.885603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:08.885603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:47:08.885603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:08.885603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:47:08.885603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:08.886856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:47:08.886856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:47:08.886856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:08.886856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:47:08.886856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:08.886856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:47:08.886856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:47:08.886856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:08.886856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:47:08.886856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:08.886856 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:47:08.890006 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:47:08.890431 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:47:08.890573 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:47:08.890728 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:47:08.891362 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:47:08.891610 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:47:08.891774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:08.892454 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:47:08.892454 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:08.892454 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:47:08.892454 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:47:08.892454 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:08.892454 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:47:08.892454 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:08.892454 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:47:08.893802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:08.893802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:47:08.893802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:47:08.893802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:47:08.893802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:47:08.893802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:47:08.893802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:47:08.893802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:08.893802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:47:08.893802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:08.893802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:47:08.893802 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:08.894078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:47:08.894078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:08.894078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:47:08.894078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:08.894078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:47:08.894078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:47:08.894078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:08.894078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:47:08.894078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:08.894078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:47:08.894078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:47:08.894078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:08.894078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:47:08.894394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:08.894394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:47:08.895386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:08.895386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:47:08.895386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:47:08.895386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:47:08.895386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:47:08.895386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:47:08.895386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:47:08.895386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:47:08.895386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:47:08.895386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:47:08.895386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:47:08.895386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:47:08.895386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:47:08.895386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:47:08.895386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:47:08.895386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:47:08.895910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:47:08.895910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:47:08.895910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:08.895910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:47:08.895910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:08.895910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:47:08.895910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:08.895910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:47:08.895910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:08.895910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:47:08.895910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:08.895910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:47:08.895910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:47:08.895910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:08.895910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:47:08.895910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:08.895910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:47:08.895910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:47:08.895910 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:08.896460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:47:08.896460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:08.896460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:47:13.681415 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:47:13.681923 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:47:13.682028 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:47:13.682677 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:47:13.707708 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:47:13.708028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:13.708723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:47:13.708723 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:13.708723 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:47:13.708723 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:47:13.708723 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:13.708723 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:47:13.708723 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:13.708723 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:47:13.709627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.027 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:13.709627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:47:13.709627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:47:13.709627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:47:13.709627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:47:13.709627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:47:13.709627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:47:13.709627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:13.709627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:47:13.709627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:13.709627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:47:13.709627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:13.710359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:47:13.710359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:13.710359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:47:13.710359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:13.710359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:47:13.710359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:47:13.710359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:13.710359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:47:13.710359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:13.710359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:47:13.710359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:47:13.710359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:13.710359 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:47:13.711424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:13.711424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:47:13.711424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:13.711424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:47:13.711424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:47:13.711424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:47:13.711424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:47:13.711424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:47:13.711424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:47:13.711424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:47:13.711424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:47:13.711424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:47:13.711424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:47:13.711424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:47:13.711424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:47:13.712287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:47:13.712287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:47:13.712287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:47:13.712287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:47:13.712287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:47:13.712287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:13.712287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:47:13.712287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:13.712287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:47:13.712287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:13.712287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:47:13.712287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:13.712287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:47:13.712287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:13.712287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:47:13.712287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:47:13.712287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:13.712287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:47:13.712287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:13.714463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:47:13.714463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:47:13.714463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:13.714463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:47:13.714463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:13.714463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:47:13.865671 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:47:13.866374 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:47:13.866458 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:47:13.867230 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:47:13.867418 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:47:13.867540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:13.868069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:47:13.868069 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:13.868069 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:47:13.868069 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:47:13.868069 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:13.868069 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:47:13.868069 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:13.868069 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:47:13.868974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:13.868974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:47:13.868974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:47:13.868974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:47:13.868974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:47:13.868974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:47:13.868974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:47:13.868974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:13.868974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:47:13.868974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:13.868974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:47:13.868974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:13.869494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:47:13.869494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:13.869494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:47:13.869494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:13.869494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:47:13.869494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:47:13.869494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:13.869494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:47:13.869494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:13.869494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:47:13.869494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:47:13.869494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:13.869494 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:47:13.869894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:13.869894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:47:13.869894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:13.869894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:47:13.869894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:47:13.869894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:47:13.869894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:47:13.869894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:47:13.869894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:47:13.869894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:47:13.869894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:47:13.869894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:47:13.869894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:47:13.869894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:47:13.869894 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:47:13.870346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:47:13.870346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:47:13.870346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:47:13.870346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:47:13.870346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:47:13.870346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:13.870346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:47:13.870346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:13.870346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:47:13.870346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:13.870346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:47:13.870346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:13.870346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:47:13.870346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:13.870346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:47:13.870346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:47:13.870346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:13.870346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:47:13.870346 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:13.871035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:47:13.871035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:47:13.871035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:13.871035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:47:13.871035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:13.871035 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:47:13.872809 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:47:13.873129 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:47:13.873216 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:47:13.873278 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:47:13.873595 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:47:13.873783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:13.874236 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:47:13.874236 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:13.874236 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:47:13.874236 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:47:13.874236 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:13.874236 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:47:13.874236 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:13.874236 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:47:13.875093 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:13.875093 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:47:13.875093 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:47:13.875093 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:47:13.875093 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:13.875093 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:47:13.875093 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:13.875093 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:47:13.875093 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:13.875093 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:47:13.875093 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:13.875093 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:47:13.875093 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:13.875093 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:47:13.875093 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:47:13.876006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:13.876006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:47:13.876006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:13.876006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:47:13.876006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:47:13.876006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:13.876006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:47:13.876006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:13.876006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:47:13.876006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:13.876006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:47:13.876006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:47:13.876006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:47:13.876006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:47:13.877478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:47:13.877478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:47:13.877478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:47:13.877478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:47:13.877478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:47:13.877478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:13.877478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:47:13.877478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:13.877478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:47:13.877478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:13.877478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:47:13.877478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:13.877478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:47:13.878017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:13.878017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:47:13.878017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:47:13.878017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:13.878017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:47:13.878017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:13.878017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:47:13.878017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:47:13.878017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:13.878017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:47:13.878017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:13.878017 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:47:13.878423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:13.878423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:47:13.878423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:47:13.878423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:47:13.878423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:47:13.878423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:47:13.878423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:47:13.878423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:47:13.878423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:47:13.878423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:47:13.878423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:47:13.878423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:47:13.878423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:47:13.878423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:47:13.878423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:47:13.878423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:47:13.879909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:47:13.879909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:47:13.879909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:47:13.879909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:47:13.879909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:47:13.879909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:13.879909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:47:13.879909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:13.879909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:47:13.879909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:13.879909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:47:13.879909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:13.879909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:47:13.879909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:13.879909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:47:13.879909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:47:13.879909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:13.879909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:47:13.881345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:13.881345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:47:13.881345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:47:13.881345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:13.881345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:47:13.881345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:13.881345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:47:13.881345 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:47:13.881345 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:47:13.881345 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:47:13.881641 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:47:13.881641 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:47:13.881933 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:13.882603 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:47:13.882603 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:13.882603 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:47:13.882603 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:47:13.882603 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:13.882603 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:47:13.882603 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:13.882603 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:47:13.883696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:13.883696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:47:13.883696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:47:13.883696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:47:13.883696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:13.883696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:47:13.883696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:13.883696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:47:13.883696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:13.883696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:47:13.883696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:13.883696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:47:13.883696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:13.883696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:47:13.883696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:47:13.884478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:13.884478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:47:13.884478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:13.884478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:47:13.884478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:47:13.884478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:13.884478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:47:13.884478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:13.884478 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:47:13.885069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:13.885069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:47:13.885069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:47:13.885069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:47:13.885069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:47:13.885069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:47:13.885069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:47:13.885069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:47:13.885069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:47:13.885069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:13.885069 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:47:13.885569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:13.885569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:47:13.885569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:13.885569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:47:13.885569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:13.885569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:47:13.885569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:13.885569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:47:13.885569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:47:13.885569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:13.885569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:47:13.885569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:13.885569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:47:13.885569 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:47:13.886128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:13.886128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:47:13.886128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:13.886128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:47:13.887471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:13.887471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:47:13.887471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:47:13.887471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:47:13.887471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:47:13.887471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:47:13.887471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:47:13.887471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:47:13.887471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:47:13.887471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:47:13.887471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:47:13.887471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:47:13.887471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:47:13.887471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:47:13.887471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:47:13.887471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:47:13.887471 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:47:13.888609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:47:13.888609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:47:13.888609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:47:13.888609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:13.888609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:47:13.888609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:13.888609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:47:13.888609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:13.888609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:47:13.888609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:13.888609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:47:13.888609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:13.888609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:47:13.888609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:47:13.888609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:13.888609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:47:13.888609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:13.888609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:47:13.888609 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:47:13.889257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:13.889257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:47:13.889257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:13.889257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:47:13.890544 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:47:13.890893 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:47:13.890991 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:47:13.891211 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:47:13.894386 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:47:13.894679 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:47:13.894838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:13.895577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:47:13.895577 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:13.895577 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:47:13.895577 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:47:13.895577 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:13.895577 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:47:13.895577 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:13.895577 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:47:13.896342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:13.896342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:47:13.896342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:47:13.896342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:47:13.896342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:47:13.896342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:47:13.896342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:47:13.896342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:13.896342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:47:13.896342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:13.896342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:47:13.896342 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:13.896831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:47:13.896831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:13.896831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:47:13.896831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:13.896831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:47:13.896831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:47:13.896831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:13.896831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:47:13.896831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:13.896831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:47:13.896831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:47:13.896831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:13.896831 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:47:13.897188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:13.897188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:47:13.897188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:13.897188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:47:13.897188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:47:13.897188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:47:13.897188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:47:13.897188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:47:13.897188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:47:13.897188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:47:13.897188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:47:13.897188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:47:13.897188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:47:13.897188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:47:13.897188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:47:13.897641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:47:13.897641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:47:13.897641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:47:13.897641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:47:13.897641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:47:13.897641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:13.897641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:47:13.897641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:13.897641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:47:13.897641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:13.897641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:47:13.897641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:13.897641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:47:13.897641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:13.897641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:47:13.897641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:47:13.897641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:13.897641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:47:13.897641 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:13.898215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:47:13.898215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:47:13.898215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:13.898215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:47:13.898215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:13.898215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:47:18.682248 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:47:18.682936 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:47:18.682936 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:47:18.684128 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:47:18.718573 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:47:18.718823 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:18.719543 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:47:18.719543 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:18.719543 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:47:18.719543 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:47:18.719543 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:18.719543 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:47:18.719543 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:18.719543 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:47:18.720443 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:18.720443 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:47:18.720443 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:47:18.720443 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:47:18.720443 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:47:18.720443 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:47:18.720443 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:47:18.720443 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:18.720443 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:47:18.720443 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:18.720443 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:47:18.720443 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:18.720788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:47:18.720788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:18.720788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:47:18.720788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:18.720788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:47:18.720788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:47:18.720788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:18.720788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:47:18.720788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:18.720788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:47:18.720788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:47:18.720788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:18.720788 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:47:18.721093 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:18.721093 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:47:18.721292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:18.721292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:47:18.721292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:47:18.721292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:47:18.721292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:47:18.721292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:47:18.721292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:47:18.721292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:47:18.721292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:47:18.721292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:47:18.721292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:47:18.721292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:47:18.721292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:47:18.721292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:47:18.721292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:47:18.721292 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:47:18.721683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:47:18.721683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:47:18.721683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:18.721683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:47:18.721683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:18.721683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:47:18.721683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:18.721683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:47:18.721683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:18.721683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:47:18.721683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:18.721683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:47:18.721683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:47:18.721683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:18.721683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:47:18.721683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:18.721683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:47:18.721683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:47:18.721683 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:18.722101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:47:18.722101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:18.722101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:47:18.866461 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:47:18.867081 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:47:18.867182 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:47:18.868234 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:47:18.868504 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:47:18.868632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:18.869241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:47:18.869241 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:18.869241 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:47:18.869241 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:47:18.869241 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:18.869241 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:47:18.869241 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:18.869241 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:47:18.870152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:18.870152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:47:18.870152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:47:18.870152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:47:18.870152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:47:18.870152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:47:18.870152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:47:18.870152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:18.870152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:47:18.870152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:18.870152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:47:18.870152 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:18.870434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:47:18.870434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:18.870434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:47:18.870434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:18.870434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:47:18.870434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:47:18.870434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:18.870434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:47:18.870434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:18.870434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:47:18.870434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:47:18.870434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:18.870434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:47:18.870694 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:18.870694 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:47:18.870950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:18.870950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:47:18.870950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:47:18.870950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:47:18.870950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:47:18.870950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:47:18.870950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:47:18.870950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:47:18.870950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:47:18.870950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:47:18.870950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:47:18.870950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:47:18.870950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:47:18.870950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:47:18.870950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:47:18.870950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:47:18.871287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:47:18.871287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:47:18.871287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:18.871287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:47:18.871287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:18.871287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:47:18.871287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:18.871287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:47:18.871287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:18.871287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:47:18.871287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:18.871287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:47:18.871287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:47:18.871287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:18.871287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:47:18.871287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:18.871287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:47:18.871287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:47:18.871287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:18.871656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:47:18.871656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:18.871656 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:47:18.873313 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:47:18.873564 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:47:18.873625 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:47:18.873760 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:47:18.874293 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:47:18.874523 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:18.875118 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:47:18.875118 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:18.875118 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:47:18.875118 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:47:18.875118 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:18.875118 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:47:18.875118 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:18.875118 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:47:18.875709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:18.875709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:47:18.875709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:47:18.875709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:47:18.875709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:18.875709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:47:18.875709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:18.875709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:47:18.875709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:18.875709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:47:18.875709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:18.875709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:47:18.875709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:18.875709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:47:18.875709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:47:18.876068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:18.876068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:47:18.876068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:18.876068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:47:18.876068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:47:18.876068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:18.876068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:47:18.876068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:18.876068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:47:18.876450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:18.876450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:47:18.876450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:47:18.876450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:47:18.876450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:47:18.876450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:47:18.876450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:47:18.876450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:47:18.876450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:47:18.876450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:47:18.876450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:18.876450 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:47:18.876726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:18.876726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:47:18.876726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:18.876726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:47:18.876726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:18.876726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:47:18.876726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:18.876726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:47:18.876726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:47:18.876726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:18.876726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:47:18.876726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:18.876726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:47:18.876726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:47:18.877001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:18.877001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:47:18.877001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:18.877001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:47:18.877212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:18.877212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:47:18.877212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:47:18.877212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:47:18.877212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:47:18.877212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:47:18.877212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:47:18.877212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:47:18.877212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:47:18.877212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:47:18.877212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:47:18.877212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:47:18.877212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:47:18.877212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:47:18.877212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:47:18.877212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:47:18.877530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:47:18.877530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:47:18.877530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:47:18.877530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:47:18.877530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:47:18.877530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:18.877530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:47:18.877530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:18.877530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:47:18.877530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:18.877530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:47:18.877530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:18.877530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:47:18.877530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:18.877530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:47:18.877530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:47:18.877530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:18.877530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:47:18.877906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:18.877906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:47:18.877906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:47:18.877906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:18.877906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:47:18.877906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:18.877906 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:47:18.880345 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:47:18.880600 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:47:18.880662 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:47:18.880828 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:47:18.881300 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:47:18.881520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:18.882072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:47:18.882072 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:18.882072 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:47:18.882072 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:47:18.882072 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:18.882072 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:47:18.882072 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:18.882072 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:47:18.882636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:18.882636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:47:18.882636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:47:18.882636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:47:18.882636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:18.882636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:47:18.882636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:18.882636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:47:18.882636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:18.882636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:47:18.882636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:18.882636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:47:18.882636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:18.882636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:47:18.882636 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:47:18.882971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:18.882971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:47:18.882971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:18.882971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:47:18.882971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:47:18.882971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:18.882971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:47:18.882971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:18.882971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:47:18.883281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:18.883281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:47:18.883281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:47:18.883281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:47:18.883281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:47:18.883281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:47:18.883281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:47:18.883281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:47:18.883281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:47:18.883281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:18.883281 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:47:18.883559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:18.883559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:47:18.883559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:18.883559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:47:18.883559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:18.883559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:47:18.883559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:18.883559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:47:18.883559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:47:18.883559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:18.883559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:47:18.883559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:18.883559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:47:18.883559 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:47:18.883854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:18.883854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:47:18.883854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:18.883854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:47:18.884061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:18.884061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:47:18.884061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:47:18.884061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:47:18.884061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:47:18.884061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:47:18.884061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:47:18.884061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:47:18.884061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:47:18.884061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:47:18.884061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:47:18.884061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:47:18.884061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:47:18.884061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:47:18.884061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:47:18.884061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:47:18.884061 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:47:18.884531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:47:18.884531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:47:18.884531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:47:18.884531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:18.884531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:47:18.884531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:18.884531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:47:18.884531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:18.884531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:47:18.884531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:18.884531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:47:18.884531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:18.884531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:47:18.884531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:47:18.884531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:18.884531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:47:18.884531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:18.884531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:47:18.884531 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:47:18.884908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:18.884908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:47:18.884908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:18.884908 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:47:18.892337 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:47:18.892689 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:47:18.892803 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:47:18.892950 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:47:18.893522 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:47:18.893762 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:47:18.893916 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:18.894498 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:47:18.894498 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:18.894498 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:47:18.894498 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:47:18.894498 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:18.894498 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:47:18.894498 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:18.894498 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:47:18.895371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:18.895371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:47:18.895371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:47:18.895371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:47:18.895371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:47:18.895371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:47:18.895371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:47:18.895371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:18.895371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:47:18.895371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:18.895371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:47:18.895371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:18.895660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:47:18.895660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:18.895660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:47:18.895660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:18.895660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:47:18.895660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:47:18.895660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:18.895660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:47:18.895660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:18.895660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:47:18.895660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:47:18.895660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:18.895660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:47:18.895940 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:18.895940 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:47:18.896123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:18.896123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:47:18.896123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:47:18.896123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:47:18.896123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:47:18.896123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:47:18.896123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:47:18.896123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:47:18.896123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:47:18.896123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:47:18.896123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:47:18.896123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:47:18.896123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:47:18.896123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:47:18.896123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:47:18.896123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:47:18.896447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:47:18.896447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:47:18.896447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:18.896447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:47:18.896447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:18.896447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:47:18.896447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:18.896447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:47:18.896447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:18.896447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:47:18.896447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:18.896447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:47:18.896447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:47:18.896447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:18.896447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:47:18.896447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:18.896447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:47:18.896447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:47:18.896447 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:18.896865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:47:18.896865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:18.896865 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:47:23.682958 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:47:23.683646 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:47:23.684074 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:47:23.685656 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:47:23.724611 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:47:23.724882 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:23.725590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:47:23.725590 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:23.725590 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:47:23.725590 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:47:23.725590 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:23.725590 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:47:23.725590 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:23.725590 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:47:23.726577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:23.726577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:47:23.726577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:47:23.726577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:47:23.726577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:47:23.726577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:47:23.726577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:47:23.726577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:23.726577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:47:23.726577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:23.726577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:47:23.726577 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:23.726952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:47:23.726952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:23.726952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:47:23.726952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:23.726952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:47:23.726952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:47:23.726952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:23.726952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:47:23.726952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:23.726952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:47:23.726952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:47:23.726952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:23.726952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:47:23.727188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:23.727188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:47:23.727460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:23.727460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:47:23.727460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:47:23.727460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:47:23.727460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:47:23.727460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:47:23.727460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:47:23.727460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:47:23.727460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:47:23.727460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:47:23.727460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:47:23.727460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:47:23.727460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:47:23.727460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:47:23.727460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:47:23.727460 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:47:23.727874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:47:23.727874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:47:23.727874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:23.727874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:47:23.727874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:23.727874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:47:23.727874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:23.727874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:47:23.727874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:23.727874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:47:23.727874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:23.727874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:47:23.727874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:47:23.727874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:23.727874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:47:23.727874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:23.727874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:47:23.727874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:47:23.727874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:23.728225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:47:23.728225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:23.728225 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:47:23.867091 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:47:23.867882 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:47:23.868126 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:47:23.869632 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:47:23.870099 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:47:23.870355 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:23.871427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:47:23.871427 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:23.871427 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:47:23.871427 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:47:23.871427 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:23.871427 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:47:23.871427 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:23.871427 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:47:23.873212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:23.873212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:47:23.873212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:47:23.873212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:47:23.873212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:47:23.873212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:47:23.873212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:47:23.873212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:23.873212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:47:23.873212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:23.873212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:47:23.873212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:23.873571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:47:23.873571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:23.873571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:47:23.873571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:23.873571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:47:23.873571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:47:23.873571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:23.873571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:47:23.873571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:23.873571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:47:23.873571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:47:23.873571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:23.873571 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:47:23.873930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:23.873930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:47:23.875258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:23.875258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:47:23.875258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:47:23.875258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:47:23.875258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:47:23.875258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:47:23.875258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:47:23.875258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:47:23.875258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:47:23.875258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:47:23.875258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:47:23.875258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:47:23.875258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:47:23.875258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:47:23.875258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:47:23.875258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:47:23.875769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:47:23.875769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:47:23.875769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:23.875769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:47:23.875769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:23.875769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:47:23.875769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:23.875769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:47:23.875769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:23.875769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:47:23.875769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:23.875769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:47:23.875769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:47:23.875769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:23.875769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:47:23.875769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:23.875769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:47:23.875769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:47:23.875769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:23.876139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:47:23.876139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:23.876139 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:47:23.876139 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:47:23.876139 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:47:23.876139 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:47:23.876826 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:47:23.877072 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:47:23.877503 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:23.878377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:47:23.878377 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:23.878377 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:47:23.878377 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:47:23.878377 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:23.878377 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:47:23.878377 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:23.878377 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:47:23.879216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:23.879216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:47:23.879216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:47:23.879216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:47:23.879216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:23.879216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:47:23.879216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:23.879216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:47:23.879216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:23.879216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:47:23.879216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:23.879216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:47:23.879216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:23.879216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:47:23.879216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:47:23.879551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:23.879551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:47:23.879551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:23.879551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:47:23.879551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:47:23.879551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:23.879551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:47:23.879551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:23.879551 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:47:23.880034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:23.880034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:47:23.880034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:47:23.880034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:47:23.880034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:47:23.880034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:47:23.880034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:47:23.880034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:47:23.880034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:47:23.880034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:47:23.880034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:23.880034 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:47:23.880361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:23.880361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:47:23.880361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:23.880361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:47:23.880361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:23.880361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:47:23.880361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:23.880361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:47:23.880361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:47:23.880361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:23.880361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:47:23.880361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:23.880361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:47:23.880361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:47:23.880653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:23.880653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:47:23.880653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:23.880653 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:47:23.881321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:23.881321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:47:23.881321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:47:23.881321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:47:23.881321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:47:23.881321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:47:23.881321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:47:23.881321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:47:23.881321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:47:23.881321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:47:23.881321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:47:23.881321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:47:23.881321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:47:23.881321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:47:23.881321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:47:23.881321 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:47:23.881681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:47:23.881681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:47:23.881681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:47:23.881681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:47:23.881681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:47:23.881681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:23.881681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:47:23.881681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:23.881681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:47:23.881681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:23.881681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:47:23.881681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:23.881681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:47:23.881681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:23.881681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:47:23.881681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:47:23.881681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:23.881681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:47:23.882039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:23.882039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:47:23.882039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:47:23.882039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:23.882039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:47:23.882039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:23.882039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:47:23.882039 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:47:23.882039 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:47:23.882039 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:47:23.882256 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:47:23.882656 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:47:23.882709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:23.883285 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:47:23.883285 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:23.883285 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:47:23.883285 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:47:23.883285 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:23.883285 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:47:23.883285 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:23.883285 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:47:23.884072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:23.884072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:47:23.884072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:47:23.884072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:47:23.884072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:23.884072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:47:23.884072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:23.884072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:47:23.884072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:23.884072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:47:23.884072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:23.884072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:47:23.884072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:23.884072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:47:23.884072 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:47:23.884415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:23.884415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:47:23.884415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:23.884415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:47:23.884415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:47:23.884415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:23.884415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:47:23.884415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:23.884415 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:47:23.885033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:23.885033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:47:23.885033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:47:23.885033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:47:23.885033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:47:23.885033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:47:23.885033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:47:23.885033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:47:23.885033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:47:23.885033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:23.885033 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:47:23.885295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:23.885295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:47:23.885295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:23.885295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:47:23.885295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:23.885295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:47:23.885295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:23.885295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:47:23.885295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:47:23.885295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:23.885295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:47:23.885295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:23.885295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:47:23.885295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:47:23.885589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:23.885589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:47:23.885589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:23.885589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:47:23.886888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:23.886888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:47:23.886888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:47:23.886888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:47:23.886888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:47:23.886888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:47:23.886888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:47:23.886888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:47:23.886888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:47:23.886888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:47:23.886888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:47:23.886888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:47:23.886888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:47:23.886888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:47:23.886888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:47:23.886888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:47:23.886888 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:47:23.887681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:47:23.887681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:47:23.887681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:47:23.887681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:23.887681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:47:23.887681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:23.887681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:47:23.887681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:23.887681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:47:23.887681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:23.887681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:47:23.887681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:23.887681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:47:23.887681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:47:23.887681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:23.887681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:47:23.887681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:23.887681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:47:23.887681 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:47:23.888123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:23.888123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:47:23.888123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:23.888123 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:47:23.892778 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:47:23.893413 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:47:23.893413 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:47:23.893413 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:47:23.893860 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:47:23.894210 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:47:23.894437 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:23.897632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:47:23.897632 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:23.897632 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:47:23.897632 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:47:23.897632 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:23.897632 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:47:23.897632 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:23.897632 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:47:23.897632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:23.897632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:47:23.897632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:47:23.897632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:47:23.897632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:47:23.897632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:47:23.897632 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:47:23.898703 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:23.898703 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:47:23.898703 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:23.898703 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:47:23.898703 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:23.898703 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:47:23.898703 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:23.898703 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:47:23.898703 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:23.898703 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:47:23.898703 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:47:23.898703 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:23.898703 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:47:23.898703 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:23.899409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:47:23.899409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:47:23.899409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:23.899409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:47:23.899409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:23.899409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:47:23.899409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:23.899409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:47:23.899409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:47:23.899409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:47:23.899409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:47:23.899409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:47:23.899409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:47:23.899409 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:47:23.900155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:47:23.900155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:47:23.900155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:47:23.900155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:47:23.900155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:47:23.900155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:47:23.900155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:47:23.900155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:47:23.900155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:47:23.900155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:47:23.900155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:23.900155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:47:23.900155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:23.900155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:47:23.900155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:23.900155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:47:23.900155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:23.900155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:47:23.900155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:23.900155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:47:23.901204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:47:23.901204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:23.901204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:47:23.901204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:23.901204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:47:23.901204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:47:23.901204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:23.901204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:47:23.901204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:23.901204 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:47:28.683632 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:47:28.684354 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:47:28.684354 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:47:28.687608 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:47:28.716122 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:47:28.716427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:28.717322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:47:28.717322 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:28.717322 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:47:28.717322 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:47:28.717322 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:28.717322 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:47:28.717322 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:28.717322 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:47:28.718146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.033 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:28.718146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:47:28.718146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:47:28.718146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:47:28.718146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:47:28.718146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:47:28.718146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:47:28.718146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:28.718146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:47:28.718146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:28.718146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:47:28.718146 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:28.718857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:47:28.718857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:28.718857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:47:28.718857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:28.718857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:47:28.718857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:47:28.718857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:28.718857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:47:28.718857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:28.718857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:47:28.718857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:47:28.718857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:28.718857 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:47:28.719458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:28.719458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:47:28.719458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:28.719458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:47:28.719458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:47:28.719458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:47:28.719458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:47:28.719458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:47:28.719458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:47:28.719458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:47:28.719458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:47:28.719458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:47:28.719458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:47:28.719458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:47:28.719458 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:47:28.720200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:47:28.720200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:47:28.720200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:47:28.720200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:47:28.720200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:47:28.720200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:28.720200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:47:28.720200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:28.720200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:47:28.720200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:28.720200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:47:28.720200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:28.720200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:47:28.720200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:28.720200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:47:28.720200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:47:28.720200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:28.720200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:47:28.720200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:28.721155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:47:28.721155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:47:28.721155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:28.721155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:47:28.721155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:28.721155 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:47:28.868336 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:47:28.868336 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:47:28.868605 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:47:28.869461 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:47:28.869713 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:47:28.869917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:28.870587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:47:28.870587 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:28.870587 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:47:28.870587 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:47:28.870587 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:28.870587 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:47:28.870587 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:28.870587 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:47:28.871870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:28.871870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:47:28.871870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:47:28.871870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:47:28.871870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:47:28.871870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:47:28.871870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:47:28.871870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:28.871870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:47:28.871870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:28.871870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:47:28.871870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:28.872410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:47:28.872410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:28.872410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:47:28.872410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:28.872410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:47:28.872410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:47:28.872410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:28.872410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:47:28.872410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:28.872410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:47:28.872410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:47:28.872410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:28.872410 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:47:28.872782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:28.872782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:47:28.872782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:28.872782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:47:28.872782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:47:28.872782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:47:28.872782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:47:28.872782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:47:28.872782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:47:28.872782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:47:28.872782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:47:28.872782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:47:28.872782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:47:28.872782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:47:28.872782 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:47:28.873301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:47:28.873301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:47:28.873301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:47:28.873301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:47:28.873301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:47:28.873301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:28.873301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:47:28.873301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:28.873301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:47:28.873301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:28.873301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:47:28.873301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:28.873301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:47:28.873301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:28.873301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:47:28.873301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:47:28.873301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:28.873301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:47:28.873301 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:28.873770 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:47:28.873770 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:47:28.873770 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:28.873770 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:47:28.873770 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:28.873770 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:47:28.875889 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:47:28.876138 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:47:28.876246 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:47:28.876364 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:47:28.876704 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:47:28.876893 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:28.877365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:47:28.877365 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:28.877365 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:47:28.877365 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:47:28.877365 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:28.877365 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:47:28.877365 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:28.877365 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:47:28.878124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:28.878124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:47:28.878124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:47:28.878124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:47:28.878124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:28.878124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:47:28.878124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:28.878124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:47:28.878124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:28.878124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:47:28.878124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:28.878124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:47:28.878124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:28.878124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:47:28.878124 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:47:28.878671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:28.878671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:47:28.878671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:28.878671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:47:28.878671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:47:28.878671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:28.878671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:47:28.878671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:28.878671 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:47:28.878999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:28.878999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:47:28.878999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:47:28.878999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:47:28.878999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:47:28.878999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:47:28.878999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:47:28.878999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:47:28.878999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:47:28.878999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:47:28.878999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:28.878999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:47:28.879379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:28.879379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:47:28.879379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:28.879379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:47:28.879379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:28.879379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:47:28.879379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:28.879379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:47:28.879379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:47:28.879379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:28.879379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:47:28.879379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:28.879379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:47:28.879379 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:47:28.879866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:28.879866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:47:28.879866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:28.879866 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:47:28.880011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:28.880011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:47:28.880011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:47:28.880011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:47:28.880011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:47:28.880011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:47:28.880011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:47:28.880011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:47:28.880011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:47:28.880011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:47:28.880011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:47:28.880011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:47:28.880011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:47:28.880011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:47:28.880011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:47:28.880011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:47:28.880553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:47:28.880553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:47:28.880553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:47:28.880553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:47:28.880553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:47:28.880553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:28.880553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:47:28.880553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:28.880553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:47:28.880553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:28.880553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:47:28.880553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:28.880553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:47:28.880553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:28.880553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:47:28.880553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:47:28.880553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:28.880553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:47:28.881212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:28.881212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:47:28.881212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:47:28.881212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:28.881212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:47:28.881212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:28.881212 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:47:28.882095 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:47:28.882095 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:47:28.882095 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:47:28.882453 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:47:28.883113 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:47:28.883466 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:28.884243 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:47:28.884243 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:28.884243 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:47:28.884243 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:47:28.884243 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:28.884243 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:47:28.884243 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:28.884243 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:47:28.884941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:28.884941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:47:28.884941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:47:28.884941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:47:28.884941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:28.884941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:47:28.884941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:28.884941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:47:28.884941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:28.884941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:47:28.884941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:28.884941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:47:28.884941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:28.884941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:47:28.884941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:47:28.885242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:28.885242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:47:28.885242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:28.885242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:47:28.885242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:47:28.885242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:28.885242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:47:28.885242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:28.885242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:47:28.885809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:28.885809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:47:28.885809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:47:28.885809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:47:28.885809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:47:28.885809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:47:28.885809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:47:28.885809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:47:28.885809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:47:28.885809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:28.885809 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:47:28.886086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:28.886086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:47:28.886086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:28.886086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:47:28.886086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:28.886086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:47:28.886086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:28.886086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:47:28.886086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:47:28.886086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:28.886086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:47:28.886086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:28.886086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:47:28.886086 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:47:28.886365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:28.886365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:47:28.886365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:28.886365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:47:28.886881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:28.886881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:47:28.886881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:47:28.886881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:47:28.886881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:47:28.886881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:47:28.886881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:47:28.886881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:47:28.886881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:47:28.886881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:47:28.886881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:47:28.886881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:47:28.886881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:47:28.886881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:47:28.886881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:47:28.886881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:47:28.886881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:47:28.887286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:47:28.887286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:47:28.887286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:47:28.887286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:28.887286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:47:28.887286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:28.887286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:47:28.887286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:28.887286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:47:28.887286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:28.887286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:47:28.887286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:28.887286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:47:28.887286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:47:28.887286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:28.887286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:47:28.887286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:28.887286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:47:28.887286 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:47:28.887666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:28.887666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:47:28.887666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:28.887666 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:47:28.893157 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:47:28.893368 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:47:28.893481 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:47:28.893642 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:47:28.894286 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:47:28.894480 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:47:28.894644 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:28.897488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:47:28.897488 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:28.897488 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:47:28.897488 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:47:28.897488 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:28.897488 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:47:28.897488 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:28.897488 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:47:28.897488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:28.897488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:47:28.897488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:47:28.897488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:47:28.897488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:47:28.897488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:47:28.897488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:47:28.898181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:28.898181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:47:28.898181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:28.898181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:47:28.898181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:28.898181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:47:28.898181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:28.898181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:47:28.898181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:28.898181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:47:28.898181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:47:28.898181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:28.898181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:47:28.898181 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:28.898700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:47:28.898700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:47:28.898700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:28.898700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:47:28.898700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:28.898700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:47:28.898700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:28.898700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:47:28.898700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:47:28.898700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:47:28.898700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:47:28.898700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:47:28.898700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:47:28.898700 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:47:28.899546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:47:28.899546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:47:28.899546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:47:28.899546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:47:28.899546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:47:28.899546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:47:28.899546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:47:28.899546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:47:28.899546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:47:28.899546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:47:28.899546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:28.899546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:47:28.899546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:28.899546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:47:28.899546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:28.899546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:47:28.899546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:28.899546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:47:28.899546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:28.899546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:47:28.900339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:47:28.900339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:28.900339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:47:28.900339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:28.900339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:47:28.900339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:47:28.900339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:28.900339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:47:28.900339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:28.900339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:47:33.684094 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:47:33.685691 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:47:33.685691 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:47:33.686995 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:47:33.722718 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:47:33.722942 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:33.723650 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:47:33.723650 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:33.723650 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:47:33.723650 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:47:33.723650 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:33.723650 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:47:33.723650 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:33.723650 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:47:33.724602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:33.724602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:47:33.724602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:47:33.724602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:47:33.724602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:47:33.724602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:47:33.724602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:47:33.724602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:33.724602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:47:33.724602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:33.724602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:47:33.724602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:33.725130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:47:33.725130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:33.725130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:47:33.725130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:33.725130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:47:33.725130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:47:33.725130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:33.725130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:47:33.725130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:33.725130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:47:33.725130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:47:33.725130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:33.725130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:47:33.725652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:33.725652 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:47:33.725917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:33.725917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:47:33.725917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:47:33.725917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:47:33.725917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:47:33.725917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:47:33.725917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:47:33.725917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:47:33.725917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:47:33.725917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:47:33.725917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:47:33.725917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:47:33.725917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:47:33.725917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:47:33.725917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:47:33.725917 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:47:33.726628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:47:33.726628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:47:33.726628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:33.726628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:47:33.726628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:33.726628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:47:33.726628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:33.726628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:47:33.726628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:33.726628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:47:33.726628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:33.726628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:47:33.726628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:47:33.726628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:33.726628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:47:33.726628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:33.726628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:47:33.726628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:47:33.726628 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:33.727538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:47:33.727538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:33.727538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:47:33.868475 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:47:33.868795 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:47:33.868895 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:47:33.869778 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:47:33.870036 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:47:33.870210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:33.870827 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:47:33.870827 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:33.870827 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:47:33.870827 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:47:33.870827 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:33.870827 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:47:33.870827 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:33.870827 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:47:33.871611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:33.871611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:47:33.871611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:47:33.871611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:47:33.871611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:47:33.871611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:47:33.871611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:47:33.871611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:33.871611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:47:33.871611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:33.871611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:47:33.871611 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:33.872322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:47:33.872322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:33.872322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:47:33.872322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:33.872322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:47:33.872322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:47:33.872322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:33.872322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:47:33.872322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:33.872322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:47:33.872322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:47:33.872322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:33.872322 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:47:33.873070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:33.873070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:47:33.873070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:33.873070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:47:33.873070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:47:33.873070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:47:33.873070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:47:33.873070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:47:33.873070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:47:33.873070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:47:33.873070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:47:33.873070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:47:33.873070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:47:33.873070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:47:33.873070 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:47:33.873930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:47:33.873930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:47:33.873930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:47:33.873930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:47:33.873930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:47:33.873930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:33.873930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:47:33.873930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:33.873930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:47:33.873930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:33.873930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:47:33.873930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:33.873930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:47:33.873930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:33.873930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:47:33.873930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:47:33.873930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:33.873930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:47:33.873930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:33.874941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:47:33.874941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:47:33.874941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:33.874941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:47:33.874941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:33.874941 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:47:33.876369 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:47:33.876615 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:47:33.876717 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:47:33.876866 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:47:33.877442 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:47:33.877775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:33.878457 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:47:33.878457 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:33.878457 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:47:33.878457 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:47:33.878457 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:33.878457 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:47:33.878457 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:33.878457 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:47:33.879256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:33.879256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:47:33.879256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:47:33.879256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:47:33.879256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:33.879256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:47:33.879256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:33.879256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:47:33.879256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:33.879256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:47:33.879256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:33.879256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:47:33.879256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:33.879256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:47:33.879256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:47:33.880120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:33.880120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:47:33.880120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:33.880120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:47:33.880120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:47:33.880120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:33.880120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:47:33.880120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:33.880120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:47:33.880120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:33.880120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:47:33.880120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:47:33.880120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:47:33.880120 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:47:33.880663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:47:33.880663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:47:33.880663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:47:33.880663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:47:33.880663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:47:33.880663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:33.880663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:47:33.880663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:33.880663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:47:33.880663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:33.880663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:47:33.880663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:33.880663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:47:33.881000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:33.881000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:47:33.881000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:47:33.881000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:33.881000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:47:33.881000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:33.881000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:47:33.881000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:47:33.881000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:33.881000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:47:33.881000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:33.881000 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:47:33.881307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:33.881307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:47:33.881307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:47:33.881307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:47:33.881307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:47:33.881307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:47:33.881307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:47:33.881307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:47:33.881307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:47:33.881307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:47:33.881307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:47:33.881307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:47:33.881307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:47:33.881307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:47:33.881307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:47:33.881307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:47:33.881803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:47:33.881803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:47:33.881803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:47:33.881803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:47:33.881803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:47:33.881803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:33.881803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:47:33.881803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:33.881803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:47:33.881803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:33.881803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:47:33.881803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:33.881803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:47:33.881803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:33.881803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:47:33.881803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:47:33.881803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:33.881803 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:47:33.882251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:33.882251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:47:33.882251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:47:33.882251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:33.882251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:47:33.882251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:33.882251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:47:33.882251 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:47:33.882957 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:47:33.882957 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:47:33.882957 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:47:33.883379 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:47:33.883651 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:33.884362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:47:33.884362 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:33.884362 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:47:33.884362 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:47:33.884362 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:33.884362 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:47:33.884362 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:33.884362 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:47:33.885125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:33.885125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:47:33.885125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:47:33.885125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:47:33.885125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:33.885125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:47:33.885125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:33.885125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:47:33.885125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:33.885125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:47:33.885125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:33.885125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:47:33.885125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:33.885125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:47:33.885125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:47:33.885521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:33.885521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:47:33.885521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:33.885521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:47:33.885521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:47:33.885521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:33.885521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:47:33.885521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:33.885521 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:47:33.886037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:33.886037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:47:33.886037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:47:33.886037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:47:33.886037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:47:33.886037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:47:33.886037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:47:33.886037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:47:33.886037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:47:33.886037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:33.886037 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:47:33.886452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:33.886452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:47:33.886452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:33.886452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:47:33.886452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:33.886452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:47:33.886452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:33.886452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:47:33.886452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:47:33.886452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:33.886452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:47:33.886452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:33.886452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:47:33.886452 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:47:33.886900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:33.886900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:47:33.886900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:33.886900 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:47:33.887109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:33.887109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:47:33.887109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:47:33.887109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:47:33.887109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:47:33.887109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:47:33.887109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:47:33.887109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:47:33.887109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:47:33.887109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:47:33.887109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:47:33.887109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:47:33.887109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:47:33.887109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:47:33.887109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:47:33.887109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:47:33.887109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:47:33.887556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:47:33.887556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:47:33.887556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:47:33.887556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:33.887556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:47:33.887556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:33.887556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:47:33.887556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:33.887556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:47:33.887556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:33.887556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:47:33.887556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:33.887556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:47:33.887556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:47:33.887556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:33.887556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:47:33.887556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:33.887556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:47:33.887556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:47:33.888160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:33.888160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:47:33.888160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:33.888160 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:47:33.893573 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:47:33.893864 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:47:33.893980 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:47:33.894127 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:47:33.894740 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:47:33.895193 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:47:33.895429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:33.896272 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:47:33.896272 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:33.896272 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:47:33.896272 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:47:33.896272 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:33.896272 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:47:33.896272 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:33.896272 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:47:33.897565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:33.897565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:47:33.897565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:47:33.897565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:47:33.897565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:47:33.897565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:47:33.897565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:47:33.897565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:33.897565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:47:33.897565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:33.897565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:47:33.897565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:33.898145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:47:33.898145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:33.898145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:47:33.898145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:33.898145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:47:33.898145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:47:33.898145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:33.898145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:47:33.898145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:33.898145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:47:33.898145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:47:33.898145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:33.898145 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:47:33.898783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:33.898783 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:47:33.899300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:33.899300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:47:33.899300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:47:33.899300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:47:33.899300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:47:33.899300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:47:33.899300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:47:33.899300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:47:33.899300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:47:33.899300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:47:33.899300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:47:33.899300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:47:33.899300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:47:33.899300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:47:33.899300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:47:33.899300 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:47:33.900156 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:47:33.900156 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:47:33.900156 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:33.900156 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:47:33.900156 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:33.900156 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:47:33.900156 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:33.900156 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:47:33.900156 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:33.900156 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:47:33.900156 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:33.900156 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:47:33.900156 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:47:33.900156 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:33.900156 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:47:33.900156 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:33.900156 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:47:33.900156 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:47:33.900156 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:33.901112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:47:33.901112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:33.901112 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:47:38.685878 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:47:38.686501 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:47:38.686501 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:47:38.687269 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:47:38.722396 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:47:38.722686 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:38.723610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:47:38.723610 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:38.723610 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:47:38.723610 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:47:38.723610 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:38.723610 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:47:38.723610 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:38.723610 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:47:38.724736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:38.724736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:47:38.724736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:47:38.724736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:47:38.724736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:47:38.724736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:47:38.724736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:47:38.724736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:38.724736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:47:38.724736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:38.724736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:47:38.724736 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:38.725356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:47:38.725356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:38.725356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:47:38.725356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:38.725356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:47:38.725356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:47:38.725356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:38.725356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:47:38.725356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:38.725356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:47:38.725356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:47:38.725356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:38.725356 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:47:38.726151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:38.726151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:47:38.726151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:38.726151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:47:38.726151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:47:38.726151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:47:38.726151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:47:38.726151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:47:38.726151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:47:38.726151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:47:38.726151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:47:38.726151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:47:38.726151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:47:38.726151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:47:38.726151 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:47:38.727257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:47:38.727257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:47:38.727257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:47:38.727257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:47:38.727257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:47:38.727257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:38.727257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:47:38.727257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:38.727257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:47:38.727257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:38.727257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:47:38.727257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:38.727257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:47:38.727257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:38.727257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:47:38.727257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:47:38.727257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:38.727257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:47:38.727257 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:38.729492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:47:38.729492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:47:38.729492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:38.729492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:47:38.729492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:38.729492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:47:38.869154 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:47:38.870052 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:47:38.870335 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:47:38.871623 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:47:38.871953 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:47:38.872196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:38.873140 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:47:38.873140 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:38.873140 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:47:38.873140 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:47:38.873140 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:38.873140 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:47:38.873140 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:38.873140 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:47:38.874899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:38.874899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:47:38.874899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:47:38.874899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:47:38.874899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:47:38.874899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:47:38.874899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:47:38.874899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:38.874899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:47:38.874899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:38.874899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:47:38.874899 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:38.875565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:47:38.875565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:38.875565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:47:38.875565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:38.875565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:47:38.875565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:47:38.875565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:38.875565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:47:38.875565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:38.875565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:47:38.875565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:47:38.875565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:38.875565 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:47:38.876161 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:38.876161 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:47:38.876936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:38.876936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:47:38.876936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:47:38.876936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:47:38.876936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:47:38.876936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:47:38.876936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:47:38.876936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:47:38.876936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:47:38.876936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:47:38.876936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:47:38.876936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:47:38.876936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:47:38.876936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:47:38.876936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:47:38.876936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:47:38.877971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:47:38.877971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:47:38.877971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:38.877971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:47:38.877971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:38.877971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:47:38.877971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:38.877971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:47:38.877971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:38.877971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:47:38.877971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:38.877971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:47:38.877971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:47:38.877971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:38.877971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:47:38.877971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:38.877971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:47:38.877971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:47:38.877971 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:38.879180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:47:38.879180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:38.879180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:47:38.879180 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:47:38.879180 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:47:38.879180 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:47:38.879180 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:47:38.879180 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:47:38.879732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:38.879732 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:47:38.879732 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:38.879732 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:47:38.879732 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:47:38.879732 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:38.879732 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:47:38.879732 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:38.879732 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:47:38.880451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:38.880451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:47:38.880451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:47:38.880451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:47:38.880451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:38.880451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:47:38.880451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:38.880451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:47:38.880451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:38.880451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:47:38.880451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:38.880451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:47:38.880451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:38.880451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:47:38.880451 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:47:38.881129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:38.881129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:47:38.881129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:38.881129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:47:38.881129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:47:38.881129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:38.881129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:47:38.881129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:38.881129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:47:38.881129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:38.881129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:47:38.881129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:47:38.881129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:47:38.881129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:47:38.881606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:47:38.881606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:47:38.881606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:47:38.881606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:47:38.881606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:47:38.881606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:38.881606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:47:38.881606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:38.881606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:47:38.881606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:38.881606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:47:38.881606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:38.881606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:47:38.882078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:38.882078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:47:38.882078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:47:38.882078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:38.882078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:47:38.882078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:38.882078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:47:38.882078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:47:38.882078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:38.882078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:47:38.882078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:38.882078 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:47:38.882554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:38.882554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:47:38.882554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:47:38.882554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:47:38.882554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:47:38.882554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:47:38.882554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:47:38.882554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:47:38.882554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:47:38.882554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:47:38.882554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:47:38.882554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:47:38.882554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:47:38.882554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:47:38.882554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:47:38.882554 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:47:38.883645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:47:38.883645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:47:38.883645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:47:38.883645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:47:38.883645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:47:38.883645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:38.883645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:47:38.883645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:38.883645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:47:38.883645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:38.883645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:47:38.883645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:38.883645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:47:38.883645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:38.883645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:47:38.883645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:47:38.883645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:38.883645 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:47:38.884491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:38.884491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:47:38.884491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:47:38.884491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:38.884491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:47:38.884491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:38.884491 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:47:38.884491 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:47:38.884491 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:47:38.884491 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:47:38.885129 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:47:38.885129 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:47:38.885129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:38.885129 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:47:38.885129 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:38.885129 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:47:38.885129 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:47:38.885129 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:38.885129 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:47:38.885129 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:38.885129 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:47:38.885657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:38.885657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:47:38.885657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:47:38.885657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:47:38.885657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:38.885657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:47:38.885657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:38.885657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:47:38.885657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:38.885657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:47:38.885657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:38.885657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:47:38.885657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:38.885657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:47:38.885657 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:47:38.886318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:38.886318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:47:38.886318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:38.886318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:47:38.886318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:47:38.886318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:38.886318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:47:38.886318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:38.886318 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:47:38.886768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:38.886768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:47:38.886768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:47:38.886768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:47:38.886768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:47:38.886768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:47:38.886768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:47:38.886768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:47:38.886768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:47:38.886768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:38.886768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:47:38.887242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:38.887242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:47:38.887242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:38.887242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:47:38.887242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:38.887242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:47:38.887242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:38.887242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:47:38.887242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:47:38.887242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:38.887242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:47:38.887242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:38.887242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:47:38.887242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:47:38.887870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:38.887870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:47:38.887870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:38.887870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:47:38.887870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:38.887870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:47:38.887870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:47:38.887870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:47:38.887870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:47:38.887870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:47:38.887870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:47:38.887870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:47:38.887870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:47:38.887870 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:47:38.888602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:47:38.888602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:47:38.888602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:47:38.888602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:47:38.888602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:47:38.888602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:47:38.888602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:47:38.888602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:47:38.888602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:47:38.888602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:47:38.888602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:38.888602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:47:38.888602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:38.888602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:47:38.888602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:38.888602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:47:38.888602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:38.888602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:47:38.888602 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:38.891607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:47:38.891607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:47:38.891607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:38.891607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:47:38.891607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:38.891607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:47:38.891607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:47:38.891607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:38.891607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:47:38.891607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:38.891607 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:47:38.894048 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:47:38.894381 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:47:38.894782 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:47:38.894782 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:47:38.895496 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:47:38.895542 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:47:38.895682 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:38.896284 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:47:38.896284 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:38.896284 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:47:38.896284 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:47:38.896284 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:38.896284 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:47:38.896284 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:38.896284 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:47:38.897192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:38.897192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:47:38.897192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:47:38.897192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:47:38.897192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:47:38.897192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:47:38.897192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:47:38.897192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:38.897192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:47:38.897192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:38.897192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:47:38.897192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:38.897462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:47:38.897462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:38.897462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:47:38.897462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:38.897462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:47:38.897462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:47:38.897462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:38.897462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:47:38.897462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:38.897462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:47:38.897462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:47:38.897462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:38.897462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:47:38.897710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:38.897710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:47:38.898188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:38.898188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:47:38.898188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:47:38.898188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:47:38.898188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:47:38.898188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:47:38.898188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:47:38.898188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:47:38.898188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:47:38.898188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:47:38.898188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:47:38.898188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:47:38.898188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:47:38.898188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:47:38.898188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:47:38.898188 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:47:38.898519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:47:38.898519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:47:38.898519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:38.898519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:47:38.898519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:38.898519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:47:38.898519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:38.898519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:47:38.898519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:38.898519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:47:38.898519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:38.898519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:47:38.898519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:47:38.898519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:38.898519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:47:38.898519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:38.898519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:47:38.898519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:47:38.898519 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:38.898896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:47:38.898896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:38.898896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:47:43.688212 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:47:43.688212 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:47:43.688212 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:47:43.689491 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:47:43.727151 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:47:43.727768 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:43.729972 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:47:43.729972 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:43.729972 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:47:43.729972 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:47:43.729972 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:43.729972 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:47:43.729972 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:43.729972 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:47:43.731231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:43.731231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:47:43.731231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:47:43.731231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:47:43.731231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:47:43.731231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:47:43.731231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:47:43.731231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:43.731231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:47:43.731231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:43.731231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:47:43.731231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:43.731694 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:47:43.731694 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:43.731694 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:47:43.731694 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:43.731694 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:47:43.731694 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:47:43.731694 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:43.731694 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:47:43.731694 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:43.731694 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:47:43.731694 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:47:43.731694 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:43.731694 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:47:43.732115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:43.732115 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:47:43.732855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:43.732855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:47:43.732855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:47:43.732855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:47:43.732855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:47:43.732855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:47:43.732855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:47:43.732855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:47:43.732855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:47:43.732855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:47:43.732855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:47:43.732855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:47:43.732855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:47:43.732855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:47:43.732855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:47:43.732855 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:47:43.733384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:47:43.733384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:47:43.733384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:43.733384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:47:43.733384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:43.733384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:47:43.733384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:43.733384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:47:43.733384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:43.733384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:47:43.733384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:43.733384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:47:43.733384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:47:43.733384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:43.733384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:47:43.733384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:43.733384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:47:43.733384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:47:43.733384 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:43.733992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:47:43.733992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:43.733992 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:47:43.869992 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:47:43.870845 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:47:43.871058 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:47:43.872336 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:47:43.872881 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:47:43.872881 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:43.873728 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:47:43.873728 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:43.873728 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:47:43.873728 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:47:43.873728 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:43.873728 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:47:43.873728 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:43.873728 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:47:43.875573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:43.875573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:47:43.875573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:47:43.875573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:47:43.875573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:47:43.875573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:47:43.875573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:47:43.875573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:43.875573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:47:43.875573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:43.875573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:47:43.875573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:43.875949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:47:43.875949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:43.875949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:47:43.875949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:43.875949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:47:43.875949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:47:43.875949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:43.875949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:47:43.875949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:43.875949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:47:43.875949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:47:43.875949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:43.875949 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:47:43.876199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:43.876199 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:47:43.877473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:43.877473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:47:43.877473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:47:43.877473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:47:43.877473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:47:43.877473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:47:43.877473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:47:43.877473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:47:43.877473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:47:43.877473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:47:43.877473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:47:43.877473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:47:43.877473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:47:43.877473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:47:43.877473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:47:43.877473 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:47:43.877969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:47:43.877969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:47:43.877969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:43.877969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:47:43.877969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:43.877969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:47:43.877969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:43.877969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:47:43.877969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:43.877969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:47:43.877969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:43.877969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:47:43.877969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:47:43.877969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:43.877969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:47:43.877969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:43.877969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:47:43.877969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:47:43.877969 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:43.878512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:47:43.878512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:43.878512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:47:43.878512 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:47:43.878512 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:47:43.878512 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:47:43.878726 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:47:43.879268 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:47:43.879573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:43.880500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:47:43.880500 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:43.880500 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:47:43.880500 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:47:43.880500 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:43.880500 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:47:43.880500 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:43.880500 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:47:43.881392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:43.881392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:47:43.881392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:47:43.881392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:47:43.881392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:43.881392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:47:43.881392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:43.881392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:47:43.881392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:43.881392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:47:43.881392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:43.881392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:47:43.881392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:43.881392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:47:43.881392 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:47:43.882186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:43.882186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:47:43.882186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:43.882186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:47:43.882186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:47:43.882186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:43.882186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:47:43.882186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:43.882186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:47:43.882186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:43.882186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:47:43.882186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:47:43.882186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:47:43.882186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:47:43.882615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:47:43.882615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:47:43.882615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:47:43.882615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:47:43.882615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:47:43.882615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:43.882615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:47:43.882615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:43.882615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:47:43.882615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:43.882615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:47:43.882615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:43.882615 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:47:43.883004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:43.883004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:47:43.883004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:47:43.883004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:43.883004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:47:43.883004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:43.883004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:47:43.883004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:47:43.883004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:43.883004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:47:43.883004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:43.883004 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:47:43.883325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:43.883325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:47:43.883325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:47:43.883325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:47:43.883325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:47:43.883325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:47:43.883325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:47:43.883325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:47:43.883325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:47:43.883325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:47:43.883325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:47:43.883325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:47:43.883325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:47:43.883325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:47:43.883325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:47:43.883325 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:47:43.883868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:47:43.883868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:47:43.883868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:47:43.883868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:47:43.883868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:47:43.883868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:43.883868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:47:43.883868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:43.883868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:47:43.883868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:43.883868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:47:43.883868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:43.883868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:47:43.883868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:43.883868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:47:43.883868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:47:43.883868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:43.883868 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:47:43.884386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:43.884386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:47:43.884386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:47:43.884386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:43.884386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:47:43.884386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:43.884386 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:47:43.884386 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:47:43.884386 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:47:43.884386 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:47:43.884691 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:47:43.884691 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:47:43.884836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:43.885404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:47:43.885404 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:43.885404 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:47:43.885404 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:47:43.885404 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:43.885404 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:47:43.885404 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:43.885404 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:47:43.886202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:43.886202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:47:43.886202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:47:43.886202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:47:43.886202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:43.886202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:47:43.886202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:43.886202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:47:43.886202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:43.886202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:47:43.886202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:43.886202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:47:43.886202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:43.886202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:47:43.886202 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:47:43.886597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:43.886597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:47:43.886597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:43.886597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:47:43.886597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:47:43.886597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:43.886597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:47:43.886597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:43.886597 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:47:43.887121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:43.887121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:47:43.887121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:47:43.887121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:47:43.887121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:47:43.887121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:47:43.887121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:47:43.887121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:47:43.887121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:47:43.887121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:43.887121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:47:43.887487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:43.887487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:47:43.887487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:43.887487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:47:43.887487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:43.887487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:47:43.887487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:43.887487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:47:43.887487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:47:43.887487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:43.887487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:47:43.887487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:43.887487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:47:43.887487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:47:43.887932 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:43.887932 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:47:43.887932 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:43.887932 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:47:43.888340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:43.888340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:47:43.888340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:47:43.888340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:47:43.888340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:47:43.888340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:47:43.888340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:47:43.888340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:47:43.888340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:47:43.888340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:47:43.888340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:47:43.888340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:47:43.888340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:47:43.888340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:47:43.888340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:47:43.888340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:47:43.888340 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:47:43.888853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:47:43.888853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:47:43.888853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:47:43.888853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:43.888853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:47:43.888853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:43.888853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:47:43.888853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:43.888853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:47:43.888853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:43.888853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:47:43.888853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:43.888853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:47:43.888853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:47:43.888853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:43.888853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:47:43.888853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:43.888853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:47:43.888853 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:47:43.889400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:43.889400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:47:43.889400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:43.889400 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:47:43.894774 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:47:43.894929 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:47:43.895040 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:47:43.895196 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:47:43.895707 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:47:43.896147 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:47:43.896370 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:43.897196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:47:43.897196 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:43.897196 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:47:43.897196 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:47:43.897196 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:43.897196 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:47:43.897196 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:43.897196 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:47:43.898619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:43.898619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:47:43.898619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:47:43.898619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:47:43.898619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:47:43.898619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:47:43.898619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:47:43.898619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:43.898619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:47:43.898619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:43.898619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:47:43.898619 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:43.899005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:47:43.899005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:43.899005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:47:43.899005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:43.899005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:47:43.899005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:47:43.899005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:43.899005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:47:43.899005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:43.899005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:47:43.899005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:47:43.899005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:43.899005 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:47:43.899343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:43.899343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:47:43.902694 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:43.902694 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:47:43.902694 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:47:43.902694 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:47:43.902694 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:47:43.902694 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:47:43.902694 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:47:43.902694 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:47:43.902694 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:47:43.902694 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:47:43.902694 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:47:43.902694 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:47:43.902694 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:47:43.902694 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:47:43.902694 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:47:43.902694 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:47:43.903182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:47:43.903182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:47:43.903182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:43.903182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:47:43.903182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:43.903182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:47:43.903182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:43.903182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:47:43.903182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:43.903182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:47:43.903182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:43.903182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:47:43.903182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:47:43.903182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:43.903182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:47:43.903182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:43.903182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:47:43.903182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:47:43.903182 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:43.903859 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:47:43.903859 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:43.903859 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:47:48.688061 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:47:48.688825 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:47:48.689019 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:47:48.690366 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:47:48.724015 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:47:48.724332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:48.725066 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:47:48.725066 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:48.725066 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:47:48.725066 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:47:48.725066 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:48.725066 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:47:48.725066 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:48.725066 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:47:48.725834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:48.725834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:47:48.725834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:47:48.725834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:47:48.725834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:47:48.725834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:47:48.725834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:47:48.725834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:48.725834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:47:48.725834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:48.725834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:47:48.725834 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:48.726530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:47:48.726530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:48.726530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:47:48.726530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:48.726530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:47:48.726530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:47:48.726530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:48.726530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:47:48.726530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:48.726530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:47:48.726530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:47:48.726530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:48.726530 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:47:48.727189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:48.727189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:47:48.727189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:48.727189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:47:48.727189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:47:48.727189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:47:48.727189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:47:48.727189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:47:48.727189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:47:48.727189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:47:48.727189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:47:48.727189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:47:48.727189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:47:48.727189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:47:48.727189 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:47:48.727912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:47:48.727912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:47:48.727912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:47:48.727912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:47:48.727912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:47:48.727912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:48.727912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:47:48.727912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:48.727912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:47:48.727912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:48.727912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:47:48.727912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:48.727912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:47:48.727912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:48.727912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:47:48.727912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:47:48.727912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:48.727912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:47:48.727912 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:48.729214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:47:48.729214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:47:48.729214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:48.729214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:47:48.729214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:48.729214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:47:48.870884 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:47:48.871627 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:47:48.871872 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:47:48.873275 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:47:48.873572 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:47:48.874206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:48.874858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:47:48.874858 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:48.874858 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:47:48.874858 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:47:48.874858 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:48.874858 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:47:48.874858 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:48.874858 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:47:48.876481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:48.876481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:47:48.876481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:47:48.876481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:47:48.876481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:47:48.876481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:47:48.876481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:47:48.876481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:48.876481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:47:48.876481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:48.876481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:47:48.876481 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:48.877245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:47:48.877245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:48.877245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:47:48.877245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:48.877245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:47:48.877245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:47:48.877245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:48.877245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:47:48.877245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:48.877245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:47:48.877245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:47:48.877245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:48.877245 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:47:48.877869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:48.877869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:47:48.878210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:48.878210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:47:48.878210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:47:48.878210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:47:48.878210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:47:48.878210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:47:48.878210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:47:48.878210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:47:48.878210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:47:48.878210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:47:48.878210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:47:48.878210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:47:48.878210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:47:48.878210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:47:48.878210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:47:48.878210 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:47:48.878962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:47:48.878962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:47:48.878962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:48.878962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:47:48.878962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:48.878962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:47:48.878962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:48.878962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:47:48.878962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:48.878962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:47:48.878962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:48.878962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:47:48.878962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:47:48.878962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:48.878962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:47:48.878962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:48.878962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:47:48.878962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:47:48.878962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:48.879806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:47:48.879806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:48.879806 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:47:48.879806 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:47:48.879806 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:47:48.879806 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:47:48.879806 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:47:48.880131 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:47:48.880375 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:48.881362 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:47:48.881362 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:48.881362 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:47:48.881362 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:47:48.881362 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:48.881362 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:47:48.881362 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:48.881362 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:47:48.882427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:48.882427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:47:48.882427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:47:48.882427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:47:48.882427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:48.882427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:47:48.882427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:48.882427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:47:48.882427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:48.882427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:47:48.882427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:48.882427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:47:48.882427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:48.882427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:47:48.882427 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:47:48.882974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:48.882974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:47:48.882974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:48.882974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:47:48.882974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:47:48.882974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:48.882974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:47:48.882974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:48.882974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:47:48.883254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:48.883254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:47:48.883254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:47:48.883254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:47:48.883254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:47:48.883254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:47:48.883254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:47:48.883254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:47:48.883254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:47:48.883254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:47:48.883254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:48.883254 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:47:48.883592 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:48.883592 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:47:48.883592 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:48.883592 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:47:48.883592 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:48.883592 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:47:48.883592 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:48.883592 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:47:48.883592 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:47:48.883592 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:48.883592 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:47:48.883592 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:48.883592 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:47:48.883592 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:47:48.884015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:48.884015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:47:48.884015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:48.884015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:47:48.884417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:48.884417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:47:48.884417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:47:48.884417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:47:48.884417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:47:48.884417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:47:48.884417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:47:48.884417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:47:48.884417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:47:48.884417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:47:48.884417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:47:48.884417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:47:48.884417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:47:48.884417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:47:48.884417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:47:48.884417 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:47:48.884948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:47:48.884948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:47:48.884948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:47:48.884948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:47:48.884948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:47:48.884948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:48.884948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:47:48.884948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:48.884948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:47:48.884948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:48.884948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:47:48.884948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:48.884948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:47:48.884948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:48.884948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:47:48.884948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:47:48.884948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:48.884948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:47:48.885485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:48.885485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:47:48.885485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:47:48.885485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:48.885485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:47:48.885485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:48.885485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:47:48.885485 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:47:48.885739 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:47:48.885739 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:47:48.885831 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:47:48.886280 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:47:48.886561 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:48.887250 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:47:48.887250 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:48.887250 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:47:48.887250 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:47:48.887250 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:48.887250 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:47:48.887250 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:48.887250 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:47:48.887999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:48.887999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:47:48.887999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:47:48.887999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:47:48.887999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:48.887999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:47:48.887999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:48.887999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:47:48.887999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:48.887999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:47:48.887999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:48.887999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:47:48.887999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:48.887999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:47:48.887999 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:47:48.888525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:48.888525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:47:48.888525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:48.888525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:47:48.888525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:47:48.888525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:48.888525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:47:48.888525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:48.888525 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:47:48.888883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:48.888883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:47:48.888883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:47:48.888883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:47:48.888883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:47:48.888883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:47:48.888883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:47:48.888883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:47:48.888883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:47:48.888883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:48.888883 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:47:48.889271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:48.889271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:47:48.889271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:48.889271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:47:48.889271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:48.889271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:47:48.889271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:48.889271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:47:48.889271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:47:48.889271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:48.889271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:47:48.889271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:48.889271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:47:48.889271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:47:48.889665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:48.889665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:47:48.889665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:48.889665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:47:48.889928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:48.889928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:47:48.889928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:47:48.889928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:47:48.889928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:47:48.889928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:47:48.889928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:47:48.889928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:47:48.889928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:47:48.889928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:47:48.889928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:47:48.889928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:47:48.889928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:47:48.889928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:47:48.889928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:47:48.889928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:47:48.889928 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:47:48.890621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:47:48.890621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:47:48.890621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:47:48.890621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:48.890621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:47:48.890621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:48.890621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:47:48.890621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:48.890621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:47:48.890621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:48.890621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:47:48.890621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:48.890621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:47:48.890621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:47:48.890621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:48.890621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:47:48.890621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:48.890621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:47:48.890621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:47:48.891255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:48.891255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:47:48.891255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:48.891255 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:47:48.894999 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:47:48.895283 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:47:48.895393 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:47:48.895547 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:47:48.896001 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:47:48.896270 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:47:48.896438 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:48.897029 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:47:48.897029 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:48.897029 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:47:48.897029 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:47:48.897029 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:48.897029 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:47:48.897029 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:48.897029 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:47:48.897859 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:48.897859 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:47:48.897859 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:47:48.897859 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:47:48.897859 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:47:48.897859 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:47:48.897859 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:47:48.897859 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:48.897859 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:47:48.897859 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:48.897859 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:47:48.897859 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:48.898186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:47:48.898186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:48.898186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:47:48.898186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:48.898186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:47:48.898186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:47:48.898186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:48.898186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:47:48.898186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:48.898186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:47:48.898186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:47:48.898186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:48.898186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:47:48.898541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:48.898541 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:47:48.898816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:48.898816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:47:48.898816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:47:48.898816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:47:48.898816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:47:48.898816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:47:48.898816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:47:48.898816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:47:48.898816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:47:48.898816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:47:48.898816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:47:48.898816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:47:48.898816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:47:48.898816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:47:48.898816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:47:48.898816 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:47:48.899243 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:47:48.899243 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:47:48.899243 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:48.899243 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:47:48.899243 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:48.899243 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:47:48.899243 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:48.899243 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:47:48.899243 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:48.899243 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:47:48.899243 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:48.899243 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:47:48.899243 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:47:48.899243 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:48.899243 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:47:48.899243 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:48.899243 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:47:48.899243 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:47:48.899243 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:48.899793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:47:48.899793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:48.899793 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:47:53.689036 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:47:53.690011 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:47:53.690011 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:47:53.691005 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:47:53.728142 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:47:53.728629 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:53.729723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:47:53.729723 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:53.729723 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:47:53.729723 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:47:53.729723 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:53.729723 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:47:53.729723 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:53.729723 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:47:53.730661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:53.730661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:47:53.730661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:47:53.730661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:47:53.730661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:47:53.730661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:47:53.730661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:47:53.730661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:53.730661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:47:53.730661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:53.730661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:47:53.730661 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:53.731488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:47:53.731488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:53.731488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:47:53.731488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:53.731488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:47:53.731488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:47:53.731488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:53.731488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:47:53.731488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:53.731488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:47:53.731488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:47:53.731488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:53.731488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:47:53.732232 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:53.732232 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:47:53.732232 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:53.732232 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:47:53.732232 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:47:53.732232 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:47:53.732232 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:47:53.732232 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:47:53.732232 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:47:53.732232 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:47:53.732232 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:47:53.732232 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:47:53.732232 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:47:53.732232 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:47:53.732232 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:47:53.733081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:47:53.733081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:47:53.733081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:47:53.733081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:47:53.733081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:47:53.733081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:53.733081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:47:53.733081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:53.733081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:47:53.733081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:53.733081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:47:53.733081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:53.733081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:47:53.733081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:53.733081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:47:53.733081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:47:53.733081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:53.733081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:47:53.733081 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:53.734094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:47:53.734094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:47:53.734094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:53.734094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:47:53.734094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:53.734094 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:47:53.871842 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:47:53.872587 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:47:53.872688 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:47:53.874038 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:47:53.874459 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:47:53.874721 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:53.875863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:47:53.875863 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:53.875863 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:47:53.875863 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:47:53.875863 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:53.875863 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:47:53.875863 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:53.875863 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:47:53.876849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:53.876849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:47:53.876849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:47:53.876849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:47:53.876849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:47:53.876849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:47:53.876849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:47:53.876849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:53.876849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:47:53.876849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:53.876849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:47:53.876849 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:53.877398 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:47:53.877398 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:53.877398 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:47:53.877398 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:53.877398 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:47:53.877398 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:47:53.877398 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:53.877398 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:47:53.877398 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:53.877398 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:47:53.877398 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:47:53.877398 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:53.877398 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:47:53.877967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:53.877967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:47:53.877967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:53.877967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:47:53.877967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:47:53.877967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:47:53.877967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:47:53.877967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:47:53.877967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:47:53.877967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:47:53.877967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:47:53.877967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:47:53.877967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:47:53.877967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:47:53.877967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:47:53.878663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:47:53.878663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:47:53.878663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:47:53.878663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:47:53.878663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:47:53.878663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:53.878663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:47:53.878663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:53.878663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:47:53.878663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:53.878663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:47:53.878663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:53.878663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:47:53.878663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:53.878663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:47:53.878663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:47:53.878663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:53.878663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:47:53.878663 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:53.879482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:47:53.879482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:47:53.879482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:53.879482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:47:53.879482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:53.879482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:47:53.879482 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:47:53.879482 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:47:53.879482 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:47:53.879954 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:47:53.880128 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:47:53.880518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:53.881323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:47:53.881323 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:53.881323 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:47:53.881323 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:47:53.881323 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:53.881323 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:47:53.881323 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:53.881323 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:47:53.882122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:53.882122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:47:53.882122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:47:53.882122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:47:53.882122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:53.882122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:47:53.882122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:53.882122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:47:53.882122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:53.882122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:47:53.882122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:53.882122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:47:53.882122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:53.882122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:47:53.882122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:47:53.883036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:53.883036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:47:53.883036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:53.883036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:47:53.883036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:47:53.883036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:53.883036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:47:53.883036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:53.883036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:47:53.883036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:53.883036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:47:53.883036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:47:53.883036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:47:53.883036 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:47:53.883678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:47:53.883678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:47:53.883678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:47:53.883678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:47:53.883678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:47:53.883678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:53.883678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:47:53.883678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:53.883678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:47:53.883678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:53.883678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:47:53.883678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:53.883678 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:47:53.884093 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:53.884093 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:47:53.884093 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:47:53.884093 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:53.884093 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:47:53.884093 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:53.884093 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:47:53.884093 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:47:53.884093 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:53.884093 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:47:53.884093 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:53.884093 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:47:53.884422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:53.884422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:47:53.884422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:47:53.884422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:47:53.884422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:47:53.884422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:47:53.884422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:47:53.884422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:47:53.884422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:47:53.884422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:47:53.884422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:47:53.884422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:47:53.884422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:47:53.884422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:47:53.884422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:47:53.884422 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:47:53.884886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:47:53.884886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:47:53.884886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:47:53.884886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:47:53.884886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:47:53.884886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:53.884886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:47:53.884886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:53.884886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:47:53.884886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:53.884886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:47:53.884886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:53.884886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:47:53.884886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:53.884886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:47:53.884886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:47:53.884886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:53.884886 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:47:53.885419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:53.885419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:47:53.885419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:47:53.885419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:53.885419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:47:53.885419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:53.885419 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:47:53.885735 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:47:53.886004 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:47:53.886087 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:47:53.886192 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:47:53.886762 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:47:53.887200 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:53.888043 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:47:53.888043 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:53.888043 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:47:53.888043 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:47:53.888043 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:53.888043 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:47:53.888043 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:53.888043 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:47:53.888627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:53.888627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:47:53.888627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:47:53.888627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:47:53.888627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:53.888627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:47:53.888627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:53.888627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:47:53.888627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:53.888627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:47:53.888627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:53.888627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:47:53.888627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:53.888627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:47:53.888627 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:47:53.889125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:53.889125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:47:53.889125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:53.889125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:47:53.889125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:47:53.889125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:53.889125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:47:53.889125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:53.889125 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:47:53.889393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:53.889393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:47:53.889393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:47:53.889393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:47:53.889393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:47:53.889393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:47:53.889393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:47:53.889393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:47:53.889393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:47:53.889393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:53.889393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:47:53.889774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:53.889774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:47:53.889774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:53.889774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:47:53.889774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:53.889774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:47:53.889774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:53.889774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:47:53.889774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:47:53.889774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:53.889774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:47:53.889774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:53.889774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:47:53.889774 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:47:53.890197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:53.890197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:47:53.890197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:53.890197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:47:53.890197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:53.890197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:47:53.890197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:47:53.890197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:47:53.890197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:47:53.890197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:47:53.890197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:47:53.890197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:47:53.890197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:47:53.890197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:47:53.890638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:47:53.890638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:47:53.890638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:47:53.890638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:47:53.890638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:47:53.890638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:47:53.890638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:47:53.890638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:47:53.890638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:47:53.890638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:47:53.890638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:53.890638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:47:53.890638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:53.890638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:47:53.890638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:53.890638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:47:53.890638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:53.890638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:47:53.890638 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:53.891294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:47:53.891294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:47:53.891294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:53.891294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:47:53.891294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:53.891294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:47:53.891294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:47:53.891294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:53.891294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:47:53.891294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:53.891294 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:47:53.895418 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:47:53.895663 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:47:53.895728 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:47:53.895875 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:47:53.896388 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:47:53.896832 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:47:53.897098 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:53.897875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:47:53.897875 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:53.897875 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:47:53.897875 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:47:53.897875 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:53.897875 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:47:53.897875 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:53.897875 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:47:53.898510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:53.898510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:47:53.898510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:47:53.898510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:47:53.898510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:47:53.898510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:47:53.898510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:47:53.898510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:53.898510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:47:53.898510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:53.898510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:47:53.898510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:53.899105 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:47:53.899105 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:53.899105 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:47:53.899105 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:53.899105 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:47:53.899105 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:47:53.899105 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:53.899105 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:47:53.899105 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:53.899105 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:47:53.899105 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:47:53.899105 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:53.899105 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:47:53.899487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:53.899487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:47:53.899487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:53.899487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:47:53.899487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:47:53.899487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:47:53.899487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:47:53.899487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:47:53.899487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:47:53.899487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:47:53.899487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:47:53.899487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:47:53.899487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:47:53.899487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:47:53.899487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:47:53.900231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:47:53.900231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:47:53.900231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:47:53.900231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:47:53.900231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:47:53.900231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:53.900231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:47:53.900231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:53.900231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:47:53.900231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:53.900231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:47:53.900231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:53.900231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:47:53.900231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:53.900231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:47:53.900231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:47:53.900231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:53.900231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:47:53.900231 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:53.901122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:47:53.901122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:47:53.901122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:53.901122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:47:53.901122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:53.901122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:47:58.690079 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:47:58.690806 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:47:58.690806 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:47:58.691959 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:47:58.694580 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG dbcounter [-] [106314] Writing DB stats neutron:SELECT=240 {{(pid=106314) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
May 09 09:47:58.749940 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:47:58.749940 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:58.749940 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:47:58.749940 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:58.749940 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:47:58.749940 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:47:58.749940 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:58.749940 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:47:58.749940 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:58.749940 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:47:58.749940 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:58.749940 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:47:58.749940 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:47:58.750960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:47:58.750960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:47:58.750960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:47:58.750960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:47:58.750960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:58.750960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:47:58.750960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:58.750960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:47:58.750960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:58.750960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:47:58.750960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:58.750960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:47:58.750960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:58.750960 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:47:58.752444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:47:58.752444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:58.752444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:47:58.752444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:58.752444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:47:58.752444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:47:58.752444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:58.752444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:47:58.752444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:58.752444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:47:58.752444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:58.752444 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:47:58.753404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:47:58.753404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:47:58.753404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:47:58.753404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:47:58.753404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:47:58.753404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:47:58.753404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:47:58.753404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:47:58.753404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:47:58.753404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:47:58.753404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:47:58.753404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:47:58.753404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:47:58.753404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:47:58.753404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:47:58.753404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:47:58.753404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:58.753404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:47:58.753404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:58.754572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:47:58.754572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:58.754572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:47:58.754572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:58.754572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:47:58.754572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:58.754572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:47:58.754572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:47:58.754572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:58.754572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:47:58.754572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:58.754572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:47:58.754572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:47:58.754572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:58.754572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:47:58.754572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:58.754572 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:47:58.872894 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:47:58.873896 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:47:58.874107 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:47:58.875476 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:47:58.875947 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:47:58.876214 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:58.877220 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:47:58.877220 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:58.877220 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:47:58.877220 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:47:58.877220 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:58.877220 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:47:58.877220 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:58.877220 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:47:58.879012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:58.879012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:47:58.879012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:47:58.879012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:47:58.879012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:47:58.879012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:47:58.879012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:47:58.879012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:58.879012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:47:58.879012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:58.879012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:47:58.879012 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:58.879659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:47:58.879659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:58.879659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:47:58.879659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:58.879659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:47:58.879659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:47:58.879659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:58.879659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:47:58.879659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:58.879659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:47:58.879659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:47:58.879659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:58.879659 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:47:58.880307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:58.880307 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:47:58.880954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:58.880954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:47:58.880954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:47:58.880954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:47:58.880954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:47:58.880954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:47:58.880954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:47:58.880954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:47:58.880954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:47:58.880954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:47:58.880954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:47:58.880954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:47:58.880954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:47:58.880954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:47:58.880954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:47:58.880954 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:47:58.881794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:47:58.881794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:47:58.881794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:58.881794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:47:58.881794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:58.881794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:47:58.881794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:58.881794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:47:58.881794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:58.881794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:47:58.881794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:58.881794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:47:58.881794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:47:58.881794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:58.881794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:47:58.881794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:58.881794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:47:58.881794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:47:58.881794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:58.882334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:47:58.882334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:58.882334 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:47:58.882334 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:47:58.882334 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:47:58.882334 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:47:58.882334 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:47:58.882865 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:47:58.883224 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:58.884404 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:47:58.884404 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:58.884404 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:47:58.884404 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:47:58.884404 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:58.884404 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:47:58.884404 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:58.884404 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:47:58.885637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:58.885637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:47:58.885637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:47:58.885637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:47:58.885637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:58.885637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:47:58.885637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:58.885637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:47:58.885637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:58.885637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:47:58.885637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:58.885637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:47:58.885637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:58.885637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:47:58.885637 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:47:58.886149 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:58.886149 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:47:58.886149 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:58.886149 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:47:58.886149 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:47:58.886149 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:58.886149 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:47:58.886149 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:58.886149 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:47:58.886810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:58.886810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:47:58.886810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:47:58.886810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:47:58.886810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:47:58.886810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:47:58.886810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:47:58.886810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:47:58.886810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:47:58.886810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:47:58.886810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:58.886810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:47:58.887270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:58.887270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:47:58.887270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:58.887270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:47:58.887270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:58.887270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:47:58.887270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:58.887270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:47:58.887270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:47:58.887270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:58.887270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:47:58.887270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:58.887270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:47:58.887270 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:47:58.887794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:58.887794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:47:58.887794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:58.887794 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:47:58.887794 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:47:58.888229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:58.888229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:47:58.888229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:47:58.888229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:47:58.888229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:47:58.888229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:47:58.888229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:47:58.888229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:47:58.888229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:47:58.888229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:47:58.888229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:47:58.888229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:47:58.888229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:47:58.888229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:47:58.888229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:47:58.888229 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:47:58.888672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:47:58.888672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:47:58.888672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:47:58.888672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:47:58.888672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:47:58.888672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:58.888672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:47:58.888672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:58.888672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:47:58.888672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:58.888672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:47:58.888672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:58.888672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:47:58.888672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:58.888672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:47:58.888672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:47:58.888672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:58.888672 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:47:58.889187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:58.889187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:47:58.889187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:47:58.889187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:58.889187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:47:58.889187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:58.889187 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:47:58.889187 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:47:58.889187 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:47:58.889187 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:47:58.889551 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:47:58.889769 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:58.890540 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:47:58.890540 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:58.890540 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:47:58.890540 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:47:58.890540 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:58.890540 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:47:58.890540 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:58.890540 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:47:58.891298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:58.891298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:47:58.891298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:47:58.891298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:47:58.891298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:58.891298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:47:58.891298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:58.891298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:47:58.891298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:58.891298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:47:58.891298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:58.891298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:47:58.891298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:58.891298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:47:58.891298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:47:58.891810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:58.891810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:47:58.891810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:58.891810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:47:58.891810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:47:58.891810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:58.891810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:47:58.891810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:58.891810 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:47:58.892162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:58.892162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:47:58.892162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:47:58.892162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:47:58.892162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:47:58.892162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:47:58.892162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:47:58.892162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:47:58.892162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:47:58.892162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:58.892162 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:47:58.892512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:58.892512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:47:58.892512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:58.892512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:47:58.892512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:58.892512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:47:58.892512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:58.892512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:47:58.892512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:47:58.892512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:58.892512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:47:58.892512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:58.892512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:47:58.892512 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:47:58.892950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:58.892950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:47:58.892950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:58.892950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:47:58.893192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:58.893192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:47:58.893192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:47:58.893192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:47:58.893192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:47:58.893192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:47:58.893192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:47:58.893192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:47:58.893192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:47:58.893192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:47:58.893192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:47:58.893192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:47:58.893192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:47:58.893192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:47:58.893192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:47:58.893192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:47:58.893192 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:47:58.893705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:47:58.893705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:47:58.893705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:47:58.893705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:58.893705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:47:58.893705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:58.893705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:47:58.893705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:58.893705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:47:58.893705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:58.893705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:47:58.893705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:58.893705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:47:58.893705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:47:58.893705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:58.893705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:47:58.893705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:58.893705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:47:58.893705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:47:58.894216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:58.894216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:47:58.894216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:58.894216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:47:58.896037 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:47:58.896368 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:47:58.896477 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:47:58.896630 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:47:58.897011 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:47:58.897238 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:47:58.897387 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:58.898015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:47:58.898015 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:58.898015 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:47:58.898015 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:47:58.898015 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:58.898015 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:47:58.898015 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:58.898015 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:47:58.898863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:58.898863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:47:58.898863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:47:58.898863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:47:58.898863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:47:58.898863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:47:58.898863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:47:58.898863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:58.898863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:47:58.898863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:58.898863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:47:58.898863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:58.899232 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:47:58.899232 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:58.899232 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:47:58.899232 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:58.899232 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:47:58.899232 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:47:58.899232 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:58.899232 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:47:58.899232 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:58.899232 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:47:58.899232 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:47:58.899232 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:58.899232 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:47:58.899586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:58.899586 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:47:58.899817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:58.899817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:47:58.899817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:47:58.899817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:47:58.899817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:47:58.899817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:47:58.899817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:47:58.899817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:47:58.899817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:47:58.899817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:47:58.899817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:47:58.899817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:47:58.899817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:47:58.899817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:47:58.899817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:47:58.899817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:47:58.900271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:47:58.900271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:47:58.900271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:58.900271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:47:58.900271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:47:58.900271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:47:58.900271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:47:58.900271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:47:58.900271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:47:58.900271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:47:58.900271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:47:58.900271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:47:58.900271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:47:58.900271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:47:58.900271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:47:58.900271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:47:58.900271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:47:58.900271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:47:58.900271 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:47:58.900817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:47:58.900817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:47:58.900817 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:48:03.690890 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:48:03.691708 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:48:03.691987 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:48:03.693539 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:48:03.731775 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:48:03.732298 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:03.733258 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:48:03.733258 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:48:03.733258 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:48:03.733258 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:48:03.733258 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:48:03.733258 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:48:03.733258 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:03.733258 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:48:03.734387 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:03.734387 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:48:03.734387 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:48:03.734387 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:48:03.734387 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:48:03.734387 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:48:03.734387 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:48:03.734387 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:03.734387 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:48:03.734387 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:48:03.734387 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:48:03.734387 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:03.735008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:48:03.735008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:48:03.735008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:48:03.735008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:48:03.735008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:48:03.735008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:48:03.735008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:48:03.735008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:48:03.735008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:48:03.735008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:48:03.735008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:48:03.735008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:48:03.735008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:48:03.735557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:03.735557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:48:03.735557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:03.735557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:48:03.735557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:48:03.735557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:48:03.735557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:48:03.735557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:48:03.735557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:48:03.735557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:48:03.735557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:48:03.735557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:48:03.735557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:48:03.735557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:48:03.735557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:48:03.736288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:48:03.736288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:48:03.736288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:48:03.736288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:48:03.736288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:48:03.736288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:03.736288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:48:03.736288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:48:03.736288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:48:03.736288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:03.736288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:48:03.736288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:48:03.736288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:48:03.736288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:48:03.736288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:48:03.736288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:48:03.736288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:48:03.736288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:48:03.736288 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:48:03.737128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:48:03.737128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:48:03.737128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:48:03.737128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:48:03.737128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:03.737128 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:48:03.873887 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:48:03.874664 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:48:03.874902 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:48:03.876358 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:48:03.876829 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:48:03.877119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:03.878518 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:48:03.878518 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:48:03.878518 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:48:03.878518 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:48:03.878518 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:48:03.878518 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:48:03.878518 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:03.878518 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:48:03.879964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:03.879964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:48:03.879964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:48:03.879964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:48:03.879964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:48:03.879964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:48:03.879964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:48:03.879964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:03.879964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:48:03.879964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:48:03.879964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:48:03.879964 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:03.880670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:48:03.880670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:48:03.880670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:48:03.880670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:48:03.880670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:48:03.880670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:48:03.880670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:48:03.880670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:48:03.880670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:48:03.880670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:48:03.880670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:48:03.880670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:48:03.880670 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:48:03.881327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:03.881327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:48:03.881327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:03.881327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:48:03.881327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:48:03.881327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:48:03.881327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:48:03.881327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:48:03.881327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:48:03.881327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:48:03.881327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:48:03.881327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:48:03.881327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:48:03.881327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:48:03.881327 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:48:03.882305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:48:03.882305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:48:03.882305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:48:03.882305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:48:03.882305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:48:03.882305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:03.882305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:48:03.882305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:48:03.882305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:48:03.882305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:03.882305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:48:03.882305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:48:03.882305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:48:03.882305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:48:03.882305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:48:03.882305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:48:03.882305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:48:03.882305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:48:03.882305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:48:03.883482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:48:03.883482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:48:03.883482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:48:03.883482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:48:03.883482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:03.883482 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:48:03.883482 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:48:03.883482 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:48:03.883482 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:48:03.883482 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:48:03.884015 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:48:03.884015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:03.884015 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:48:03.884015 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:48:03.884015 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:48:03.884015 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:48:03.884015 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:48:03.884015 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:48:03.884015 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:03.884015 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:48:03.885343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:03.885343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:48:03.885343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:48:03.885343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:48:03.885343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:03.885343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:48:03.885343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:48:03.885343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:48:03.885343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:03.885343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:48:03.885343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:48:03.885343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:48:03.885343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:48:03.885343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:48:03.885343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:48:03.885836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:48:03.885836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:48:03.885836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:48:03.885836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:48:03.885836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:48:03.885836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:48:03.885836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:48:03.885836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:03.885836 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:48:03.887437 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:03.887437 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:48:03.887437 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:48:03.887437 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:48:03.887437 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:48:03.887437 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:48:03.887437 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:48:03.887437 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:48:03.887437 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:48:03.887437 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:48:03.887437 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:03.887437 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:48:03.888013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:48:03.888013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:48:03.888013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:03.888013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:48:03.888013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:48:03.888013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:48:03.888013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:48:03.888013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:48:03.888013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:48:03.888013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:48:03.888013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:48:03.888013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:48:03.888013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:48:03.888013 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:48:03.888423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:48:03.888423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:48:03.888423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:03.888423 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:48:03.889510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:03.889510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:48:03.889510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:48:03.889510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:48:03.889510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:48:03.889510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:48:03.889510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:48:03.889510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:48:03.889510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:48:03.889510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:48:03.889510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:48:03.889510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:48:03.889510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:48:03.889510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:48:03.889510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:48:03.889510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:48:03.890221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:48:03.890221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:48:03.890221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:48:03.890221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:48:03.890221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:48:03.890221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:03.890221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:48:03.890221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:48:03.890221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:48:03.890221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:03.890221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:48:03.890221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:48:03.890221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:48:03.890221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:48:03.890221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:48:03.890221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:48:03.890221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:48:03.890221 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:48:03.890698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:48:03.890698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:48:03.890698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:48:03.890698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:48:03.890698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:48:03.890698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:03.890698 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:48:03.890698 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:48:03.890698 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:48:03.890698 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:48:03.891009 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:48:03.891009 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:48:03.891009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:03.891538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:48:03.891538 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:48:03.891538 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:48:03.891538 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:48:03.891538 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:48:03.891538 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:48:03.891538 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:03.891538 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:48:03.892311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:03.892311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:48:03.892311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:48:03.892311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:48:03.892311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:03.892311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:48:03.892311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:48:03.892311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:48:03.892311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:03.892311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:48:03.892311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:48:03.892311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:48:03.892311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:48:03.892311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:48:03.892311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:48:03.892826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:48:03.892826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:48:03.892826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:48:03.892826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:48:03.892826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:48:03.892826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:48:03.892826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:48:03.892826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:03.892826 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:48:03.893500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:03.893500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:48:03.893500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:48:03.893500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:48:03.893500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:48:03.893500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:48:03.893500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:48:03.893500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:48:03.893500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:48:03.893500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:03.893500 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:48:03.893851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:48:03.893851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:48:03.893851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:03.893851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:48:03.893851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:48:03.893851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:48:03.893851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:48:03.893851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:48:03.893851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:48:03.893851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:48:03.893851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:48:03.893851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:48:03.893851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:48:03.893851 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:48:03.894323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:48:03.894323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:48:03.894323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:03.894323 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:48:03.894589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:03.894589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:48:03.894589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:48:03.894589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:48:03.894589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:48:03.894589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:48:03.894589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:48:03.894589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:48:03.894589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:48:03.894589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:48:03.894589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:48:03.894589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:48:03.894589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:48:03.894589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:48:03.894589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:48:03.894589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:48:03.894589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:48:03.895137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:48:03.895137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:48:03.895137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:48:03.895137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:03.895137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:48:03.895137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:48:03.895137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:48:03.895137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:03.895137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:48:03.895137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:48:03.895137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:48:03.895137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:48:03.895137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:48:03.895137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:48:03.895137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:48:03.895137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:48:03.895137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:48:03.895137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:48:03.895137 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:48:03.895706 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:48:03.895706 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:48:03.895706 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:03.895706 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:48:03.896571 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:48:03.896907 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:48:03.897021 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:48:03.897177 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:48:03.897640 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:48:03.898027 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:48:03.898234 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:03.899008 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:48:03.899008 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:48:03.899008 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:48:03.899008 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:48:03.899008 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:48:03.899008 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:48:03.899008 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:03.899008 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:48:03.899859 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:03.899859 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:48:03.899859 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:48:03.899859 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:48:03.899859 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:48:03.899859 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:48:03.899859 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:48:03.899859 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:03.899859 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:48:03.899859 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:48:03.899859 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:48:03.899859 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:03.900610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:48:03.900610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:48:03.900610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:48:03.900610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:48:03.900610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:48:03.900610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:48:03.900610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:48:03.900610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:48:03.900610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:48:03.900610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:48:03.900610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:48:03.900610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:48:03.900610 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:48:03.901241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:03.901241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:48:03.901241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:03.901241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:48:03.901241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:48:03.901241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:48:03.901241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:48:03.901241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:48:03.901241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:48:03.901241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:48:03.901241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:48:03.901241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:48:03.901241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:48:03.901241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:48:03.901241 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:48:03.901996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:48:03.901996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:48:03.901996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:48:03.901996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:48:03.901996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:48:03.901996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:03.901996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:48:03.901996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:48:03.901996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:48:03.901996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:03.901996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:48:03.901996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:48:03.901996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:48:03.901996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:48:03.901996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:48:03.901996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:48:03.901996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:48:03.901996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:48:03.901996 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:48:03.903039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:48:03.903039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:48:03.903039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:48:03.903039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:48:03.903039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:03.903039 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:48:08.691815 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:48:08.692904 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:48:08.692904 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:48:08.693045 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:48:08.729118 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:48:08.729366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:08.730374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:48:08.730374 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:48:08.730374 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:48:08.730374 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:48:08.730374 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:48:08.730374 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:48:08.730374 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:08.730374 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:48:08.731179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:08.731179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:48:08.731179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:48:08.731179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:48:08.731179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:48:08.731179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:48:08.731179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:48:08.731179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:08.731179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:48:08.731179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:48:08.731179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:48:08.731179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:08.731578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:48:08.731578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:48:08.731578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:48:08.731578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:48:08.731578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:48:08.731578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:48:08.731578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:48:08.731578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:48:08.731578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:48:08.731578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:48:08.731578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:48:08.731578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:48:08.731578 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:48:08.731927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:08.731927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:48:08.732180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:08.732180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:48:08.732180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:48:08.732180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:48:08.732180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:48:08.732180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:48:08.732180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:48:08.732180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:48:08.732180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:48:08.732180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:48:08.732180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:48:08.732180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:48:08.732180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:48:08.732180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:48:08.732180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:48:08.732180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:48:08.732634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:48:08.732634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:48:08.732634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:08.732634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:48:08.732634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:48:08.732634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:48:08.732634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:08.732634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:48:08.732634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:48:08.732634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:48:08.732634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:48:08.732634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:48:08.732634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:48:08.732634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:48:08.732634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:48:08.732634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:48:08.732634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:48:08.732634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:48:08.732634 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:48:08.733102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:48:08.733102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:08.733102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:48:08.874798 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:48:08.875299 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:48:08.875480 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:48:08.876410 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:48:08.876674 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:48:08.876863 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:08.877548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:48:08.877548 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:48:08.877548 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:48:08.877548 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:48:08.877548 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:48:08.877548 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:48:08.877548 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:08.877548 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:48:08.879197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:08.879197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:48:08.879197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:48:08.879197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:48:08.879197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:48:08.879197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:48:08.879197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:48:08.879197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:08.879197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:48:08.879197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:48:08.879197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:48:08.879197 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:08.879858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:48:08.879858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:48:08.879858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:48:08.879858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:48:08.879858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:48:08.879858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:48:08.879858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:48:08.879858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:48:08.879858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:48:08.879858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:48:08.879858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:48:08.879858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:48:08.879858 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:48:08.880470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:08.880470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:48:08.880470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:08.880470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:48:08.880470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:48:08.880470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:48:08.880470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:48:08.880470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:48:08.880470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:48:08.880470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:48:08.880470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:48:08.880470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:48:08.880470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:48:08.880470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:48:08.880470 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:48:08.881206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:48:08.881206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:48:08.881206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:48:08.881206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:48:08.881206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:48:08.881206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:08.881206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:48:08.881206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:48:08.881206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:48:08.881206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:08.881206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:48:08.881206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:48:08.881206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:48:08.881206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:48:08.881206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:48:08.881206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:48:08.881206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:48:08.881206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:48:08.881206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:48:08.882130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:48:08.882130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:48:08.882130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:48:08.882130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:48:08.882130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:08.882130 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:48:08.882130 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:48:08.882487 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:48:08.882487 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:48:08.882487 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:48:08.882817 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:48:08.883016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:08.883566 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:48:08.883566 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:48:08.883566 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:48:08.883566 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:48:08.883566 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:48:08.883566 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:48:08.883566 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:08.883566 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:48:08.884365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:08.884365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:48:08.884365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:48:08.884365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:48:08.884365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:08.884365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:48:08.884365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:48:08.884365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:48:08.884365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:08.884365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:48:08.884365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:48:08.884365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:48:08.884365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:48:08.884365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:48:08.884365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:48:08.885179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:48:08.885179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:48:08.885179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:48:08.885179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:48:08.885179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:48:08.885179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:48:08.885179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:48:08.885179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:08.885179 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:48:08.885665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:08.885665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:48:08.885665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:48:08.885665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:48:08.885665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:48:08.885665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:48:08.885665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:48:08.885665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:48:08.885665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:48:08.885665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:48:08.885665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:08.885665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:48:08.886287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:48:08.886287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:48:08.886287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:08.886287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:48:08.886287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:48:08.886287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:48:08.886287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:48:08.886287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:48:08.886287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:48:08.886287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:48:08.886287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:48:08.886287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:48:08.886287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:48:08.886287 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:48:08.886940 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:48:08.886940 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:48:08.886940 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:08.886940 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:48:08.886940 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:08.886940 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:48:08.886940 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:48:08.886940 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:48:08.886940 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:48:08.886940 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:48:08.886940 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:48:08.886940 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:48:08.886940 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:48:08.886940 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:48:08.887654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:48:08.887654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:48:08.887654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:48:08.887654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:48:08.887654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:48:08.887654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:48:08.887654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:48:08.887654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:48:08.887654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:48:08.887654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:48:08.887654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:48:08.887654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:08.887654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:48:08.887654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:48:08.887654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:48:08.887654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:08.887654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:48:08.887654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:48:08.887654 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:48:08.888242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:48:08.888242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:48:08.888242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:48:08.888242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:48:08.888242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:48:08.888242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:48:08.888242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:48:08.888242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:48:08.888242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:48:08.888242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:48:08.888242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:08.888242 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:48:08.890086 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:48:08.890266 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:48:08.890383 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:48:08.890508 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:48:08.891014 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:48:08.891424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:08.892126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:48:08.892126 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:48:08.892126 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:48:08.892126 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:48:08.892126 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:48:08.892126 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:48:08.892126 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:08.892126 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:48:08.892869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:08.892869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:48:08.892869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:48:08.892869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:48:08.892869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:08.892869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:48:08.892869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:48:08.892869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:48:08.892869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:08.892869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:48:08.892869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:48:08.892869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:48:08.892869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:48:08.892869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:48:08.892869 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:48:08.893305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:48:08.893305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:48:08.893305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:48:08.893305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:48:08.893305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:48:08.893305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:48:08.893305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:48:08.893305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:08.893305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:48:08.893712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:08.893712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:48:08.893712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:48:08.893712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:48:08.893712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:48:08.893712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:48:08.893712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:48:08.893712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:48:08.893712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:48:08.893712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:08.893712 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:48:08.894084 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:48:08.894084 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:48:08.894084 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:08.894084 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:48:08.894084 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:48:08.894084 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:48:08.894084 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:48:08.894084 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:48:08.894084 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:48:08.894084 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:48:08.894084 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:48:08.894084 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:48:08.894084 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:48:08.894084 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:48:08.894967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:48:08.894967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:48:08.894967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:08.894967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:48:08.894967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:08.894967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:48:08.894967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:48:08.894967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:48:08.894967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:48:08.894967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:48:08.894967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:48:08.894967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:48:08.894967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:48:08.894967 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:48:08.895790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:48:08.895790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:48:08.895790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:48:08.895790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:48:08.895790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:48:08.895790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:48:08.895790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:48:08.895790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:48:08.895790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:48:08.895790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:48:08.895790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:08.895790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:48:08.895790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:48:08.895790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:48:08.895790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:08.895790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:48:08.895790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:48:08.895790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:48:08.895790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:48:08.896674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:48:08.896674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:48:08.896674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:48:08.896674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:48:08.896674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:48:08.896674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:48:08.896674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:48:08.896674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:48:08.896674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:48:08.896674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:08.896674 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:48:08.897194 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:48:08.897194 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:48:08.897194 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:48:08.897387 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:48:08.897948 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:48:08.898349 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:48:08.898621 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:08.899406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:48:08.899406 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:48:08.899406 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:48:08.899406 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:48:08.899406 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:48:08.899406 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:48:08.899406 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:08.899406 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:48:08.900331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:08.900331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:48:08.900331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:48:08.900331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:48:08.900331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:48:08.900331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:48:08.900331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:48:08.900331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:08.900331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:48:08.900331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:48:08.900331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:48:08.900331 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:08.900973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:48:08.900973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:48:08.900973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:48:08.900973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:48:08.900973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:48:08.900973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:48:08.900973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:48:08.900973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:48:08.900973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:48:08.900973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:48:08.900973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:48:08.900973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:48:08.900973 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:48:08.901463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:08.901463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:48:08.901463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:08.901463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:48:08.901463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:48:08.901463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:48:08.901463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:48:08.901463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:48:08.901463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:48:08.901463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:48:08.901463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:48:08.901463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:48:08.901463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:48:08.901463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:48:08.901463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:48:08.901897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:48:08.901897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:48:08.901897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:48:08.901897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:48:08.901897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:48:08.901897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:08.901897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:48:08.901897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:48:08.901897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:48:08.901897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:08.901897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:48:08.901897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:48:08.901897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:48:08.901897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:48:08.901897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:48:08.901897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:48:08.901897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:48:08.901897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:48:08.901897 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:48:08.902435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:48:08.902435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:48:08.902435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:48:08.902435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:48:08.902435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:08.902435 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:48:13.692366 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:48:13.693050 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:48:13.693050 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:48:13.693615 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:48:13.726640 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:48:13.727106 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:13.728126 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:48:13.728126 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:48:13.728126 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:48:13.728126 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:48:13.728126 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:48:13.728126 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:48:13.728126 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:13.728126 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:48:13.728950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:13.728950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:48:13.728950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:48:13.728950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:48:13.728950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:48:13.728950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:48:13.728950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:48:13.728950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:13.728950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:48:13.728950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:48:13.728950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:48:13.728950 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:13.729573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:48:13.729573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:48:13.729573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:48:13.729573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:48:13.729573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:48:13.729573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:48:13.729573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:48:13.729573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:48:13.729573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:48:13.729573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:48:13.729573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:48:13.729573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:48:13.729573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:48:13.730222 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:13.730222 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:48:13.730222 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:13.730222 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:48:13.730222 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:48:13.730222 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:48:13.730222 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:48:13.730222 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:48:13.730222 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:48:13.730222 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:48:13.730222 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:48:13.730222 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:48:13.730222 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:48:13.730222 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:48:13.730222 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:48:13.730958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:48:13.730958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:48:13.730958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:48:13.730958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:48:13.730958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:48:13.730958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:13.730958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:48:13.730958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:48:13.730958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:48:13.730958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:13.730958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:48:13.730958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:48:13.730958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:48:13.730958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:48:13.730958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:48:13.730958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:48:13.730958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:48:13.730958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:48:13.730958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:48:13.733009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:48:13.733009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:48:13.733009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:48:13.733009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:48:13.733009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:13.733009 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:48:13.875525 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:48:13.875952 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:48:13.876106 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:48:13.882432 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:48:13.890514 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:48:13.897315 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:48:13.915280 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:48:13.915280 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:48:13.915280 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:13.916092 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:48:13.916092 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:48:13.916092 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:48:13.916092 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:48:13.916092 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:48:13.916092 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:48:13.916092 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:13.916092 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:48:13.917319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:13.917319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:48:13.917319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:48:13.917319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:48:13.917319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:48:13.917319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:48:13.917319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:48:13.917319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:13.917319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:48:13.917319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:48:13.917319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:48:13.917319 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:13.918813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:48:13.918813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:48:13.918813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:48:13.918813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:48:13.918813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:48:13.918813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:48:13.918813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:48:13.918813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:48:13.918813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:48:13.918813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:48:13.918813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:48:13.918813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:48:13.918813 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:48:13.919550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:13.919550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:48:13.919550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:13.919550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:48:13.919550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:48:13.919550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:48:13.919550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:48:13.919550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:48:13.919550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:48:13.919550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:48:13.919550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:48:13.919550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:48:13.919550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:48:13.919550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:48:13.919550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:48:13.920215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:48:13.920215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:48:13.920215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:48:13.920215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:48:13.920215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:48:13.920215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:13.920215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:48:13.920215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:48:13.920215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:48:13.920215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:13.920215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:48:13.920215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:48:13.920215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:48:13.920215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:48:13.920215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:48:13.920215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:48:13.920215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:48:13.920215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:48:13.920215 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:48:13.920990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:48:13.920990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:48:13.920990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:48:13.920990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:48:13.920990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:13.920990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:48:13.920990 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:48:13.920990 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:48:13.920990 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:48:13.920990 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:48:13.921429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:13.921429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:48:13.921429 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:48:13.921429 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:48:13.921429 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:48:13.921429 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:48:13.921429 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:48:13.921429 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:13.921429 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:48:13.922343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:13.922343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:48:13.922343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:48:13.922343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:48:13.922343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:13.922343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:48:13.922343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:48:13.922343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:48:13.922343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:13.922343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:48:13.922343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:48:13.922343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:48:13.922343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:48:13.922343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:48:13.922343 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:48:13.923149 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:48:13.923149 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:48:13.923149 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:48:13.923149 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:48:13.923149 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:48:13.923149 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:48:13.923149 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:48:13.923149 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:13.923149 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:48:13.923546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:13.923546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:48:13.923546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:48:13.923546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:48:13.923546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:48:13.923546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:48:13.923546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:48:13.923546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:48:13.923546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:48:13.923546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:48:13.923546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:13.923546 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:48:13.923874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:48:13.923874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:48:13.923874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:13.923874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:48:13.923874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:48:13.923874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:48:13.923874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:48:13.923874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:48:13.923874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:48:13.923874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:48:13.923874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:48:13.923874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:48:13.923874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:48:13.923874 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:48:13.924119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:48:13.924119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:48:13.924119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:13.924119 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:48:13.924588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:13.924588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:48:13.924588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:48:13.924588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:48:13.924588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:48:13.924588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:48:13.924588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:48:13.924588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:48:13.924588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:48:13.924588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:48:13.924588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:48:13.924588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:48:13.924588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:48:13.924588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:48:13.924588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:48:13.924588 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:48:13.924903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:48:13.924903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:48:13.924903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:48:13.924903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:48:13.924903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:48:13.924903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:13.924903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:48:13.924903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:48:13.924903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:48:13.924903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:13.924903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:48:13.924903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:48:13.924903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:48:13.924903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:48:13.924903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:48:13.924903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:48:13.924903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:48:13.924903 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:48:13.925239 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:48:13.925239 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:48:13.925239 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:48:13.925239 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:48:13.925239 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:48:13.925239 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:13.925239 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:48:13.925239 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:48:13.925239 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:48:13.925239 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:48:13.925709 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:48:13.925974 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:13.926622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:48:13.926622 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:48:13.926622 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:48:13.926622 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:48:13.926622 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:48:13.926622 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:48:13.926622 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:13.926622 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:48:13.927455 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:13.927455 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:48:13.927455 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:48:13.927455 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:48:13.927455 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:13.927455 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:48:13.927455 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:48:13.927455 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:48:13.927455 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:13.927455 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:48:13.927455 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:48:13.927455 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:48:13.927455 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:48:13.927455 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:48:13.927455 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:48:13.927739 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:48:13.927739 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:48:13.927739 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:48:13.927739 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:48:13.927739 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:48:13.927739 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:48:13.927739 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:48:13.927739 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:13.927739 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:48:13.928487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:13.928487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:48:13.928487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:48:13.928487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:48:13.928487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:48:13.928487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:48:13.928487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:48:13.928487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:48:13.928487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:48:13.928487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:13.928487 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:48:13.928726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:48:13.928726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:48:13.928726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:13.928726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:48:13.928726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:48:13.928726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:48:13.928726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:48:13.928726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:48:13.928726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:48:13.928726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:48:13.928726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:48:13.928726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:48:13.928726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:48:13.928726 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:48:13.928990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:48:13.928990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:48:13.928990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:13.928990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:48:13.929593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:13.929593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:48:13.929593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:48:13.929593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:48:13.929593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:48:13.929593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:48:13.929593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:48:13.929593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:48:13.929593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:48:13.929593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:48:13.929593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:48:13.929593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:48:13.929593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:48:13.929593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:48:13.929593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:48:13.929593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:48:13.929593 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:48:13.929977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:48:13.929977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:48:13.929977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:48:13.929977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:13.929977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:48:13.929977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:48:13.929977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:48:13.929977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:13.929977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:48:13.929977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:48:13.929977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:48:13.929977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:48:13.929977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:48:13.929977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:48:13.929977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:48:13.929977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:48:13.929977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:48:13.929977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:48:13.929977 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:48:13.930309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:48:13.930309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:48:13.930309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:13.930309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:48:13.930309 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:48:13.930309 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:48:13.930309 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:48:13.930451 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:48:13.930955 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:48:13.931180 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:13.931717 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:48:13.931717 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:48:13.931717 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:48:13.931717 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:48:13.931717 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:48:13.931717 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:48:13.931717 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:13.931717 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:48:13.932332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:13.932332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:48:13.932332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:48:13.932332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:48:13.932332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:48:13.932332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:48:13.932332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:48:13.932332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:13.932332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:48:13.932332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:48:13.932332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:48:13.932332 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:13.933062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:48:13.933062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:48:13.933062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:48:13.933062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:48:13.933062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:48:13.933062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:48:13.933062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:48:13.933062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:48:13.933062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:48:13.933062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:48:13.933062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:48:13.933062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:48:13.933062 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:48:13.933488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:13.933488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:48:13.933488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:13.933488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:48:13.933488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:48:13.933488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:48:13.933488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:48:13.933488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:48:13.933488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:48:13.933488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:48:13.933488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:48:13.933488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:48:13.933488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:48:13.933488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:48:13.933488 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:48:13.933930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:48:13.933930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:48:13.933930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:48:13.933930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:48:13.933930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:48:13.933930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:13.933930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:48:13.933930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:48:13.933930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:48:13.933930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:13.933930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:48:13.933930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:48:13.933930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:48:13.933930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:48:13.933930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:48:13.933930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:48:13.933930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:48:13.933930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:48:13.933930 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:48:13.934394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:48:13.934394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:48:13.934394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:48:13.934394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:48:13.934394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:13.934394 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:48:18.693726 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:48:18.694453 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:48:18.695166 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:48:18.695912 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:48:18.732663 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:48:18.733063 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:18.735723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:48:18.735723 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:48:18.735723 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:48:18.735723 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:48:18.735723 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:48:18.735723 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:48:18.735723 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:18.735723 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:48:18.736854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:18.736854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:48:18.736854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:48:18.736854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:48:18.736854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:48:18.736854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:48:18.736854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:48:18.736854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:18.736854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:48:18.736854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:48:18.736854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:48:18.736854 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:18.737279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:48:18.737279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:48:18.737279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:48:18.737279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:48:18.737279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:48:18.737279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:48:18.737279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:48:18.737279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:48:18.737279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:48:18.737279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:48:18.737279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:48:18.737279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:48:18.737279 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:48:18.737677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:18.737677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:48:18.737677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:18.737677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:48:18.737677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:48:18.737677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:48:18.737677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:48:18.737677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:48:18.737677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:48:18.737677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:48:18.737677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:48:18.737677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:48:18.737677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:48:18.737677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:48:18.737677 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:48:18.738196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:48:18.738196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:48:18.738196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:48:18.738196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:48:18.738196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:48:18.738196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:18.738196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:48:18.738196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:48:18.738196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:48:18.738196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:18.738196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:48:18.738196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:48:18.738196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:48:18.738196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:48:18.738196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:48:18.738196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:48:18.738196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:48:18.738196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:48:18.738196 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:48:18.738775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:48:18.738775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:48:18.738775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:48:18.738775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:48:18.738775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:18.738775 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:48:18.877041 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:48:18.877041 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:48:18.877041 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:48:18.878040 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:48:18.878927 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:48:18.879251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:18.881315 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:48:18.881315 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:48:18.881315 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:48:18.881315 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:48:18.881315 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:48:18.881315 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:48:18.881315 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:18.881315 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:48:18.882339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:18.882339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:48:18.882339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:48:18.882339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:48:18.882339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:48:18.882339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:48:18.882339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:48:18.882339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:18.882339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:48:18.882339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:48:18.882339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:48:18.882339 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:18.883046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:48:18.883046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:48:18.883046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:48:18.883046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:48:18.883046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:48:18.883046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:48:18.883046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:48:18.883046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:48:18.883046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:48:18.883046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:48:18.883046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:48:18.883046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:48:18.883046 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:48:18.883711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:18.883711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:48:18.883711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:18.883711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:48:18.883711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:48:18.883711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:48:18.883711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:48:18.883711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:48:18.883711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:48:18.883711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:48:18.883711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:48:18.883711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:48:18.883711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:48:18.883711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:48:18.883711 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:48:18.884573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:48:18.884573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:48:18.884573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:48:18.884573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:48:18.884573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:48:18.884573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:18.884573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:48:18.884573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:48:18.884573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:48:18.884573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:18.884573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:48:18.884573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:48:18.884573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:48:18.884573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:48:18.884573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:48:18.884573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:48:18.884573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:48:18.884573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:48:18.884573 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:48:18.885173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:48:18.885173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:48:18.885173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:48:18.885173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:48:18.885173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:18.885173 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:48:18.919924 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:48:18.920339 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:48:18.920593 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:48:18.920922 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:48:18.921537 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:48:18.921927 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:18.922563 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:48:18.922563 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:48:18.922563 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:48:18.922563 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:48:18.922563 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:48:18.922563 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:48:18.922563 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:18.922563 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:48:18.924311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:18.924311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:48:18.924311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:48:18.924311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:48:18.924311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:18.924311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:48:18.924311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:48:18.924311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:48:18.924311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:18.924311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:48:18.924311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:48:18.924311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:48:18.924311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:48:18.924311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:48:18.924311 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:48:18.925526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:48:18.925526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:48:18.925526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:48:18.925526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:48:18.925526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:48:18.925526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:48:18.925526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:48:18.925526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:18.925526 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:48:18.926406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:18.926406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:48:18.926406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:48:18.926406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:48:18.926406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:48:18.926406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:48:18.926406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:48:18.926406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:48:18.926406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:48:18.926406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:48:18.926406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:18.926406 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:48:18.927256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:48:18.927256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:48:18.927256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:18.927256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:48:18.927256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:48:18.927256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:48:18.927256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:48:18.927256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:48:18.927256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:48:18.927256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:48:18.927256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:48:18.927256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:48:18.927256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:48:18.927256 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:48:18.928102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:48:18.928102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:48:18.928102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:18.928102 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:48:18.928102 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:48:18.929024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:18.929024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:48:18.929024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:48:18.929024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:48:18.929024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:48:18.929024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:48:18.929024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:48:18.929024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:48:18.929024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:48:18.929024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:48:18.929024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:48:18.929024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:48:18.929024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:48:18.929024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:48:18.929024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:48:18.929024 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:48:18.929510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:48:18.929510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:48:18.929510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:48:18.929510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:48:18.929510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:48:18.929510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:18.929510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:48:18.929510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:48:18.929510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:48:18.929510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:18.929510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:48:18.929510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:48:18.929510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:48:18.929510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:48:18.929510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:48:18.929510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:48:18.929510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:48:18.929510 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:48:18.930006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:48:18.930006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:48:18.930006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:48:18.930006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:48:18.930006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:48:18.930006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:18.930006 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:48:18.930006 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:48:18.930006 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:48:18.930006 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:48:18.930212 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:48:18.930436 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:48:18.930852 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:18.931480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:48:18.931480 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:48:18.931480 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:48:18.931480 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:48:18.931480 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:48:18.931480 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:48:18.931480 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:18.931480 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:48:18.932206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:18.932206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:48:18.932206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:48:18.932206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:48:18.932206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:18.932206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:48:18.932206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:48:18.932206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:48:18.932206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:18.932206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:48:18.932206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:48:18.932206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:48:18.932206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:48:18.932206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:48:18.932206 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:48:18.932709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:48:18.932709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:48:18.932709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:48:18.932709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:48:18.932709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:48:18.932709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:48:18.932709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:48:18.932709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:18.932709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:48:18.933305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:18.933305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:48:18.933305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:48:18.933305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:48:18.933305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:48:18.933305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:48:18.933305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:48:18.933305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:48:18.933305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:48:18.933305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:18.933305 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:48:18.934504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:48:18.934504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:48:18.934504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:18.934504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:48:18.934504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:48:18.934504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:48:18.934504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:48:18.934504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:48:18.934504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:48:18.934504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:48:18.934504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:48:18.934504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:48:18.934504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:48:18.934504 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:48:18.935068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:48:18.935068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:48:18.935068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:18.935068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:48:18.935068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:18.935068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:48:18.935068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:48:18.935068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:48:18.935068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:48:18.935068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:48:18.935068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:48:18.935068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:48:18.935068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:48:18.935068 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:48:18.935391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:48:18.935391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:48:18.935391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:48:18.935391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:48:18.935391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:48:18.935391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:48:18.935391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:48:18.935391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:48:18.935391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:48:18.935391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:48:18.935391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:18.935391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:48:18.935391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:48:18.935391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:48:18.935391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:18.935391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:48:18.935391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:48:18.935391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:48:18.935391 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:48:18.935814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:48:18.935814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:48:18.935814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:48:18.935814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:48:18.935814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:48:18.935814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:48:18.935814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:48:18.935814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:48:18.935814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:48:18.935814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:18.935814 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:48:18.935814 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:48:18.935814 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:48:18.936121 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:48:18.936121 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:48:18.936121 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:48:18.936121 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:18.936618 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:48:18.936618 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:48:18.936618 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:48:18.936618 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:48:18.936618 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:48:18.936618 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:48:18.936618 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:18.936618 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:48:18.937306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:18.937306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:48:18.937306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:48:18.937306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:48:18.937306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:48:18.937306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:48:18.937306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:48:18.937306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:18.937306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:48:18.937306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:48:18.937306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:48:18.937306 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:18.937703 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:48:18.937703 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:48:18.937703 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:48:18.937703 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:48:18.937703 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:48:18.937703 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:48:18.937703 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:48:18.937703 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:48:18.937703 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:48:18.937703 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:48:18.937703 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:48:18.937703 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:48:18.937703 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:48:18.938122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:18.938122 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:48:18.938308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:18.938308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:48:18.938308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:48:18.938308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:48:18.938308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:48:18.938308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:48:18.938308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:48:18.938308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:48:18.938308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:48:18.938308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:48:18.938308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:48:18.938308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:48:18.938308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:48:18.938308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:48:18.938308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:48:18.938308 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:48:18.938648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:48:18.938648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:48:18.938648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:18.938648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:48:18.938648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:48:18.938648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:48:18.938648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:18.938648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:48:18.938648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:48:18.938648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:48:18.938648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:48:18.938648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:48:18.938648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:48:18.938648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:48:18.938648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:48:18.938648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:48:18.938648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:48:18.938648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:48:18.938648 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:48:18.939048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:48:18.939048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:18.939048 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:48:23.694893 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:48:23.696067 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:48:23.696067 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:48:23.697268 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:48:23.733740 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:48:23.734366 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:23.735261 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:48:23.735261 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:48:23.735261 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:48:23.735261 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:48:23.735261 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:48:23.735261 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:48:23.735261 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:23.735261 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:48:23.736589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:23.736589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:48:23.736589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:48:23.736589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:48:23.736589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:48:23.736589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:48:23.736589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:48:23.736589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:23.736589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:48:23.736589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:48:23.736589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:48:23.736589 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:23.737723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:48:23.737723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:48:23.737723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:48:23.737723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:48:23.737723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:48:23.737723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:48:23.737723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:48:23.737723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:48:23.737723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:48:23.737723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:48:23.737723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:48:23.737723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:48:23.737723 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:48:23.738360 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:23.738360 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:48:23.738360 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:23.738360 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:48:23.738360 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:48:23.738360 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:48:23.738360 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:48:23.738360 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:48:23.738360 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:48:23.738360 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:48:23.738360 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:48:23.738360 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:48:23.738360 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:48:23.738360 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:48:23.738360 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:48:23.739097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:48:23.739097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:48:23.739097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:48:23.739097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:48:23.739097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:48:23.739097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:23.739097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:48:23.739097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:48:23.739097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:48:23.739097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:23.739097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:48:23.739097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:48:23.739097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:48:23.739097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:48:23.739097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:48:23.739097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:48:23.739097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:48:23.739097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:48:23.739097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:48:23.741022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:48:23.741022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:48:23.741022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:48:23.741022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:48:23.741022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:23.741022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:48:23.877651 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:48:23.878519 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:48:23.878737 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:48:23.880148 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:48:23.880504 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:48:23.880799 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:23.881861 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:48:23.881861 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:48:23.881861 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:48:23.881861 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:48:23.881861 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:48:23.881861 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:48:23.881861 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:23.881861 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:48:23.883776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:23.883776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:48:23.883776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:48:23.883776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:48:23.883776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:48:23.883776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:48:23.883776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:48:23.883776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:23.883776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:48:23.883776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:48:23.883776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:48:23.883776 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:23.884560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:48:23.884560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:48:23.884560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:48:23.884560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:48:23.884560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:48:23.884560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:48:23.884560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:48:23.884560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:48:23.884560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:48:23.884560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:48:23.884560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:48:23.884560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:48:23.884560 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:48:23.885186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:23.885186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:48:23.885665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:23.885665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:48:23.885665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:48:23.885665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:48:23.885665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:48:23.885665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:48:23.885665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:48:23.885665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:48:23.885665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:48:23.885665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:48:23.885665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:48:23.885665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:48:23.885665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:48:23.885665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:48:23.885665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:48:23.885665 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:48:23.886434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:48:23.886434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:48:23.886434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:23.886434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:48:23.886434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:48:23.886434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:48:23.886434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:23.886434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:48:23.886434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:48:23.886434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:48:23.886434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:48:23.886434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:48:23.886434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:48:23.886434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:48:23.886434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:48:23.886434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:48:23.886434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:48:23.886434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:48:23.886434 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:48:23.887251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:48:23.887251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:23.887251 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:48:23.920739 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:48:23.921250 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:48:23.921439 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:48:23.921647 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:48:23.922251 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:48:23.922614 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:23.923463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:48:23.923463 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:48:23.923463 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:48:23.923463 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:48:23.923463 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:48:23.923463 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:48:23.923463 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:23.923463 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:48:23.925051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:23.925051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:48:23.925051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:48:23.925051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:48:23.925051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:23.925051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:48:23.925051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:48:23.925051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:48:23.925051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:23.925051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:48:23.925051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:48:23.925051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:48:23.925051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:48:23.925051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:48:23.925051 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:48:23.925871 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:48:23.925871 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:48:23.925871 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:48:23.925871 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:48:23.925871 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:48:23.925871 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:48:23.925871 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:48:23.925871 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:23.925871 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:48:23.926639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:23.926639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:48:23.926639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:48:23.926639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:48:23.926639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:48:23.926639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:48:23.926639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:48:23.926639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:48:23.926639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:48:23.926639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:48:23.926639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:23.926639 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:48:23.927027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:48:23.927027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:48:23.927027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:23.927027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:48:23.927027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:48:23.927027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:48:23.927027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:48:23.927027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:48:23.927027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:48:23.927027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:48:23.927027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:48:23.927027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:48:23.927027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:48:23.927027 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:48:23.927374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:48:23.927374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:48:23.927374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:23.927374 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:48:23.928896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:23.928896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:48:23.928896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:48:23.928896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:48:23.928896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:48:23.928896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:48:23.928896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:48:23.928896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:48:23.928896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:48:23.928896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:48:23.928896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:48:23.928896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:48:23.928896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:48:23.928896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:48:23.928896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:48:23.928896 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:48:23.929433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:48:23.929433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:48:23.929433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:48:23.929433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:48:23.929433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:48:23.929433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:23.929433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:48:23.929433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:48:23.929433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:48:23.929433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:23.929433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:48:23.929433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:48:23.929433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:48:23.929433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:48:23.929433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:48:23.929433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:48:23.929433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:48:23.929433 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:48:23.929938 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:48:23.929938 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:48:23.929938 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:48:23.929938 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:48:23.929938 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:48:23.929938 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:23.929938 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:48:23.929938 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:48:23.929938 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:48:23.929938 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:48:23.930227 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:48:23.930833 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:48:23.930936 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:23.931705 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:48:23.931705 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:48:23.931705 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:48:23.931705 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:48:23.931705 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:48:23.931705 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:48:23.931705 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:23.931705 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:48:23.932684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:23.932684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:48:23.932684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:48:23.932684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:48:23.932684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:23.932684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:48:23.932684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:48:23.932684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:48:23.932684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:23.932684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:48:23.932684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:48:23.932684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:48:23.932684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:48:23.932684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:48:23.932684 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:48:23.933623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:48:23.933623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:48:23.933623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:48:23.933623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:48:23.933623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:48:23.933623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:48:23.933623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:48:23.933623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:23.933623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:48:23.933623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:23.933623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:48:23.933623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:48:23.933623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:48:23.933623 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:48:23.934533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:48:23.934533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:48:23.934533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:48:23.934533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:48:23.934533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:23.934533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:48:23.934533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:48:23.934533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:48:23.934533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:23.934533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:48:23.934533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:48:23.934533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:48:23.934533 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:48:23.935049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:48:23.935049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:48:23.935049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:48:23.935049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:48:23.935049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:48:23.935049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:48:23.935049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:48:23.935049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:48:23.935049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:48:23.935049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:23.935049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:48:23.935049 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:23.935393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:48:23.935393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:48:23.935393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:48:23.935393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:48:23.935393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:48:23.935393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:48:23.935393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:48:23.935393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:48:23.935393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:48:23.935393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:48:23.935393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:48:23.935393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:48:23.935393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:48:23.935393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:48:23.935393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:48:23.935393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:48:23.935393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:48:23.935393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:48:23.935393 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:48:23.935952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:23.935952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:48:23.935952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:48:23.935952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:48:23.935952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:23.935952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:48:23.935952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:48:23.935952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:48:23.935952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:48:23.935952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:48:23.935952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:48:23.935952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:48:23.935952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:48:23.935952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:48:23.935952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:48:23.935952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:48:23.935952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:48:23.935952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:48:23.935952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:23.935952 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:48:23.937594 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:48:23.937594 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:48:23.937594 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:48:23.937594 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:48:23.937594 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:48:23.937594 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:48:23.938177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:23.938177 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:48:23.938177 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:48:23.938177 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:48:23.938177 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:48:23.938177 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:48:23.938177 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:48:23.938177 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:23.938177 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:48:23.938875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:23.938875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:48:23.938875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:48:23.938875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:48:23.938875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:48:23.938875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:48:23.938875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:48:23.938875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:23.938875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:48:23.938875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:48:23.938875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:48:23.938875 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:23.939815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:48:23.939815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:48:23.939815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:48:23.939815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:48:23.939815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:48:23.939815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:48:23.939815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:48:23.939815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:48:23.939815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:48:23.939815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:48:23.939815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:48:23.939815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:48:23.939815 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:48:23.940313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:23.940313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:48:23.940313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:23.940313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:48:23.940313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:48:23.940313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:48:23.940313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:48:23.940313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:48:23.940313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:48:23.940313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:48:23.940313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:48:23.940313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:48:23.940313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:48:23.940313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:48:23.940313 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:48:23.941172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:48:23.941172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:48:23.941172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:48:23.941172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:48:23.941172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:48:23.941172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:23.941172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:48:23.941172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:48:23.941172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:48:23.941172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:23.941172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:48:23.941172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:48:23.941172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:48:23.941172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:48:23.941172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:48:23.941172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:48:23.941172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:48:23.941172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:48:23.941172 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:48:23.942606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:48:23.942606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:48:23.942606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:48:23.942606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:48:23.942606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:23.942606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:48:28.695689 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:48:28.696535 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:48:28.696535 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:48:28.697769 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:48:28.736631 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:48:28.737080 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:28.737980 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:48:28.737980 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:48:28.737980 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:48:28.737980 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:48:28.737980 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:48:28.737980 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:48:28.737980 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:28.737980 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:48:28.738841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:28.738841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:48:28.738841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:48:28.738841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:48:28.738841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:48:28.738841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:48:28.738841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:48:28.738841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:28.738841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:48:28.738841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:48:28.738841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:48:28.738841 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:28.739283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:48:28.739283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:48:28.739283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:48:28.739283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:48:28.739283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:48:28.739283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:48:28.739283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:48:28.739283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:48:28.739283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:48:28.739283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:48:28.739283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:48:28.739283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:48:28.739283 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:48:28.740372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:28.740372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:48:28.740372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:28.740372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:48:28.740372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:48:28.740372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:48:28.740372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:48:28.740372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:48:28.740372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:48:28.740372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:48:28.740372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:48:28.740372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:48:28.740372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:48:28.740372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:48:28.740372 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:48:28.740880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:48:28.740880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:48:28.740880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:48:28.740880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:48:28.740880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:48:28.740880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:28.740880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:48:28.740880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:48:28.740880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:48:28.740880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:28.740880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:48:28.740880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:48:28.740880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:48:28.740880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:48:28.740880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:48:28.740880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:48:28.740880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:48:28.740880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:48:28.740880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:48:28.741429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:48:28.741429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:48:28.741429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:48:28.741429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:48:28.741429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:28.741429 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:48:28.878712 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:48:28.879941 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:48:28.879941 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:48:28.881047 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:48:28.881342 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:48:28.881990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:28.882345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:48:28.882345 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:48:28.882345 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:48:28.882345 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:48:28.882345 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:48:28.882345 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:48:28.882345 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:28.882345 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:48:28.883548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:28.883548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:48:28.883548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:48:28.883548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:48:28.883548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:48:28.883548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:48:28.883548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:48:28.883548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:28.883548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:48:28.883548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:48:28.883548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:48:28.883548 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:28.884011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:48:28.884011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:48:28.884011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:48:28.884011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:48:28.884011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:48:28.884011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:48:28.884011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:48:28.884011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:48:28.884011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:48:28.884011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:48:28.884011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:48:28.884011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:48:28.884011 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:48:28.884923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:28.884923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:48:28.884923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:28.884923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:48:28.884923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:48:28.884923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:48:28.884923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:48:28.884923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:48:28.884923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:48:28.884923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:48:28.884923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:48:28.884923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:48:28.884923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:48:28.884923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:48:28.884923 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:48:28.885371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:48:28.885371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:48:28.885371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:48:28.885371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:48:28.885371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:48:28.885371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:28.885371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:48:28.885371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:48:28.885371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:48:28.885371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:28.885371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:48:28.885371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:48:28.885371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:48:28.885371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:48:28.885371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:48:28.885371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:48:28.885371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:48:28.885371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:48:28.885371 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:48:28.885984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:48:28.885984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:48:28.885984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:48:28.885984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:48:28.885984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:28.885984 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:48:28.921325 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:48:28.921931 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:48:28.922057 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:48:28.922199 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:48:28.923025 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:48:28.923361 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:28.924329 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:48:28.924329 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:48:28.924329 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:48:28.924329 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:48:28.924329 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:48:28.924329 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:48:28.924329 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:28.924329 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:48:28.925480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:28.925480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:48:28.925480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:48:28.925480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:48:28.925480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:28.925480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:48:28.925480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:48:28.925480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:48:28.925480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:28.925480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:48:28.925480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:48:28.925480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:48:28.925480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:48:28.925480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:48:28.925480 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:48:28.926016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:48:28.926016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:48:28.926016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:48:28.926016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:48:28.926016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:48:28.926016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:48:28.926016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:48:28.926016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:28.926016 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:48:28.926710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:28.926710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:48:28.926710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:48:28.926710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:48:28.926710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:48:28.926710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:48:28.926710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:48:28.926710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:48:28.926710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:48:28.926710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:48:28.926710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:28.926710 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:48:28.927127 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:48:28.927127 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:48:28.927127 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:28.927127 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:48:28.927127 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:48:28.927127 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:48:28.927127 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:48:28.927127 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:48:28.927127 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:48:28.927127 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:48:28.927127 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:48:28.927127 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:48:28.927127 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:48:28.927127 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:48:28.927520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:48:28.927520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:48:28.927520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:28.927520 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:48:28.929970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:28.929970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:48:28.929970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:48:28.929970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:48:28.929970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:48:28.929970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:48:28.929970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:48:28.929970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:48:28.929970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:48:28.929970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:48:28.929970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:48:28.929970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:48:28.929970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:48:28.929970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:48:28.929970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:48:28.929970 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:48:28.930557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:48:28.930557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:48:28.930557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:48:28.930557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:48:28.930557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:48:28.930557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:28.930557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:48:28.930557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:48:28.930557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:48:28.930557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:28.930557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:48:28.930557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:48:28.930557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:48:28.930557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:48:28.930557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:48:28.930557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:48:28.930557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:48:28.930557 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:48:28.931159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:48:28.931159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:48:28.931159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:48:28.931159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:48:28.931159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:48:28.931159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:28.931159 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:48:28.931159 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:48:28.931159 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:48:28.931159 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:48:28.931477 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:48:28.931477 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:48:28.931640 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:28.932492 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:48:28.932492 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:48:28.932492 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:48:28.932492 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:48:28.932492 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:48:28.932492 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:48:28.932492 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:28.932492 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:48:28.933426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:28.933426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:48:28.933426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:48:28.933426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:48:28.933426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:28.933426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:48:28.933426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:48:28.933426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:48:28.933426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:28.933426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:48:28.933426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:48:28.933426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:48:28.933426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:48:28.933426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:48:28.933426 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:48:28.933880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:48:28.933880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:48:28.933880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:48:28.933880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:48:28.933880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:48:28.933880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:48:28.933880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:48:28.933880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:28.933880 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:48:28.935001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:28.935001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:48:28.935001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:48:28.935001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:48:28.935001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:48:28.935001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:48:28.935001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:48:28.935001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:48:28.935001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:48:28.935001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:28.935001 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:48:28.935377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:48:28.935377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:48:28.935377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:28.935377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:48:28.935377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:48:28.935377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:48:28.935377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:48:28.935377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:48:28.935377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:48:28.935377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:48:28.935377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:48:28.935377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:48:28.935377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:48:28.935377 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:48:28.935787 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:48:28.935787 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:48:28.935787 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:28.935787 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:48:28.935787 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:28.935787 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:48:28.935787 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:48:28.935787 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:48:28.935787 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:48:28.935787 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:48:28.935787 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:48:28.935787 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:48:28.935787 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:48:28.935787 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:48:28.936211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:48:28.936211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:48:28.936211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:48:28.936211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:48:28.936211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:48:28.936211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:48:28.936211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:48:28.936211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:48:28.936211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:48:28.936211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:48:28.936211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:28.936211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:48:28.936211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:48:28.936211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:48:28.936211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:28.936211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:48:28.936211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:48:28.936211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:48:28.936211 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:48:28.936779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:48:28.936779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:48:28.936779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:48:28.936779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:48:28.936779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:48:28.936779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:48:28.936779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:48:28.936779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:48:28.936779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:48:28.936779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:28.936779 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:48:28.936779 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:48:28.936779 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:48:28.937269 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:48:28.937269 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:48:28.937269 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:48:28.939553 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:48:28.939553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:28.939553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:48:28.939553 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:48:28.939553 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:48:28.939553 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:48:28.939553 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:48:28.939553 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:48:28.939553 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:28.939553 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:48:28.939553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:28.939553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:48:28.939553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:48:28.939553 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:48:28.939884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:48:28.939884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:48:28.939884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:48:28.939884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:28.939884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:48:28.939884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:48:28.939884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:48:28.939884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:28.939884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:48:28.939884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:48:28.939884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:48:28.939884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:48:28.939884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:48:28.939884 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:48:28.940166 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:48:28.940166 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:48:28.940166 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:48:28.940166 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:48:28.940166 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:48:28.940166 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:48:28.940166 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:48:28.940166 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:28.940166 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:48:28.940844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:28.940844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:48:28.940844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:48:28.940844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:48:28.940844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:48:28.940844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:48:28.940844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:48:28.940844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:48:28.940844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:48:28.940844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:48:28.940844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:48:28.940844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:48:28.940844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:48:28.940844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:48:28.940844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:48:28.940844 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:48:28.941341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:48:28.941341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:48:28.941341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:28.941341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:48:28.941341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:48:28.941341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:48:28.941341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:28.941341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:48:28.941341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:48:28.941341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:48:28.941341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:48:28.941341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:48:28.941341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:48:28.941341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:48:28.941341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:48:28.941341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:48:28.941341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:48:28.941341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:48:28.941341 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:48:28.941738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:48:28.941738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:28.941738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:48:33.696862 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:48:33.697860 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:48:33.697860 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:48:33.698371 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:48:33.731523 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:48:33.731738 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:33.732360 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:48:33.732360 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:48:33.732360 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:48:33.732360 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:48:33.732360 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:48:33.732360 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:48:33.732360 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:33.732360 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:48:33.733052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:33.733052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:48:33.733052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:48:33.733052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:48:33.733052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:48:33.733052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:48:33.733052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:48:33.733052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:33.733052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:48:33.733052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:48:33.733052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:48:33.733052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:33.734097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:48:33.734097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:48:33.734097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:48:33.734097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:48:33.734097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:48:33.734097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:48:33.734097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:48:33.734097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:48:33.734097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:48:33.734097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:48:33.734097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:48:33.734097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:48:33.734097 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:48:33.734696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:33.734696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:48:33.734696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:33.734696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:48:33.734696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:48:33.734696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:48:33.734696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:48:33.734696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:48:33.734696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:48:33.734696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:48:33.734696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:48:33.734696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:48:33.734696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:48:33.734696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:48:33.734696 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:48:33.735449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:48:33.735449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:48:33.735449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:48:33.735449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:48:33.735449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:48:33.735449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:33.735449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:48:33.735449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:48:33.735449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:48:33.735449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:33.735449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:48:33.735449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:48:33.735449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:48:33.735449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:48:33.735449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:48:33.735449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:48:33.735449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:48:33.735449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:48:33.735449 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:48:33.736345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:48:33.736345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:48:33.736345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:48:33.736345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:48:33.736345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:33.736345 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:48:33.879303 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:48:33.879955 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:48:33.880147 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:48:33.881031 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:48:33.881250 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:48:33.881432 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:33.882216 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:48:33.882216 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:48:33.882216 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:48:33.882216 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:48:33.882216 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:48:33.882216 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:48:33.882216 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:33.882216 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:48:33.883709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:33.883709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:48:33.883709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:48:33.883709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:48:33.883709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:48:33.883709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:48:33.883709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:48:33.883709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:33.883709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:48:33.883709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:48:33.883709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:48:33.883709 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:33.884143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:48:33.884143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:48:33.884143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:48:33.884143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:48:33.884143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:48:33.884143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:48:33.884143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:48:33.884143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:48:33.884143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:48:33.884143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:48:33.884143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:48:33.884143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:48:33.884143 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:48:33.884485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:33.884485 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:48:33.885467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:33.885467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:48:33.885467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:48:33.885467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:48:33.885467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:48:33.885467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:48:33.885467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:48:33.885467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:48:33.885467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:48:33.885467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:48:33.885467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:48:33.885467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:48:33.885467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:48:33.885467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:48:33.885467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:48:33.885467 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:48:33.885958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:48:33.885958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:48:33.885958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:33.885958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:48:33.885958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:48:33.885958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:48:33.885958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:33.885958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:48:33.885958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:48:33.885958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:48:33.885958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:48:33.885958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:48:33.885958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:48:33.885958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:48:33.885958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:48:33.885958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:48:33.885958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:48:33.885958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:48:33.885958 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:48:33.886507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:48:33.886507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:33.886507 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:48:33.921969 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:48:33.922431 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:48:33.922671 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:48:33.922984 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:48:33.923339 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:48:33.923535 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:33.924109 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:48:33.924109 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:48:33.924109 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:48:33.924109 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:48:33.924109 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:48:33.924109 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:48:33.924109 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:33.924109 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:48:33.925350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:33.925350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:48:33.925350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:48:33.925350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:48:33.925350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:33.925350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:48:33.925350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:48:33.925350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:48:33.925350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:33.925350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:48:33.925350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:48:33.925350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:48:33.925350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:48:33.925350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:48:33.925350 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:48:33.925790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:48:33.925790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:48:33.925790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:48:33.925790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:48:33.925790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:48:33.925790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:48:33.925790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:48:33.925790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:33.925790 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:48:33.927110 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:33.927110 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:48:33.927110 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:48:33.927110 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:48:33.927110 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:48:33.927110 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:48:33.927110 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:48:33.927110 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:48:33.927110 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:48:33.927110 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:48:33.927110 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:33.927110 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:48:33.927446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:48:33.927446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:48:33.927446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:33.927446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:48:33.927446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:48:33.927446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:48:33.927446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:48:33.927446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:48:33.927446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:48:33.927446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:48:33.927446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:48:33.927446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:48:33.927446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:48:33.927446 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:48:33.927847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:48:33.927847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:48:33.927847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:33.927847 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:48:33.928538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:33.928538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:48:33.928538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:48:33.928538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:48:33.928538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:48:33.928538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:48:33.928538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:48:33.928538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:48:33.928538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:48:33.928538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:48:33.928538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:48:33.928538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:48:33.928538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:48:33.928538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:48:33.928538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:48:33.928538 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:48:33.929101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:48:33.929101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:48:33.929101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:48:33.929101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:48:33.929101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:48:33.929101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:33.929101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:48:33.929101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:48:33.929101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:48:33.929101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:33.929101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:48:33.929101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:48:33.929101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:48:33.929101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:48:33.929101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:48:33.929101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:48:33.929101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:48:33.929101 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:48:33.929590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:48:33.929590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:48:33.929590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:48:33.929590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:48:33.929590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:48:33.929590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:33.929590 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:48:33.930619 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:48:33.930984 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:48:33.931116 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:48:33.931246 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:48:33.931566 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:48:33.931800 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:33.932269 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:48:33.932269 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:48:33.932269 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:48:33.932269 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:48:33.932269 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:48:33.932269 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:48:33.932269 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:33.932269 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:48:33.933028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:33.933028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:48:33.933028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:48:33.933028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:48:33.933028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:33.933028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:48:33.933028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:48:33.933028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:48:33.933028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:33.933028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:48:33.933028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:48:33.933028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:48:33.933028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:48:33.933028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:48:33.933028 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:48:33.933838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:48:33.933838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:48:33.933838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:48:33.933838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:48:33.933838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:48:33.933838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:48:33.933838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:48:33.933838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:33.933838 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:48:33.934295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:33.934295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:48:33.934295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:48:33.934295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:48:33.934295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:48:33.934295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:48:33.934295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:48:33.934295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:48:33.934295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:48:33.934295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:33.934295 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:48:33.934907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:48:33.934907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:48:33.934907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:33.934907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:48:33.934907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:48:33.934907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:48:33.934907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:48:33.934907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:48:33.934907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:48:33.934907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:48:33.934907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:48:33.934907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:48:33.934907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:48:33.934907 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:48:33.935594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:48:33.935594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:48:33.935594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:33.935594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:48:33.935594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:33.935594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:48:33.935594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:48:33.935594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:48:33.935594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:48:33.935594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:48:33.935594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:48:33.935594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:48:33.935594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:48:33.935594 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:48:33.936276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:48:33.936276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:48:33.936276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:48:33.936276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:48:33.936276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:48:33.936276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:48:33.936276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:48:33.936276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:48:33.936276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:48:33.936276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:48:33.936276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:33.936276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:48:33.936276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:48:33.936276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:48:33.936276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:33.936276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:48:33.936276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:48:33.936276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:48:33.936276 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:48:33.939962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:48:33.939962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:48:33.939962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:48:33.939962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:48:33.939962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:48:33.939962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:48:33.939962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:48:33.939962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:48:33.939962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:48:33.939962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:33.939962 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:48:33.939962 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:48:33.939962 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:48:33.940235 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:48:33.940235 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:48:33.940235 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:48:33.940235 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:48:33.940235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:33.940235 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:48:33.940235 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:48:33.940235 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:48:33.940235 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:48:33.940424 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:48:33.940424 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:48:33.940424 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:33.940424 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:48:33.940424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:33.940424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:48:33.940424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:48:33.940424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:48:33.940424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:48:33.940424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:48:33.940424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:48:33.940424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:33.940424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:48:33.940424 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:48:33.940713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:48:33.940713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:33.940713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:48:33.940713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:48:33.940713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:48:33.940713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:48:33.940713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:48:33.940713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:48:33.940713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:48:33.940713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:48:33.940713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:48:33.940713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:48:33.940713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:48:33.940713 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:48:33.940990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:48:33.940990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:33.940990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:48:33.940990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:33.940990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:48:33.940990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:48:33.940990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:48:33.940990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:48:33.940990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:48:33.940990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:48:33.940990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:48:33.940990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:48:33.940990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:48:33.940990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:48:33.940990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:48:33.940990 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:48:33.941309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:48:33.941309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:48:33.941309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:48:33.941309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:48:33.941309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:48:33.941309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:33.941309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:48:33.941309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:48:33.941309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:48:33.941309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:33.941309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:48:33.941309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:48:33.941309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:48:33.941309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:48:33.941309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:48:33.941309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:48:33.941309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:48:33.941309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:48:33.941309 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:48:33.941660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:48:33.941660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:48:33.941660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:48:33.941660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:48:33.941660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:33.941660 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:48:38.697542 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:48:38.698463 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:48:38.698463 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:48:38.699415 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:48:38.743207 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:48:38.743622 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:38.744428 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:48:38.744428 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:48:38.744428 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:48:38.744428 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:48:38.744428 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:48:38.744428 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:48:38.744428 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:38.744428 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:48:38.745567 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_fdb_aging_settings failed after 0.046 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:38.745567 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:48:38.745567 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:48:38.745567 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:48:38.745567 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:48:38.745567 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:48:38.745567 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:48:38.745567 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:38.745567 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:48:38.745567 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:48:38.745567 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:48:38.745567 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:38.747052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:48:38.747052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:48:38.747052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:48:38.747052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:48:38.747052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:48:38.747052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:48:38.747052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:48:38.747052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:48:38.747052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:48:38.747052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:48:38.747052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:48:38.747052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:48:38.747052 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:48:38.747909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:38.747909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:48:38.747909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:38.747909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:48:38.747909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:48:38.747909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:48:38.747909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:48:38.747909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:48:38.747909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:48:38.747909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:48:38.747909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:48:38.747909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:48:38.747909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:48:38.747909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:48:38.747909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:48:38.748719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:48:38.748719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:48:38.748719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:48:38.748719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
May 09 09:48:38.748719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:48:38.748719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:38.748719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:48:38.748719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:48:38.748719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:48:38.748719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:38.748719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:48:38.748719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:48:38.748719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:48:38.748719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:48:38.748719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:48:38.748719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:48:38.748719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:48:38.748719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:48:38.748719 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:48:38.750365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:48:38.750365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:48:38.750365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:48:38.750365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:48:38.750365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:38.750365 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:48:38.880129 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:48:38.880780 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:48:38.881011 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:48:38.882009 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-a766a392-b08a-4e6f-abb6-a919e8e71db1, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:48:38.882209 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-cab10384-14c5-492c-8bd0-061f5838310c, routes=[], if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:48:38.882357 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:38.882948 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:48:38.882948 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:48:38.882948 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:48:38.882948 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:48:38.882948 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:48:38.882948 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:48:38.882948 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:38.882948 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:48:38.884730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:38.884730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:48:38.884730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:48:38.884730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:48:38.884730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:48:38.884730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:48:38.884730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:48:38.884730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:38.884730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:48:38.884730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:48:38.884730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:48:38.884730 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:38.885680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:48:38.885680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:48:38.885680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:48:38.885680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:48:38.885680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:48:38.885680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:48:38.885680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:48:38.885680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:48:38.885680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:48:38.885680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:48:38.885680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:48:38.885680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:48:38.885680 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:48:38.886550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:38.886550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:48:38.886550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:38.886550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:48:38.886550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:48:38.886550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:48:38.886550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:48:38.886550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:48:38.886550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:48:38.886550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:48:38.886550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:48:38.886550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:48:38.886550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:48:38.886550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:48:38.886550 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:48:38.887462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:48:38.887462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:48:38.887462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:48:38.887462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
May 09 09:48:38.887462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:48:38.887462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:38.887462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:48:38.887462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:48:38.887462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:48:38.887462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:38.887462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:48:38.887462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:48:38.887462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:48:38.887462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:48:38.887462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:48:38.887462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:48:38.887462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:48:38.887462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:48:38.887462 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:48:38.888127 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:48:38.888127 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:48:38.888127 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:48:38.888127 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:48:38.888127 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:38.888127 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:48:38.922538 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:48:38.923407 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:48:38.923407 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:48:38.923814 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106314) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
May 09 09:48:38.924022 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:48:38.924022 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:38.924502 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:48:38.924502 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:48:38.924502 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:48:38.924502 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:48:38.924502 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:48:38.924502 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:48:38.924502 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:38.924502 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:48:38.925895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:38.925895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:48:38.925895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:48:38.925895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:48:38.925895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:38.925895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:48:38.925895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:48:38.925895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:48:38.925895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:38.925895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:48:38.925895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:48:38.925895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:48:38.925895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:48:38.925895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:48:38.925895 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:48:38.926501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:48:38.926501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:48:38.926501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:48:38.926501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:48:38.926501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:48:38.926501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:48:38.926501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:48:38.926501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:38.926501 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:48:38.927695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:38.927695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:48:38.927695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:48:38.927695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:48:38.927695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:48:38.927695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:48:38.927695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:48:38.927695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
May 09 09:48:38.927695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:48:38.927695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:48:38.927695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:38.927695 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:48:38.928587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:48:38.928587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:48:38.928587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:38.928587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:48:38.928587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:48:38.928587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:48:38.928587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:48:38.928587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:48:38.928587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:48:38.928587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:48:38.928587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:48:38.928587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:48:38.928587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:48:38.928587 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:48:38.929579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:48:38.929579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:48:38.929579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:38.929579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:48:38.929579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:38.929579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:48:38.929579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:48:38.929579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:48:38.929579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:48:38.929579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:48:38.929579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:48:38.929579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:48:38.929579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:48:38.929579 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:48:38.930780 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:48:38.930780 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:48:38.930780 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:48:38.930780 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:48:38.930780 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:48:38.930780 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:48:38.930780 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
May 09 09:48:38.930780 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
May 09 09:48:38.930780 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                                                           ^^^^^^^^
May 09 09:48:38.930780 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:48:38.930780 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:48:38.930780 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:38.930780 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:48:38.930780 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:48:38.930780 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:48:38.930780 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:38.930780 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:48:38.930780 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:48:38.930780 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:48:38.932060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:48:38.932060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:48:38.932060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:48:38.932060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:48:38.932060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:48:38.932060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:48:38.932060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:48:38.932060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:48:38.932060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:48:38.932060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:48:38.932060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:38.932060 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:48:38.932060 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:48:38.932060 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:48:38.932556 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:48:38.932556 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:48:38.932556 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:48:38.932556 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:38.933100 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:48:38.933100 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:48:38.933100 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:48:38.933100 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:48:38.933100 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:48:38.933100 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:48:38.933100 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:38.933100 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:48:38.933919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:38.933919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
May 09 09:48:38.933919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:48:38.933919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:48:38.933919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:38.933919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:48:38.933919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:48:38.933919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
May 09 09:48:38.933919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:38.933919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
May 09 09:48:38.933919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:48:38.933919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
May 09 09:48:38.933919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:48:38.933919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
May 09 09:48:38.933919 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
May 09 09:48:38.934944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:48:38.934944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
May 09 09:48:38.934944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:48:38.934944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
May 09 09:48:38.934944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
May 09 09:48:38.934944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:48:38.934944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
May 09 09:48:38.934944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:38.934944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.command 
May 09 09:48:38.934944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:38.934944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:48:38.934944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:48:38.934944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:48:38.934944 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:48:38.935463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:48:38.935463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:48:38.935463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:48:38.935463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:48:38.935463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:38.935463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:48:38.935463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:48:38.935463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:48:38.935463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:38.935463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:48:38.935463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:48:38.935463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:48:38.935463 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:48:38.935909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:48:38.935909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:48:38.935909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:48:38.935909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:48:38.935909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:48:38.935909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:48:38.935909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:48:38.935909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:48:38.935909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:48:38.935909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:38.935909 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:48:38.938442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:38.938442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:48:38.938442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:48:38.938442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:48:38.938442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:48:38.938442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:48:38.938442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:48:38.938442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:48:38.938442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:48:38.938442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:48:38.938442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:48:38.938442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:48:38.938442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:48:38.938442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:48:38.938442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:48:38.938442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:48:38.938442 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
May 09 09:48:38.939606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
May 09 09:48:38.939606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
May 09 09:48:38.939606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
May 09 09:48:38.939606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:38.939606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:48:38.939606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:48:38.939606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:48:38.939606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:38.939606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:48:38.939606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:48:38.939606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:48:38.939606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:48:38.939606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:48:38.939606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:48:38.939606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:48:38.939606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:48:38.939606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:48:38.939606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:48:38.939606 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:48:38.940461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:48:38.940461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:48:38.940461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:38.940461 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics 
May 09 09:48:38.940461 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106314) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
May 09 09:48:38.940461 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
May 09 09:48:38.940461 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106314) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
May 09 09:48:38.940461 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106314) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:951}}
May 09 09:48:38.940798 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:48:38.940798 np0000173033 neutron-ovn-maintenance-worker[106314]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106314) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
May 09 09:48:38.940798 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:38.941141 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Traceback (most recent call last):
May 09 09:48:38.941141 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:48:38.941141 np0000173033 neutron-ovn-maintenance-worker[106314]:     txn.results.put(txn.do_commit())
May 09 09:48:38.941141 np0000173033 neutron-ovn-maintenance-worker[106314]:                     ^^^^^^^^^^^^^^^
May 09 09:48:38.941141 np0000173033 neutron-ovn-maintenance-worker[106314]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:48:38.941141 np0000173033 neutron-ovn-maintenance-worker[106314]:     raise RuntimeError(msg)
May 09 09:48:38.941141 np0000173033 neutron-ovn-maintenance-worker[106314]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:38.941141 np0000173033 neutron-ovn-maintenance-worker[106314]: 
May 09 09:48:38.941706 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:38.941706 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
May 09 09:48:38.941706 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:48:38.941706 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
May 09 09:48:38.941706 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
May 09 09:48:38.941706 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:48:38.941706 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:48:38.941706 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:38.941706 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:48:38.941706 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:48:38.941706 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
May 09 09:48:38.941706 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:38.942879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
May 09 09:48:38.942879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:48:38.942879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
May 09 09:48:38.942879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:48:38.942879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
May 09 09:48:38.942879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
May 09 09:48:38.942879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:48:38.942879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
May 09 09:48:38.942879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:48:38.942879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
May 09 09:48:38.942879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
May 09 09:48:38.942879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:48:38.942879 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
May 09 09:48:38.943364 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:38.943364 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
May 09 09:48:38.943364 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics [None req-49bed715-58f7-46d6-a09c-62492341e649 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:38.943364 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics Traceback (most recent call last):
May 09 09:48:38.943364 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
May 09 09:48:38.943364 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     work()
May 09 09:48:38.943364 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
May 09 09:48:38.943364 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
May 09 09:48:38.943364 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:48:38.943364 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
May 09 09:48:38.943364 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(*args, **kwargs)
May 09 09:48:38.943364 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
May 09 09:48:38.943364 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
May 09 09:48:38.943364 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     return f(self, *args, **kwargs)
May 09 09:48:38.943364 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
May 09 09:48:38.943901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
May 09 09:48:38.943901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     result = f(*args, **kwargs)
May 09 09:48:38.943901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
May 09 09:48:38.943901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
May 09 09:48:38.943901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
May 09 09:48:38.943901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:38.943901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:48:38.943901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 272, in transaction
May 09 09:48:38.943901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
May 09 09:48:38.943901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
May 09 09:48:38.943901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     next(self.gen)
May 09 09:48:38.943901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
May 09 09:48:38.943901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     with self.create_transaction(
May 09 09:48:38.943901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
May 09 09:48:38.943901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     self.result = self.commit()
May 09 09:48:38.943901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
May 09 09:48:38.943901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
May 09 09:48:38.943901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise result.ex
May 09 09:48:38.943901 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
May 09 09:48:38.945186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
May 09 09:48:38.945186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
May 09 09:48:38.945186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
May 09 09:48:38.945186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics     raise RuntimeError(msg)
May 09 09:48:38.945186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
May 09 09:48:38.945186 np0000173033 neutron-ovn-maintenance-worker[106314]: ERROR futurist.periodics